You are correct, the final image of your deployment is not rebuilt from scratch every time, instead it relies on a common base image to build on top of.
That being said, multiple things happen when you click on the "Deploy" button :
- first, an updated image of your API service is built,
- then, this image is pushed to the container registry configured for your Infrastructure,
- finally, the Kubernetes deployment itself is updated, meaning that the updated image is pulled from the container registry and the pods of your deployments are spawned. Depending on the number of replicas in your Kubernetes deployment, this can take some time.
All in all, the build step is probably not where the bottleneck is, you may be able to pinpoint it more precisely by looking at the timestamps of each step in the backend logs.
Hope this helps.