What exactly is a SavedModel, and how does TensorFlow Serving use it to run your model without your Python code?
Quick Answer
SavedModel is a self-contained directory: the serialized graph (MetaGraph with signatures), weights (variables/), and assets. TF Serving loads it directly and exposes the named signatures over gRPC/REST — no Python, no model class, no custom code.
Detailed Answer
A SavedModel contains saved_model.pb (the graph program including preprocessing ops that were traced), variables/ (checkpointed weights), assets/ (e.g., vocab files), and optionally a fingerprint. The key concept is signatures: named entry points (default 'serving_default') with typed tensor inputs/outputs, produced from @tf.function-traced functions. Because the graph is the program, TF Serving — a C++ binary — can execute it with zero Python dependencies. That's also why anything not traceable into the graph (Python-side feature fetching, dynamic control flow on Python objects) silently won't ship: it must either become TF ops inside the exported function or move to a pre-processing service. Serving conventions: models live at /models/<name>/<version>/ and the server hot-loads new integer versions, enabling atomic rollout/rollback by directory manipulation. Keras 3's model.export() (vs .save('.keras') for training checkpoints) is the current API for producing inference SavedModels.
Code Example
model.export('exported/my_model/1') # Keras 3 inference export
docker run -p 8501:8501 \
-v $PWD/exported/my_model:/models/my_model \
-e MODEL_NAME=my_model tensorflow/serving
curl -d '{"instances": [[...]]}' localhost:8501/v1/models/my_model:predictInterview Tip
Distinguish .keras (training checkpoint, needs Python) from exported SavedModel (deployment artifact, needs nothing) — Keras 3 made this split explicit and many candidates still conflate them.