Loading...
TensorFlow
Everything for TensorFlow in one place — pick a section below. 16 reviewed items across 4 content types.
📝Interview Questions7Q&A with detailed answers and code examples→
🏭Use Cases3Real-world production patterns and scenarios→
🔥Production Issues3On-call incidents with root causes and fixes→
⭐Best Practices3Security, scaling, and day-2 operations→
7 / 7
⚡
Quick Answer
TF2 runs eagerly by default — ops execute immediately like NumPy. @tf.function traces the Python function into a static graph that gets optimized (fusion, pruning, parallelism) and reused, giving big speedups but introducing tracing semantics you must understand.
Detailed Answer
Eager mode is imperative: each op runs as Python executes, easy to debug with print/pdb. @tf.function converts the function into a tf.Graph via tracing: the first call executes the Python once, records TF ops into a graph, and later calls run the compiled graph without touching your Python (a 'concrete function' per input signature). Consequences interviewers probe: Python side effects (print, list.append) run only during tracing, not per call; every new input shape/dtype (or new Python-object argument) triggers a retrace — a common performance bug with variable shapes unless you pass input_signature or use dynamic dimensions; and control flow on tensor values compiles to tf.cond/tf.while via AutoGraph. Graph mode is what enables serialization to SavedModel and deployment to TF Serving/TFLite, so production paths are effectively always graph mode.
Code Example
@tf.function(input_signature=[tf.TensorSpec([None, 224, 224, 3], tf.float32)])
def serve(x):
return model(x, training=False)
print(serve.pretty_printed_concrete_signatures()) # inspect traces💡
Interview Tip
The retracing trap is the discriminator question. Say 'a new Python argument value causes a retrace, a new tensor value does not' and explain input_signature — that's the difference between having read the docs and having debugged it.
💬 Comments
⚡
Quick Answer
Low GPU utilization almost always means the tf.data pipeline can't feed the GPU. Profile with the TensorBoard profiler, then fix with parallel map (num_parallel_calls=AUTOTUNE), prefetch(AUTOTUNE), caching, and moving heavy Python work out of the hot path.
Detailed Answer
Diagnose first: the TensorBoard profiler's input-pipeline analyzer shows exactly how long the model waits on input each step. Common causes, in the order I check: (1) sequential map calls doing Python-level decode/augment — set num_parallel_calls=tf.data.AUTOTUNE and vectorize where possible; (2) missing prefetch(AUTOTUNE) at the end, so CPU prep and GPU compute alternate instead of overlap; (3) re-doing expensive work each epoch that .cache() (RAM or file) could amortize; (4) using py_function for augmentation, which is single-threaded through the GIL — replace with native TF ops or preprocess offline; (5) small files on network storage — convert to TFRecord shards with interleave for parallel reads; (6) shuffle buffer too large causing startup stalls, or too small hurting randomness. Order matters too: the canonical chain is interleave(read) -> shuffle -> map(parse, AUTOTUNE) -> batch -> map(augment on batch) -> prefetch.
Code Example
ds = (tf.data.TFRecordDataset(files, num_parallel_reads=tf.data.AUTOTUNE)
.shuffle(50_000)
.map(parse, num_parallel_calls=tf.data.AUTOTUNE)
.batch(256)
.prefetch(tf.data.AUTOTUNE))
# TensorBoard: Profile tab -> input-pipeline analyzer💡
Interview Tip
Name the profiler first — candidates who jump straight to 'add prefetch' without a measurement step read as cargo-cult. Then the AUTOTUNE trio: parallel reads, parallel map, prefetch.
💬 Comments
⚡
Quick Answer
SavedModel is a self-contained directory: the serialized graph (MetaGraph with signatures), weights (variables/), and assets. TF Serving loads it directly and exposes the named signatures over gRPC/REST — no Python, no model class, no custom code.
Detailed Answer
A SavedModel contains saved_model.pb (the graph program including preprocessing ops that were traced), variables/ (checkpointed weights), assets/ (e.g., vocab files), and optionally a fingerprint. The key concept is signatures: named entry points (default 'serving_default') with typed tensor inputs/outputs, produced from @tf.function-traced functions. Because the graph is the program, TF Serving — a C++ binary — can execute it with zero Python dependencies. That's also why anything not traceable into the graph (Python-side feature fetching, dynamic control flow on Python objects) silently won't ship: it must either become TF ops inside the exported function or move to a pre-processing service. Serving conventions: models live at /models/<name>/<version>/ and the server hot-loads new integer versions, enabling atomic rollout/rollback by directory manipulation. Keras 3's model.export() (vs .save('.keras') for training checkpoints) is the current API for producing inference SavedModels.
Code Example
model.export('exported/my_model/1') # Keras 3 inference export
docker run -p 8501:8501 \
-v $PWD/exported/my_model:/models/my_model \
-e MODEL_NAME=my_model tensorflow/serving
curl -d '{"instances": [[...]]}' localhost:8501/v1/models/my_model:predict💡
Interview Tip
Distinguish .keras (training checkpoint, needs Python) from exported SavedModel (deployment artifact, needs nothing) — Keras 3 made this split explicit and many candidates still conflate them.
💬 Comments
⚡
Quick Answer
Enable memory growth per GPU (tf.config.experimental.set_memory_growth) so TF allocates on demand, or set a hard per-process cap with set_logical_device_configuration. The grab-everything default is right when one training job owns the whole GPU — it avoids fragmentation and allocator overhead.
Detailed Answer
By default TF maps nearly all GPU memory at initialization to run its own sub-allocator (BFC) on one big arena — efficient and fragmentation-resistant for a single tenant. It becomes a problem when the GPU is shared: notebooks + a service, multiple inference processes, or TF alongside PyTorch. set_memory_growth(gpu, True) makes TF start small and extend the arena as needed (memory is not returned once grown). set_logical_device_configuration with a memory_limit gives a hard cap, letting you bin-pack N processes deterministically onto one card — usually better than growth for co-located inference services because a leak in one process can't starve the rest. Both must be set before any op initializes CUDA. On the ops side, this pairs with Kubernetes GPU sharing choices (time-slicing/MPS/MIG), where per-process caps keep tenants honest.
Code Example
gpus = tf.config.list_physical_devices('GPU')
for g in gpus:
tf.config.experimental.set_memory_growth(g, True)
# Hard cap: two 20GB tenants on one 48GB card
tf.config.set_logical_device_configuration(
gpus[0], [tf.config.LogicalDeviceConfiguration(memory_limit=20480)])💡
Interview Tip
Most candidates only know 'set memory growth'. Adding when the default is correct (single-tenant training, fragmentation-resistant) and the hard-cap alternative for co-located services shows real operational judgment.
💬 Comments
⚡
Quick Answer
Keras 3 became a standalone multi-backend framework running on TensorFlow, JAX, or PyTorch. For TF-standardized teams: the Keras API stays familiar, but saving formats split (.keras vs exported SavedModel), some tf.* idioms inside custom layers break backend portability, and JAX becomes an easy performance experiment.
Detailed Answer
Keras 3 (the 'keras' package, replacing tf.keras as the default in TF 2.16+) reimplemented Keras on a backend-agnostic ops layer (keras.ops). Practical implications: (1) Model files — .keras is the native format for checkpoints; deployment to TF Serving goes through model.export(), and loading legacy TF SavedModels as Keras models needs keras.layers.TFSMLayer; (2) custom layers written with raw tf.* calls pin you to the TF backend — writing them with keras.ops keeps the door open to switching the backend to JAX (often free speedups on TPU) with an environment variable; (3) migration gotchas are real: behavior differences in some defaults and removed legacy APIs mean pinned versions and a regression suite matter during the 2.15 -> 2.16+ jump; (4) distribution — the Keras distribution API abstracts over tf.distribute and JAX sharding. Teams commonly stay on the TF backend for serving-stack stability while gaining the option value.
Code Example
# Backend selected before import, no code change
# KERAS_BACKEND=jax python train.py
import keras
class MyLayer(keras.layers.Layer):
def call(self, x):
return keras.ops.relu(x) * keras.ops.sqrt(2.0) # portable, not tf.*💡
Interview Tip
The .keras-vs-export split and TFSMLayer are the concrete details that prove you've actually done the 2.16 migration rather than read the announcement.
💬 Comments
⚡
Quick Answer
MirroredStrategy replicates the model across GPUs in one process with all-reduce sync; MultiWorkerMirroredStrategy extends it across machines using the TF_CONFIG environment variable for cluster discovery. Code changes are minimal (same strategy.scope() pattern); the work is in TF_CONFIG orchestration, data sharding, and fault-tolerant checkpointing.
Detailed Answer
Both are synchronous data parallelism. In-process MirroredStrategy needs: create variables under strategy.scope(), scale the batch size by replica count, and use Model.fit or a distributed custom loop. Multi-worker adds infrastructure semantics: every worker runs the same script; TF_CONFIG (JSON: cluster spec + this task's type/index) tells each process who's who; collective ops run over gRPC/NCCL between hosts. Operationally you must (1) shard data — tf.data auto-shards by file with AUTO policy, which requires enough input files, or shard manually per worker; (2) checkpoint only on the chief but to storage all workers can read, using BackupAndRestore for preemption recovery; (3) scale the global batch and LR together; (4) on Kubernetes, this maps cleanly to a Kubeflow TFJob or an indexed Job that renders TF_CONFIG per pod. Failure semantics are all-or-nothing per step (synchronous), so slow/flaky nodes stall the cluster — same class of problem as NCCL timeouts in PyTorch DDP.
Code Example
strategy = tf.distribute.MultiWorkerMirroredStrategy()
with strategy.scope():
model = build_and_compile()
model.fit(ds, epochs=10,
callbacks=[tf.keras.callbacks.BackupAndRestore('gs://ckpt/backup')])
# TF_CONFIG per worker (rendered by the operator):
# {"cluster": {"worker": ["w0:2222", "w1:2222"]}, "task": {"type": "worker", "index": 0}}💡
Interview Tip
Interviewers listen for TF_CONFIG, chief-only checkpointing, and file-based auto-sharding needing enough files — the three things that actually bite when you first go multi-node.
💬 Comments
⚡
Quick Answer
That signature points at queueing — most likely server-side request batching tuned wrong (batch_timeout too high or max_batch_size too big), thread pool saturation, or several models sharing one GPU. Check the batching config, monitoring metrics, and per-model isolation.
Detailed Answer
p50 flat + p99 exploding under load means most requests are fine but some wait in a queue. In TF Serving, work through: (1) Batching config — enable_batching groups requests to boost GPU throughput at the cost of waiting up to batch_timeout_micros; oversized max_batch_size or long timeouts trade tail latency for throughput. Tune max_batch_size down, timeout to ~1-5ms, and set num_batch_threads ~ CPU cores and a bounded max_enqueued_batches so overload fails fast instead of queueing forever. (2) Saturation — check the rest/grpc worker threads and whether TF intra/inter-op parallelism is fighting other processes on the box. (3) Multi-model contention — several models in one server share the GPU and batching queues; hot models deserve their own server/GPU. (4) Version reloads — model hot-swaps at peak can stall requests (there are documented memory/latency spikes on reload); schedule rollouts off-peak. Instrument via TF Serving's Prometheus endpoint (:tensorflow:serving:batching queuing latency histograms) before turning any knob.
Code Example
# batching_parameters.txt
max_batch_size { value: 16 }
batch_timeout_micros { value: 2000 }
num_batch_threads { value: 8 }
max_enqueued_batches { value: 100 }
tensorflow_model_server --enable_batching=true \
--batching_parameters_file=batching_parameters.txt \
--monitoring_config_file=prometheus.conf💡
Interview Tip
Recognize the p50-vs-p99 divergence as a queueing signature out loud — that one sentence of reasoning is worth more than listing every TF Serving flag.
💬 Comments