Explain feature store design and why train/serve skew happens.
Quick Answer
A feature store centralizes feature computation and storage so the same feature logic is used for both training (batch, historical) and serving (real-time, low-latency); train/serve skew happens when those two paths compute a "same-named" feature differently, so the model sees different inputs in production than it learned from.
Detailed Answer
Without a feature store, teams typically write feature logic twice: once in a batch/offline pipeline (e.g., Spark/SQL) to build training datasets, and once in an online service (e.g., application code) to compute features at request time for real-time predictions. Any divergence between those two implementations — different time windows, different null-handling, different rounding, a bug fixed in one but not the other — produces train/serve skew: the model performs well offline but degrades in production because it's effectively seeing out-of-distribution inputs.
A feature store addresses this by defining feature transformation logic once and materializing it to two stores: an offline store (historical, point-in-time correct, used for training) and an online store (low-latency key-value lookup, used for serving), both fed from the same feature definitions. Point-in-time correctness matters specifically to avoid label leakage — when building training data, you must look up feature values as they existed at the time of the historical event, not the current value. Good feature stores also version feature definitions and track lineage, so you can tell exactly which feature version a given model was trained against.
Code Example
# Feature definition used for both offline training and online serving
@feature_view(entities=["user_id"], ttl="7d")
def user_purchase_features(user_id: str, as_of: datetime) -> dict:
return {
"purchases_7d": count_purchases(user_id, window="7d", as_of=as_of),
"avg_order_value_30d": avg_order_value(user_id, window="30d", as_of=as_of),
}
# Offline: materialized in batch for training set construction (point-in-time joins)
# Online: same definition, precomputed/cached for millisecond lookups at inference timeInterview Tip
Mention point-in-time correctness explicitly — it's the detail that separates a surface-level answer from one that shows real feature-store experience.