How do you build alerting for certificate expiry across hundreds of services so nobody gets paged by a surprise expiration?
Quick Answer
Build or use a scanning job that connects to every known TLS endpoint on a schedule, extracts each certificate's expiry date, exposes days-until-expiry as a metric per service, and alert on tiered thresholds, for example a ticket at 30 days, a page at 7 days, and a critical page at 1 to 2 days, rather than a single binary expired or not-expired check. The hard part isn't the expiry check itself; it's maintaining an accurate, complete inventory of every certificate across every service, internal and external, since certificates nobody remembers to add to the inventory are exactly the ones that cause 2am surprises.
Detailed Answer
Imagine a large household with a dozen different subscriptions, insurance, magazine, gym membership, domain name registration, each with its own renewal date, some auto-renewing and some not. A well-organized household keeps one master calendar with every renewal date logged, checked weekly, with reminders set well in advance of each one. A disorganized household relies on each subscription's own notification email arriving on time and someone happening to read it, which works fine until one email lands in spam or a card on file expires, and suddenly a service everyone depends on stops working with no warning at all.
Certificate expiry monitoring at scale requires exactly that master-calendar approach rather than trusting individual reminders, because the TLS layer itself gives you no built-in fleet-wide visibility. A certificate expiring is invisible until a client tries to connect and fails, by which point it's already an outage. This is why mature platform teams build a dedicated inventory and scanning system rather than relying on whatever renewal notification email each CA happens to send, since those emails go to whichever address was used at issuance time, which drifts out of date as teams and ownership change over months and years.
A typical implementation runs a scheduled scanner, a cron job, or a tool like the Prometheus blackbox exporter's TLS probe module, or a dedicated cert-monitoring service, that iterates over a maintained list of every TLS endpoint, internal services like payments-api.internal and checkout-worker.internal, and external-facing domains, connects via TLS, extracts the leaf certificate's notAfter date, computes days-remaining, and exposes it as a labeled metric, something like ssl_cert_expiry_days with a service label. An alerting rule engine, Prometheus Alertmanager or equivalent, then evaluates that metric against tiered thresholds, routing a 30-days-out warning to a ticket or Slack channel, and a sub-7-day warning to an actual page, since at that point the risk of a missed manual renewal turning into an outage is genuinely high.
In production, the actual operational challenge is less about the scanning and alerting mechanics and more about inventory completeness. New services spin up constantly, and unless certificate provisioning is wired into the same automated pipeline that registers a new service into the scanning inventory, for example via infrastructure-as-code that both provisions a certificate and registers the endpoint with the monitoring scanner in the same change, it's extremely easy for a newly-launched service to silently fall outside the safety net until its first, and possibly only, certificate quietly expires months later without anyone noticing.
The non-obvious gotcha: teams often build solid expiry alerting for public-facing HTTPS endpoints but completely miss internal service-to-service certificates, mTLS certificates used inside a service mesh, or internal CA-issued certificates for database connections, because those aren't reachable by a scanner probing from outside the network the way a public website is. The monitoring coverage needs to explicitly include internal-only endpoints and internal CA-issued certificates, not just anything visible from the public internet, or you end up with a false sense of complete coverage that has a large blind spot exactly where outages are hardest to diagnose, deep inside internal infrastructure.