What the health!? Implementing health probes for highly available, self-healing, global services
Health probes are essential to running a highly available service, yet they are surprisingly tricky to implement without inadvertently making your uptime worse . Even popular frameworks like Spring Boot (until recently 1 ) used unfortunate defaults that may accidentally encourage a hurried developer to fall into some surprising traps that reduce their services' uptime. Over time, my team has come up with a set of rules to avoid these traps . I'll lay them out first succinctly, and then explain them in detail along with the model we use that embodies them. Don't overload health endpoints – favor more over reuse Keep logic out of monitors – put in endpoints Be very conservative when determining if un healthy Run health checks in background – not on demand If your in a hurry , you can skip straight to the summary . This post may use Kubernetes terms like "container" and "probe", but these ideas can be applied to services running on any ...