This is less of a question and more of a solicitation of ideas. We currently have a hybrid Linux environment that spans AWS (multiple accounts), on-prem and, at some point in the near future, Azure.
We are currently using AWS Systems Manager for our patching, but I’m curious how other folks are approaching patching in a similar hybrid environment.
Things we have taken into consideration are:
- Maintenance schedules: Our patching currently runs every night at 1:00am
- Lifecycle stages: We group our assets based on the lifecycle stage (dev, qa, stage, prod) and create patch baselines that require patches to reach a certain age before being deployed. This helps ensure that patches are tested in lower environments before getting into prod.
- Clustered systems/Application concerns: Some of our applications are either clustered or must be restarted via some sort of manual process in order to ensure that multiple cluster hosts are not restarting simultaneously or the application fully launches after restart before the next node is restarted.
- Tag-based: We use tagging to determine which lifecycle stage a system is and how to use the appropriate patch baseline.
- SNS: We use AWS SNS subscriptions to notify the appropriate teams when specific systems have been patched and may need to be restarted.
I’m always looking for ideas on how to make our process better, more resilient and more fully automated (where possible). If you would like to share how you handle this sort of thing, I’d love to see how other people handle their Linux patching.