Observability
In the original postwe defined steps to implement a bullet-proof Cloud Infrastructure Security Posture. Now it’s time to dive into the topic of 🔍 Observability.
❓ How would you approach the implementation of the following?
– Implement Centralized Logging (from cloud services and applications);
– Configure Alerts on Critical and High Impact VMS Events.
💪 Here is how, in a few well-defined steps:
1. Option No.1: **Managed setup**.
1.1. Leverage cloud service provider’s SIEM, e.g. CloudWatch in AWS – https://lnkd.in/ev6ssGz7, Cloud Operations Suite in GCP –https://lnkd.in/eFTaRrQ7, Azure Monitor in MS Azure – https://lnkd.in/eZsZAriE.
1.2. Stream all logs from all services and applications directly to the CSP-provided Centralized Logging Service.
2. Option No.2: **Custom setup**.
2.1. (Optional, but provides a structured approach to log collection). Start with creating a centralized cloud storage for all application and service logs. Create separate folder per application/cloud service, and configure streaming of logs into those folders.
2.2. Install SIEM solution in your cloud environment. For example, ElasticSearch (https://lnkd.in/e-3yvV33) or OpenSearch (https://opensearch.org/) are widely popular.
2.3. Configure streaming of all log records from different folders in centralized cloud storage (or directly from applications and services) into the corresponding indexes in SIEM.
3. In your SIEM of choice (Option No.1, or No.2), take time to review the incoming log records, and for each type of service/application define a set of filters that will help you detect application/service errors or concerning trends.
3.1. Build Dashboards to visualize your filters.
3.2. For critical errors, set up email alerts.
3.3. Deploy Alarms or Monitors that help you detect anomalies in the log records, as well as their corresponding Alerts.
3.4. Test your Dashboards and Alerts to make sure they provide enough data reference points and good insights into data in case of security or operational incidents.
4. Review the Dashboards provided by your XDR (e.g. Wazuh, for example). Over time, fine-tune filters to reduce the number of incoming security events.
5. Configure email Alerts in your XDR on High and Critical impact events.
6. Make rule to visit your SIEM and XDR Dashboards at least 2-3 times a week.
7. Over time, reduce the number of incoming email Alerts to the most significant ones.
Be pro-active with data, filter out noise, prioritize the most impactful events, to achieve the best observability and incident management results.