How to Create Log Threshold Rules in Elastic Observability
In the previous article, we discussed the importance of alerts and how to configure rules to receive notifications for specific Windows events through the Microsoft Teams tool. In this article, we will delve into why monitoring data sources in your organization is crucial and learn how to set up rules that will alert us when the transmission of logs from these sources stops.
Data sources are the lifeblood of any monitoring system. The information they provide is essential for decision-making, security, and incident response. Therefore, ensuring their constant availability becomes an undisputed priority.
By ensuring that these sources are always active and reporting events, you enable the Security Information and Event Management (SIEM) system to continuously collect and analyze data. This is crucial to affirm coverage and, consequently, to detect threats and respond to incidents in real-time.
When we stop receiving data from a source, we run the risk of losing valuable information. For instance, if a security data source ceases to function, we might lose visibility into potential significant events, making it challenging to detect suspicious activities or behaviors.
In this article, we will focus on learning how to configure Log Threshold rules in Elastic Observability. This type of rule is based on the number of logs received from a data source during a specified period.
Before moving on to the practical part, let me give you a brief overview of Elastic Observability. It's a unified observability platform that can provide a comprehensive view of your IT environment. It allows you to collect data from various sources such as applications, infrastructure, networks, and devices, and analyze them to identify issues, resolve them quickly, and enhance the performance and reliability of your systems.
Let's start configuring the rule!
We access Elasticsearch to carry out the necessary configurations.
If you don't have Elasticsearch connected to the “Microsoft Teams” application, I invite you to read this article.
2. Now, let's navigate to “Stack Management,” then “Alerts and Insights,” and click on “Rules.” Next, select “Create rule” to set up the Log Threshold rule.
3. Next, we name our rule “No logs — Windows Server 2022” and add a tag called “Windows 2022” to it.
4. Here, we choose the type of rule we want to establish; in this case, we opt for “Log threshold”.
5.Now, we will proceed to set up a rule that will be triggered when the number of logs generated by the data source “win-ske9jeq7tlg” in the last 20 minutes is less than 5. This rule will allow us to monitor the log flow from this server and generate alerts if the number of logs falls below a defined threshold.
Data Source: Windows Server 2022
Threshold: 5
Duration: 20 minutes
Action: Send an alert to the Microsoft Teams application
Taking this example into consideration, if the Windows Server 2022 is used for applications primarily during weekdays, you might want to configure a higher threshold for non-working days. For instance, you could set a threshold of 10 for non-working days and a threshold of 5 for working days.
6. We choose the connector type, which is “Microsoft Teams,” and select“Infrasecuritycode Teams”.
7. We move to “Message” where we configure the message we would like to appear in the alert notification. We can include information about the data source, the triggered condition, and any other details we consider relevant. You can use the default message.
To add more context to the message, you can click on the icon above the message text box. A list of available variables will appear. Choose the ones you want to include in the message; in our case, we customize the message a bit.
8. Let's test the alert! To do this, let's stop the Elastic Agent service on the server. This will cause the server to stop generating logs, triggering the rule's condition.
9. Let's verify that the alert has been generated! To do this, let's go to Elastic Observability. There, we can see that the alert has been generated. Next, let's go to the “Microsoft Teams” application to confirm that we have received the notification.
Before concluding, I want to share with you some key tips to keep in mind when configuring this type of rule:
Review the historical traffic of the data source to determine the normal traffic range.
Ensure you choose an appropriate duration time for your environment. A too short duration may generate too many false alerts, while a too long duration may delay the detection of real issues.
We have finished with this article. I hope it has been useful for you.
Stay tuned for future contents. Don't miss out!
Hi! I am Kendra Mazara
Senior Information Security Specialist | MBA | Cofounder MujeresTICs RD | Speaker | LinkedIn Learning Instructor
