Splunk Alert and Report

In this topic, we will see how to create Splunk alerts. We will also see how to create and share Splunk reports.

Learn how to use Splunk, from beginner basics to advanced techniques, with online video tutorials taught by industry experts. Enroll for Free Splunk Training Demo !

Splunk Alerts overview

In SPLUNK, an alert is a search that runs periodically with a condition evaluated on the search results. When the condition matches, an action is executed (e.g. an email is sent to the administrator or a script is run).

It is possible to configure a variety of alerting scenarios for both the real-time and historical searches. You can have your historical searches run automatically on regular schedules, and you can set up both types of searches so they send emails to the administrator when their results meet specific conditions. You can base these alerts on a wide range of threshold and trend-based scenarios, such as empty shopping carts, brute force firewall attacks, login errors, and server system errors.

There are three types of alerts in Splunk:

• Scheduled alert – an alert based on a historical search that runs periodically in accordance with a set schedule. An example of this type of alert is triggering an alert when the number of 404 errors in any 2 hour interval exceeds 50.
• Per-result alert – an alert based on a real-time search that runs over all time. An example of this type of alert is triggering an alert when a disk full error occurs a host.
• Rolling-window alert – an alert based on real-time search that is set to run within a rolling time window that you define. An example of this type of alert is triggering an alert whenever there are five consecutive failed logins for a user within a 10-minute window.

Know more about Splunk in this Splunk Tutorial 

NOTE – Alerting can be throttled such that alerts do not continuously fire if similar conditions are met repeatedly.

Create an alert

You can create an alert from the most searches you run in Splunk Web. Let us see how to create an alert that will be triggered if the number of search results is greater than 100.

First, we need to run our search:

Next, go to Save As > Alert:

The Save As Alert dialog window opens. We need to define the following parameters:

• Title – the name of the alert.
• Description – the alert description.
• Permissions – select whether the alert will be private or shared with all other users of the app.
• Alert type – select whether you wish to schedule your alert to run when scheduled or in real-time.
• Trigger alert when – set the alarm trigger condition. In our example, we will trigger an alert when the number of search results during 300 days exceeds 100.
• Trigger – select whether you would like to trigger the alarm once or for each result.
• Throttle – select the throttle period during which alerts will not be triggered.
• Triggered actions – select the action that will be performed if the alarm is triggered. We’ve chosen to add an event to the Triggered Alerts page.

And that’s it! If the number of search results during 300 days exceeds 100, an event will be displayed in the Triggered Alerts page.

Create an alert that runs a script

The run a script alert action is officially deprecated. It has been replaced with custom alert actions as a more scalable and robust framework for integrating custom actions.

You can run an alert script when an alert triggers. Select Run a script from the Add Actions menu. Enter the file name of the script that you want to run.

For example, you can configure an alert to run a script that generates a Simple Network Management Protocol (SNMP) trap notification. The script sends the notification to another system such as a Network Systems Management console. You can configure a different alert that runs a script that calls an API, which in turn sends the triggering event to another system.

Note: For security reasons, place all alert scripts in either of the following locations:

• $SPLUNK_HOME/bin/scripts

• $SPLUNK_HOME/etc//bin/scripts

Frequently Asked Splunk Interview Questions & Answers

Splunk Reports overview

In Splunk Enterprise, reports are created whenever you save a search or a pivot for later reuse. After a report is created, there’s a lot you can do with it.

As reports can be created from either side of the Splunk Enterprise fence, we’ve created a manual to isolate all of the functionality related to reports and reporting in one place.

• Manually create and edit reports. Add reports to the Report listing page from either Search or Pivot. Configure a report manually in savedsearches.conf. Convert a dashboard panel to a report. Share your report with others by changing its permissions.
• Accelerate slow-completing reports, either during the report creation process, or at a later point.
• Set up scheduled reports–reports that run at a regular interval and which trigger an alert action (such as the sending of an email with search results) each time they run. Scheduled reports are also used for summary indexing.
• Configure the priority of scheduled reports. Learn how the Report Scheduler manages multiple concurrent reports and learn how to configure your Report Scheduler options.
• Understand how Splunk Enterprise generates PDFs of reports, dashboards, searches, and pivots. Enable non-latin fonts in PDFs. Configure PDF generation via .conf files. Review the exceptions to this functionality.

Create a report

When you create a search or a pivot that you would like to run again or share with others, you can save it as a report. This means that you can create reports from both the Search and the Pivot sides of Splunk Enterprise.

In this section, we will create a report from a search in the Search App. Here are the steps:

Run the search you would like to save as a report:

 

to start creating a report, select Save As > Report:

This opens up a Save As Report window. You need to configure the following parameters:

Title – the name of the report.

Description – the optional description of the report.

Content – if your search is a transforming search that displays results in the form of a table or visualization, use this option to determine whether the report will contain the table, the visualization, or both.

Time Range Picker – choose whether or not the report will include a Time Range Picker. If you choose not to, the report will always run over the same time range.

The Your Report Has Been Created window opens. There are other options in this window: you can configure permissions, add the report to a dashboard, set up the report to run on a schedule, or accelerate the report.

To edit your report, go to Settings > Searches, reports, and alerts. Your report should be listed:

Share a report

By default, any report you save is initially private and only available to you. If your permissions allow it, you can change the permissions that belong to the report when you first save it by clicking Permissions on the Your Report Has Been Created dialog. This takes you to the Edit Permissions dialog.

 

Related Blog: Splunk Alert And Report

Here, depending on your permissions, you have the ability to determine whether a report can be viewed by the users of just one app, or all users in all apps. You furthermore can set read and write permissions by role.

For example, you could make a report “globally” available to everyone that uses your Splunk Enterprise implementation. Or you could narrow the saved search permissions so that only specific roles within the current app can use it. You can also arrange for particular roles or users to have “write” access to the report, enabling them to change its underlying search or pivot, or to update its result display formatting.

You can also define or update permissions for a report by:

• Going to the Reports listing page, clicking Edit, and selecting Permissions.
• Going to the report viewing page (click on the report name on the Report listing page to do this), clicking Edit, and selecting Edit Permissions. (To get to the report viewing page, click on the report name on the Report listing page).
• Navigating to Settings > Searches and reports and clicking Permissions for the report you’d like to edit.