The Playbook Metrics feature allows our customers to gain important insights into their Playbooks. Security engineers can use this vital information to tweak the Playbooks in order to get maximum optimal performance.
Summary is to minimize the time that an analyst needs to get decisions when handling a case. Dashboards are to get general information about the quality of your playbooks.
The following places in the Siemplify Platform can provide you with greater visibility into the Playbooks execution:
- Playbook Monitoring: The Monitoring feature allows customers to use automation to its full capacity. This interface is displayed for each individual Playbook.
- Playbook Summary: The Summary feature is to minimize the time that an analyst needs to get decisions when handling a case. This interface is displayed for each running Playbook on the Cases screen.
The Monitoring screen contains the following information:
- Runs: How many times the Playbook/Block ran during the defined time period. Thousands will be represented by a K. Millions will be represented by an M.
- Redundant: Number of times the Playbook/Block didn’t run in the predefined time period (because it exceeded the maximum number of playbooks (3) that can be automatically added to an alert). If the number is larger than 1 – this could be a good indication to tweak the Playbook – maybe by using Blocks or other logical steps
- Closed Alerts: Percentage of alerts that were closed by this Playbook.
- Average Run Time: Average amount of time that this Playbook took to run. This statistic can prove useful in identifying identify weak points in playbooks – manual actions, frequently-errored steps etc.
- Playbook Status Pie Chart: Shows three options. Options are finished successfully, failed, or waiting for user action. This chart shows you playbook statuses according to the defined time period and is cumulative. Each option is clickable and will take you to a Search results page displaying the cases that this playbook with the specific status was attached to.
- Playbook Trends Line Chart: Shows completed runs, failed runs and a total of runs (both failed and successful). Hover your mouse over each dot on the line to see a pop-up showing more information. This chart can come in useful if a new playbook that you recently created is running as you’ve expected, or if an existing playbook that you recently improved was actually improved as you’ve expected or if more enhancements are needed in order to meet your expectations. For example, let’s say you see that the Playbook didn’t run twenty times over the last month, you might then tweak the trigger logic to make the Playbook more selective. You could then look at the Trends chart to check that the Playbook ran successfully from that time onwards.
- Environments Bar Chart: Displays all the environments that this Playbook ran in. Each section is clickable and will take you to a Search results page.
In addition, hovering over the Actions will display popup showing success/failure rates of that Action during a Playbook run, and hovering over a Conditions branch will display a popup showing how many times that branch was selected.
When clicking on a Playbook, the Context Details appears as a Playbook Summary. This shows the following information:
- Playbook Name and Status
- Waiting for User Input: If the Playbook is waiting for the security engineer to do something, this will be displayed prominently at the top of the Playbook Summary. In addition, a Push notification will be sent to the relevant user letting them know that the Playbook is waiting for them.
- Time and Length of Playbook Run
- Integrations: list of Integrations being used by this Playbook. When clicking on an integration, the specific step will be marked in the playbook viewer so that the analyst can easily find the step that they wants to focus on.
- Playbook Flow: each step that was run with its status and step result.
- Errors: any errors will be listed here. If an error caused the playbook to stop it will be highlighted at the top of the summary, but if it was skipped , it will be at the bottom. Each error is clickable and will direct you to the Kibana logs page. You can also choose to rerun the Action or Playbook from here.
Need more help with this?
Click here to open a Support ticket