This article describes the results of performance benchmark tests performed using Siemplify 5.6.0 software in October 2021. The tests were conducted using Siemplify cloud (SaaS) deployment in two deployment sizing scenarios:

  • Simple deployment for small and medium customers
  • Advanced deployment for big customers

Benchmarking Process

The performance benchmark process was executed using automated tests that injected and ran 1000 alerts with an automated playbook to completion.

Ingestion Process

The ingestion process used the Demo Connector, which creates Alerts with the following predefined characteristics:

  • Number of Events
  • Number of Entities
  • Alert size

The Alerts were ingested using batch ingestion methods. The benchmarking process was executed using an automated test, which allows for batch incident parsing, mapping, classifying, ingestion, and execution of a specified playbook. The test was measured by the total time it took from the first alert received to closing the cyber case.

Each ingested alert included 5 security events and 10 entities with total 5K size. One playbook was executed per each alert with 10 steps.

Results

Ingestion Results

Alerts per day Time to process a single Alert (in ms)
1,000 197.36
5,000 198.56
10,000 199.01
20,000 199.65
50,000 230.11
60,000 256.23
100,000 381.73
120,000 570.11

Playbook Results

Simple Deployment
The deployment supports up to 30K Playbooks per day with 300K Playbook Actions.

Playbooks per day Time to run a single Playbook (in ms)
1,000 2749.13
5,000 2856.77
10,000 2987.61
20,000 3521.49
30,000 5972.57

Advanced Deployment
The deployment supports up to 65K Playbooks per day with 650K Playbook Actions.

Playbooks per day Time to run a single Playbook (in ms)
40,000 3873.89
50,000 4256.61
60,000 5506.80
65,000 7133.28

NOTE: The data specified in the Performance benchmark were processed without data compression. The results might vary based on many factors, including the Alerts size, system configurations, Playbook complexity, data settings, and the type of actions performed.

Need more help with this?
Click here to open a Support ticket

Thanks for your feedback.