Skip to main content

Working with JSON Arrays and the Event Breaking Function

JSON is a very popular event format. It is lightweight, can encapsulate a large amount of information easily, and is almost universally understood by SIEMs and observability tools. Sometimes you'll work with JSON data that is in an array of records or events. Separating these records into multiple outputs can make processing easier in your destination systems of analysis.

JSON Array Event Breaker​

First, we'll work with data originating from a JSON REST API, returned in an array format. You can see more about configuring the REST Collector in our REST API Collector Sandbox.

important
  1. Close any modals, if open.
  2. Navigate to Data > Sources.
  3. Click the REST Collector Source tile.
  4. Click the ► Run button beside the configured json REST Collector.
  5. Click the blue Run button to start the Collection job.

Observe the single event, containing 10 separate items. You'll want to separate each of these items into its own individual event. To do this, you'll configure a custom Event Breaker.

important
  1. Close any modals, if open.
  2. Navigate to Processing > Knowledge.
  3. In the left nav, select Event Breaker Rules.
  4. Click the blue Add Ruleset button in the top right.
  5. In the ID field, enter REST JSON Array.
  6. Click the Add Rule button.
  7. In the Rule Name field, enter Array.
  8. From the Event Breaker Type drop-down, select JSON Array.
  9. Click the blue OK button at bottom right.
  10. Click the blue Yes button to acknowledge the performance warning dialog.
  11. Click the blue Save button at bottom right.
  12. Navigate to Data > Sources.
  13. Select the REST Collectors Source tile.
  14. Click the configured rest_json_array REST Collector to edit its configuration.
  15. From the left tabs, select Event Breakers.
  16. Click the Add Ruleset button.
  17. From the Event Breaker Type drop-down, select REST JSON Array.
  18. Click the ► Save & Run button at bottom left.
  19. Click the blue Run button.

Now your data looks correct! All the items were automatically separated into individual events.

Next, let's see what happens when we can't apply an Event Breaker to a data Source.

The Event Breaker Function​

Maybe your data has already been received by Cribl, and it's in a JSON array format. Maybe your using an HTTP-based Source that doesn't permit event breaking. How can you split arrays into multiple events?

One way of accomplishing this is by using Cribl's Event Breaker Function in a Pipeline. In this example, you'll take Amazon Web Services CloudTrail logs and break them apart into separate events. The sample file imported for you contains data in a single event. This is not what we wanted.

important
  1. Close any modals, if open.
  2. Navigate to Processing > Pipelines.
  3. Click the blue Add Pipeline button and, from the drop-down, select Create Pipeline.
  4. In the ID field, enter cloudtrail.
  5. Click the blue Save button.
  6. In the right Sample Data pane, select the cloudtrail.log file.
  7. Click the Add Function button, then search for and select the Event Breaker Function.
  8. In the new Event Breaker Function's Existing or new? drop-down, select Create New.
  9. From the Event Breaker Type drop-down, select JSON Array.
  10. In the Array Field, enter Records.
  11. Click the blue Save button.

Observe that you now have 11 events. This is what we wanted! The Event Breaker Function is a powerful tool to use when you can't apply event breaking directly on a Cribl Stream/Edge Source.

Conclusion​

In this module, you learned how to work with JSON Arrays with Sources and inside Pipelines. In the next module, you'll explore working with events that have field names in file headers.