Skip to main content

Event Breaking

In this module, we'll configure a custom Event Breaker to automatically unroll the JSON arrays into individual events. This is useful when you're obtaining data from REST endpoints and forwarding individual events through Cribl Stream's Routes and Pipelines.

note

Run the following command in your sandbox terminal:

curl 'http://rest-server/collect/array?size=25'

You'll see a result similar to the following:

[{"item":1},{"item":2},{"item":3},{"item":4},{"item":5},{"item":6},{"item":7},{"item":8},{"item":9},{"item":10},{"item":11},{"item":12},{"item":13},{"item":14},{"item":15},{"item":16},{"item":17},{"item":18},{"item":19},{"item":20},{"item":21},{"item":22},{"item":23},{"item":24},{"item":25}]

But ideally, you'd want to forward individual events, in this cleaner format:

{"item":1}
{"item":2}
{"item":3}
{"item":4}
{"item":5}
...

Configure the Cribl Stream REST Collector​

important

If necessary, navigate back to the REST Collector Source page. From the top nav of your Cribl Stream Sandbox, select Manage > Data > Sources, then select Collectors > REST from the Data Sources page's tiles. Click Add Collector to open the REST > Add Collector modal, which provides the following options and fields.

  1. In the Collector ID field, enter array_event_breaker.

  2. Copy the following URL to the Collect URL field.

    'http://rest-server/collect/array'

  3. In the Collect parameters section, click + Add parameter and fill out the following:

    • In the Name box, enter size.
    • In the Value box, enter 25.

    This will add a query string parameter ?size=25 to the Collect URL.

  4. At the bottom left, click ► Save & Run. In the Run configuration modal, click Run again.

The Preview modal should display a single event, containing an array of 25 individual items.

Configure the Custom Event Breaker​

Now you'll configure a custom Event Breaker, to automatically unroll the array of items into individual events.

important
  1. Close the Preview modal (if open).
  2. From the top nav of your Cribl Stream Sandbox tab, select Manage > Processing > Knowledge.
  3. From the left tabs, select Event Breaker Rules.
  4. On the Event Breaker Rulesets page, click New Ruleset to create a new Event Breaker ruleset.
  5. In the ID field, enter REST API Array Breaker.
  6. Click + Add Rule.
  7. In the Rule Name field, enter Array.
  8. In the Event Breaker Type drop-down, select JSON Array.
  9. Try out this Event Breaker by copying and pasting the JSON array into the input section. Click anywhere in the input pane, outside the box labeled Paste your events here or upload a sample file.
[{"item":1},{"item":2},{"item":3},{"item":4},{"item":5},{"item":6},{"item":7},{"item":8},{"item":9},{"item":10},{"item":11},{"item":12},{"item":13},{"item":14},{"item":15},{"item":16},{"item":17},{"item":18},{"item":19},{"item":20},{"item":21},{"item":22},{"item":23},{"item":24},{"item":25}]

Notice how the blue and red color-coding illustrates the event breaking.

Verify and Save the Event Breaker​

important
  1. Click the Out tab at the top to view the array displaying 25 events with individual objects.
  2. At the bottom right, click OK to exit.
  3. In the warning dialog, click Yes to ignore the warning that recommends using a Filter Condition other than the default true. (We are applying the Event Breaker only to the REST Collector, so here, the default is fine.)
  4. At the bottom, right, click Save.

Apply the Event Breaker to Data Collection​

Now let's apply the Event Breaker to the array_event_breaker Collector configuration.

important
  1. Navigate back to the REST Collector Source page. (From the top nav of your Cribl Stream Sandbox, select Manage > Data > Sources, then from the Data Sources page's tiles or left nav, select Collectors > REST.)
  2. On the Manage REST Collectors page, click the array_event_breaker Collector row to open its configuration.
  3. From the config modal's left Result Settings section, select the Event Breakers tab.
  4. Click + Add ruleset and, from the drop-down, select REST API Array Breaker. You may need to scroll to the bottom of the list.
  5. At the bottom left, click ► Save & Run. In the Run configuration modal, click Run again.

The Preview modal should display the expected 25 events.

Conclusion​

You've now configured a simple Event Breaker to automatically unroll JSON Array responses from REST API endpoints!

If you want to try the custom Event Breaker with the paginated endpoints, the only modification required is to add items to the Array Field in the Event Breaker rule.

In the next module, we'll explore how to collect data from a REST API endpoint on a schedule.