Breaking Multi-line Events

Sometimes event boundaries aren't clean. These unclear boundaries can create challenges when you're trying to turn a data stream into discrete events.

Luckily, regular expressions can help here. Below, you will apply what you assimilated earlier about how regular expressions work in an Event Breaker.

There is a fun ASCII art event being played on the Raw TCP Source. Let's go make it work correctly.

important

1. Close any modals, if open.
2. Navigate to Data > Sources.
3. Click the TCP Source tile.
4. In the in_tcp Source's Status column, click the Live button.

Observe that the ASCII art is being mistakenly broken up into each line, instead of into a single event. Now you'll fix this with a custom Event Breaker.

important

1. If the Sample Data preview window is open from the previous section, close it. (Click the X at the top right corner, or click the Cancel button at the bottom.)
2. Navigate to Processing > Knowledge.
3. In the left nav, select Event Breaker Rules.
4. Click the blue Add Ruleset button.
5. In the ID field, enter criblania.
6. Click the Add Rule button.

The ASCII art begins with a line of dashes (-). You'll need to change the Event Breaker field to match this pattern.

important

1. In the Rule Name field, enter criblania.
2. Copy the following sample event to the Event Breaker Rules builder's Sample Data > In tab. This sample looks slightly different, to not spoil the ASCII art surprise.

----
| this
| event
| is
| long
----
| this
| event
| is
| long

3. The regex below configures the Breaker to break events as follows: on newline characters, if the character preceding the line break is a dash, and the line following the line break begins with a vertical pipe. Change the Event Breaker field to the following:

[\n\r]+(?=-+\n\|)

4. In the Timestamp Format section, select Current Time.
5. Click the Out tab at the top of the Preview window to see that there are now two events instead of 10. Additionally, when you updated the regex, the colored highlighting on the input tab changed.
6. Click the blue OK button in the bottom right corner.
7. Click the blue Yes button to acknowledge the performance warning dialog.
8. Click the blue Save button in the bottom right corner.
9. Navigate to Data > Sources.
10. Click on the TCP Source tile.
11. Click the in_tcp ID to open this Source's configuration modal.
12. From the left tabs, select Event Breakers.
13. Click the Add Ruleset button.
14. From the drop-down, select criblania.
15. Click the blue Save button in the bottom right corner.
16. In the in_tcp Source's Status column, click the Live button.

If everything worked correctly, you should now see the ASCII art as it was intended to be seen! In the _raw field, make sure to click the Show more link to see the entire piece of art.

While this seems like a silly example, it illustrates how you sometimes need to build a custom regex-based Event Breaker to handle events with odd boundaries.

Conclusion

And that's it! You've completed all the hands-on labs for this course. We do hope that you found it useful, and can now build event breakers with ease!

Now, let's wrap up with the course conclusion...