Course Overview

When feeding IT and Security tools with data, it's often critical to "reshape" the data to put it into the right format for the tool. This can be as simple as changing the JSON structure of the data (unrolling it or flattening it, for example), or removing unwanted fields. Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy, enabling customers to collect, process, route, and analyze all IT and security data.

This course is about reshaping data in the stream, and assumes that you've already gone through the Cribl Stream Overview course. This sandbox has a Cribl Stream instance (which you see at the right) that's already set up with "employee" data coming in via a data generator, and flowing out to a Filesystem Destination. There's an empty Pipeline, named "employee", which is where we'll be focusing.

If you run the below command, you'll see the latest file written to the employee Destination. (You can use the Copy button to easily paste it into the terminal.)

cat $(ls -t /tmp/employee/CriblOut-* | head -n 1)

The format is not pretty – each record contains a _raw field with an embedded JSON list of employees. We're going to use Cribl Stream to break each employee record out into an individual event, and to flatten the structure.

Let's get started!