Skip to main content

Get Some Data In

Let’s begin by creating a Source to get data flowing into Stream. In our future environment, we will have a myriad of Sources flowing to a multitude of Destinations. The journey of a thousand logs begins with the first Source.

Create a Source
  1. From your Cribl.Cloud homepage, click Manage next to Stream.
  2. Click into the default Worker Group.
  3. Select the Data submenu and click Sources.
  4. Under System and Internal Sources, click Datagen.
  5. On the top right, click Add Source.
note

Once you click the Sources page, you can immediately start typing to find the Source you need. This makes it easier to navigate the mass of Sources supported in Stream.

Normally, you would configure a real-life data Source here. However, since you are looking to learn how to use Cribl, we're going to go ahead and configure a Datagen. These can be useful for testing purposes where you want to build out Pipelines and Routes prior to actually plumbing up the real Sources and Destinations.

Fill out the new Source
  1. In the Input ID, enter sbx_paloalto
  2. In Datagen section, for Data Generator File, select palo_alto_traffic.log.
  3. Click Save.

Your settings should now look like this (click to expand the picture below): Course Map - Syslog Settings

We have now told Stream to imitate a Palo Alto firewall sending syslog traffic. We can see the data being generated to double check our work... After we Commit & Deploy.

Commit & Deploy
Production Networks Beware!

Hey, if this is a production Cribl.Cloud account (which you probably shouldn't be on for sandboxes) make sure you are not about to deploy changes that aren't ready to be deployed.

  1. In the top right, click Commit.
  2. In the resulting modal, click Commit & Deploy at the bottom right.

Since we are using Cribl.Cloud, our Stream instance is in Distributed mode. That means all configuration changes are deployed from the Leader once they have been "git committed". If we tried to get live data prior to a C&D, nothing would show up. Let's try now.

important

In the Status column at right, click Live .

Where is the Data?

It can take up to a minute to deploy changes to cloud Workers. If at first you get an error, try, try again.

You see all the events as they are "sent from our firewall and hit Stream" (using the power of imagination!). However, this data doesn’t go anywhere, yet. Let’s configure our Destination!