Skip to main content

Search Your Feelings

Part Deux for yeux and yeux and yeux! With Cribl 4.17, Search gets a major revamp, allowing for direct ingestion and storage of data. This means you can onboard data in just minutes, without needing to set up separate ingestion pipelines or storage layers. We also sprinkled some AI in there, but let's start with what drives this data car: the engine!

Start your engines!
  1. Click Explore in the Cribl Search tile
  2. On the left-hand side, click Data
  3. Click Create an Engine to Start
  4. Fill out the input ID with sbx_engine
  5. Select XSmall for the Engine Size
  6. Click Save

Welcome to the new "Get Data In" experience! Instead of sending your data to Stream and then to Lake and finally accessing via Search, you can now simply point your data sources at Search. That's it. Just one step. No more fiddling with multiple products. Granted, we did provision an engine just now (which will process the incoming data), but you didn't need to do the heavy lifting of setting up Stream and a pipeline and a dataset in Lake.

Now, I don't have a ready-made data source for you to point at Search, so I can only show you what to click and where, but let's do that at least (If you have the itch to test it out yourself, you can try it now with your free-tier Cribl.Cloud account. Don't worry, I'll wait here for you to come back).

Pretend to Point your data at Search!
  1. At the top, click Get Data In
  2. At the top right, click Add Source
  3. Click Syslog
  4. Fill out sbx_syslog for the ID
  5. Click Save

Now Search is ready and listening for syslog data on the default port of 9514 (TCP). You can just copy that handy URL at the bottom of the syslog box and paste it into your senders, then sit forward and Search. Although, there are more buttons. So maybe we should check them out.

Check them out!

Click 2 Datatyping in the top middle

This is where the AI comes in. Search will automatically analyze your data and suggest fields to extract and index, so you can start searching on them right away. OR if you like to touch everything yourself, you can add a rule that specifically types your data either as a pre-defined datatype (like pan_traffic_syslog) or you can create your own custom datatype (which we needed to do first, so you'll have to find that part on your own later). Neat!

Where does it go?

Click 3 Datasets up top (not to be confused with Datasets a little higher up)

Last, but not least (definitely not least, it's actually pretty important), is where your data goes once it's in Search. Datasets are where you define where to store your data and how long to keep it. You can have multiple datasets for different types of data, and you can even have rules that route data to specific datasets based on its content. This is where the magic happens for making your data searchable and performant. Right now you just have the main dataset because your lakehouse engine is still provisioning, but once it comes online, you'll be able to create more datasets and route data to them as needed.

Actually, let's go try to create a dataset now, since the engine is probably ready by now. If not (you can tell by clicking Engines up top and checking the Statuscolumn), you can go watch On Release Days We Wear Teal and come back in a few minutes.

Try it out!
  1. Click Datasets up top (top top, not 3 Datasets)
  2. Click Add Dataset at the top right
  3. Fill out sbx_dataset for the ID
  4. Stop to notice the retention options (which goes up to 10 years)
  5. Click Save

Now we can go back to Get Data In and add a rule to route our syslog data to the new dataset we just created. But you're probably a busy person, so I'll let you explore that on your own. Just know that Search is now an end-to-end solution for ingesting, storing, and analyzing your data, all in one place. No more need to set up separate products or pipelines. Just point, click, and Search!

Let's move on to what's new with Cribl Stream.