Skip to main content

Splash in Cribl Stream

TL;DR

Version 4.6 brings a lot of quality of life (QoL) improvements to our product suite. These improvements are designed to enable customers to manage larger and more complex deployments with ease. These enhancements come in the form of minor toggles that have a significant impact on the overall functionality and usability of our product.

Version 4.6 introduces a wide range of quality of life (QoL) improvements to our product suite. These enhancements are specifically aimed at empowering customers to effortlessly handle larger and more intricate deployments. By incorporating minor toggles, we have ensured that these improvements have a substantial impact on the overall functionality and usability of our product. With version 4.6, customers can now effectively manage the challenges posed by scaling and complexity, allowing them to focus on their core business objectives.

Since the enhancements in Edge are also in Stream, we'll show them simply by navigating through Stream and when one of them overlaps, we'll call it out.

Take a Splash in Cribl Stream
  1. From the Cribl.Cloud homepage, click the manage-button button for Stream
  2. Click into the default Worker Group
  3. With the Manage tab active, click into Data > Sources
  4. Click into the Syslog Source tile
  5. Click into the in_syslog Source
  6. In the resulting pop-up, click Advanced Settings on the left-hand side
  7. Ogle the Enable TCP load balancing toggle

TCP Syslog Load Balancing​

🎶One toggle is all it takes. Falling in...🎶 nevermind. Anyway, what does this magic toggle do?

When enabled, Cribl will fork and launch 1 new special process which acts as an incoming syslog “load balancer” for all TCP syslog data. This new special “load balancer” process accepts all incoming syslog TCP traffic, splits it up without doing any parsing, and round robins the split messages to each regular Worker Process on the Node. This can be useful when you have a large number of TCP syslog connections and you want to spread the load across multiple Worker Processes.

Netflow v5 Source Tile​

Let the Net...flow
  1. If still open, close out of the in_syslog Source by clicking Cancel at the bottom right
  2. On the left-hand side of the Stream UI, find and click into the Netflow Source
  3. Click Add Source in the top right

I mean with Netflow IYKYK, am I right? But for those who don't know, Netflow is a network protocol developed by Cisco for collecting IP traffic information and monitoring network traffic. With the Netflow v5 Source, you can now natively collect Netflow v5 data in both Stream and Edge. You can read more about our Netflow implementation on our illustrious docs site.

State Tracking & Checkpointing​

Usually for writing classes they say, "Show, don't tell." But for this feature, we're going to tell you about it. Why? Because it's a toggle. And toggles are boring to show. But they're also super useful. So here we go.

REST Collector State Tracking​

When enabled, the REST Collector will track the state of the data it has collected and will checkpoint that state. This means that REST Collection jobs will pick up where they left off in the case of a Leader failure / interrupt. This is especially useful when you have a REST API that doesn't support pagination or when you want to ensure that you don't miss any data.

Users can also track state between REST collections; allowing them to say, “Give me everything new since my last collection” and then go do that collection since Cribl doesn't support voice input yet.

Where is this?

State Tracking is visible when running the collection job. It doesn't show in the Source configuration.

S3 Checkpointing​

Same sh... stuff, different Source. When enabled, the S3 Source will track the state of the data it has collected and will checkpoint that state. This means that S3 Collection jobs will pick up where they left off if a Worker Process goes down. Which I know feels like we're down in the weeds, but it's actually super useful for ensuring you don't miss any data. And at Cribl, that's our goal: to unlock the value of all your data, not just most of it.

Close the Modal!

Before we move on, let's close the Add Source modal by clicking Cancel at the bottom right. Thanks!

OK, that's it for Stream & Edge. Let's head over to Search to see what's new there. Hint: it's more QoL improvements! 🎉