Skip to main content

Navigating Data Configs

Let's take a gander at a how these configurations look in Cribl Search.

Dataset Configurations​

May the Source be with you
  1. On the Cribl.Cloud home page, click the Explore button within the Search pane.
  2. Click Data in the top navigation bar.
  3. Click Datasets in the left navigation bar.

Behold! The land of datasets, where you can edit dataset configurations on current datasets and create new ones. Let's first take a look at an existing dataset.

The Freebie

Click on cribl_search_sample.

The first thing to note here is that the ID field is populated with the name of the dataset. When searching for logs within the dataset this is the field that you will use to refer to the dataset.

Ex.

dataset="cribl_search_sample"

Once the ID is configured it cannot be modified.

important

Ensure Dataset Provider is selected in the left navigation bar.

In the Dataset Provider section of the configurations you can see the dataset provider that this dataset is configured to use. In this case, cribl_search_sample will use the credentials stored in the cribl_s3sample_provider dataset provider to access an Amazon S3bucket.

Next, we see that the Bucket Path contains the path to our data within the Amazon S3 bucket. Furthermore, the Path Filter section allows us to create a Javascript filter (similar to Cribl Stream Functions) to specify exactly which files we would like to search within the configured bucket path. A Path Filter of true is the default and indicates that we want to search all files within the bucket.

Game, (data)Set, Match

If we wanted to attribute differents sets of data to different datasets within the same Amazon S3 bucket (or any other dataset provider type) we could simply create another dataset and configure each dataset's path filter to read different data within the bucket. In this way we can see that datasets can have a one-to-one or a many-to-one relationship with dataset providers.

Dataset Provider Configurations​

important
  1. Click X on the Datasets modal.
  2. Click Dataset Providers in the left navigation bar.

This is where dataset providers are created and/or configured. Let's take a look at the dataset provider that we saw the cribl_search_sample was configured to use.

important

Click cribl_s3sample_provider.

Again, you can see that the name of the dataset provider is contained in the ID field and cannot be changed. Additionally, we can see that the dataset provider type is Amazon S3 (which also cannot be updated). We can see that this provider uses the AssumeRole authentication method to connect to the Amazon S3 bucket.