Skip to main content

Evaluating

In the world of analytics, you can only work with the data that you have right? WRONG! Dead wrong. If there is a field that you don't have in your data, make it up!!

Cribl Search's functions allow you to perform different operations on the data to alter it just about any way that you see fit.

Extend​

With the extend operator in Cribl Search you can create fields and assign them static values or have new fields created based on evaluations of the data. Extend allows you to perform mathmatical operations as well as leverage scalar functions similar to the summarize operator.

important
  1. Clear the query box.
  2. Enter the following query: 
    dataset="cribl_search_sample" dataSource="access_combined"
    | limit 100000
    | extend outcome = iif(status>=300,"SUCCESS", "FAIL")
  3. Click SEARCH.

In this example the extend operator is being used to create a new field called outcome (as indicated by the outcome =). The iif function is being used to set the value of the outcome field based on the status of the web request. If the status is below 300 the outcome is set to SUCCESS, otherwise it is set to FAIL.

Let's take this one step further.

important
  1. Clear the query box.
  2. Enter the following query: 
    dataset="cribl_search_sample" dataSource="access_combined"
    | limit 100000
    | extend outcome = iif(status>=300,"SUCCESS", "FAIL")
  3. Click SEARCH.

Extract​

In some instances, you need to split a field in to several fields or cut out a portion of a field. Nothing is going to help you chop that field into pieces better than the extract operator.

important
  1. Clear the query box.
  2. Enter the following query: 
    dataset="cribl_search_sample" dataSource="access_combined"
    | limit 100000
    | project request
  3. Click SEARCH.

This search provides us with just the request field from the access_combined data source. You'll notice the request is made up of the method along with the url and the scheme. Well, that just won't do now, will it? Let's break that up.

these are the breaks
  1. Clear the query box.
  2. Enter the following query: 
    dataset="cribl_search_sample" dataSource="access_combined"
    | limit 100000
    | project request
    | extract source="request" type=regex @'^(?<method>.*?)\s(?<url>.*?)\s(?<scheme>.*?)$'
  3. Click SEARCH.

Looks like we've added 3 more fields to the party! Let's see how they got their invitation.

The first thing that we define in the extract command is the source this defines what field we want to parse data from. By default, _raw (the entire raw payload) is the source.

Next, we select the type of extraction we'd like to perform. In this case, we are going to use a regex method of parsing the data.

Lastly, we add our regex. The @ symbol denotes we are starting our regex. The regex is then enclosed in ' '. Named capture groups take the data that matches the group and applies the provided field name.

There is another way that we could have done this. It seems all of our fields are separated by a space. In these types of cases, a delimiter would also be a suitable way to extract the data (with much less effort).

Da sky's delimit
  1. Clear the query box.
  2. Enter the following query: 
    dataset="cribl_search_sample" dataSource="access_combined"
    | limit 100000
    | project request
    | extract source="request" type=delim delimchar=" " "method,url,scheme"
  3. Click SEARCH.

Instead of using a type of regex we are using a type of delim. We then specify the character that is being used as the delimiter with the delimchar property which in this chase is a space. Finally, we assign field names in the order that they occur in source using a comma separated list.