Skip to main content

CIDR Lookup

Before we can configure the Lookup table, we need to extract the correct fields.

Parse the Events​

In our previous challenge, we used the Regex Extract Function. But, because our new data follows a standard structure – in this case, CSV – we can use one of Cribl Stream's most powerful Functions: Parser.

important
  1. With Manage active in Stream's top nav, click through to Processing > Pipelines.
  2. If necessary, refresh the right pane by clicking Sample Data, then clicking Simple next to the new netflow_sample.csv sample file.
  3. Click the Add Pipeline button, then select Create Pipeline.
  4. In the new Pipeline's ID field, enter cidr_lookup.
  5. Click Save.
  6. Click Add Functionon and select Standard > Parser.
  7. Make sure the new Function's Type is set to CSV.`
  8. In the List of Fields, paste: 
    StartTime,Dur,Proto,SrcAddr,Sport,Dir,DstAddr,Dport,State,sTos,dTos,TotPkts,TotBytes,SrcBytes,Label
  9. Your interface should now look like this (click to enlarge): pipelines
  10. Click Save.

The Parser Function has split each event based on the CSV type. To the resulting fields, it has then applied the field names you pasted in.

Create the Lookup Table​

Now we're ready to create the Lookup table.

important

  1. Select the Processing submenu in the top nav and click Knowledge.

    Note: Depending on the size of your window, the top nav will consolidate items that won't fit in a pulldown represented by an ellipsis (...) - if so, click on the ellipsis and then select Processing and click on Knowledge.

  2. If Lookups is not already selected in the left sidebar, click to select it.

  3. At the upper right, click Add Lookup File, then select Create with Text Editor.

  4. Copy this text to the clipboard:

    cidr,team
    192.168.0.0/26,TeamA
    192.168.0.64/26,TeamB
    192.168.0.128/26,TeamC
    192.168.0.192/26,TeamD

    This Lookup table lists the four subnets and each subnet's associated team.

  5. Paste the clipboard contents into large text field.

  6. Type or paste cidr_lookup.csv into the Filename field. Your interface should now look like this (click to enlarge): pipelines

  7. Click Save.

Configure the Lookup​

Now, we are ready to configure the Lookup Function.

important
  1. Select the Processing submenu in Stream's top nav and click Pipelines.
  2. Choose the cidr_lookup Pipeline from the list of Pipelines on the left.
  3. As before, in the right pane, click Simple next to the netflow_sample.csv sample.
  4. To add the Lookup Function to the Pipeline, click Add Functionon, then Standard, then Lookup.
  5. Configure the Function to match this screenshot (click to enlarge): Pipelines
  6. Click Save.
  7. In the right Preview pane, toggle between IN and OUT. When selecting OUT, you should see the lookup values added to each event. Pipelines

Send to Correct Index​

Now we can use the value of the new team field to send the data to the correct index. We'll use the same Eval method we used in the last scenario.

important
  1. Make sure Stream's left pane still displays the cidr_lookup Pipeline.
  2. In the right pane, make sure the Preview Simple tab still has focus, displaying events from your netflow_sample.csv sample.
  3. To add the Eval Function, click Add Functionon, then Standard, then Eval.
  4. Scroll down and click into your new Eval Function.
  5. Configure the Function to match this screenshot (click to enlarge): Pipelines
  6. Click Save.
  7. In the right Preview pane, toggle between IN and OUT. When selecting OUT, you should see the new index values added to each event.

Note that we're setting the index value to team_src. Since team_src is not in quotes, it is not a literal – it refers to the value of the team_src field for each event. (If team_src had single quotes around it, like 'team_src', it would set each index value to the literal string 'team_src'.)

Drop Non-Team Events​

To complete this scenario, our customer wanted to filter out events that were not sent by any of their teams. For this, we used the Drop Function to filter out events without the team_src field.

important
  1. Make sure Stream's left pane still displays the cidr_lookup Pipeline.
  2. As before, in the right pane, make sure the Preview Simple tab still has focus, displaying events from your netflow_sample.csv sample.
  3. To add the Drop Function, click Add Functionon, then Standard, then Drop.
  4. Scroll down and click into your new Drop Function's Filter field.
  5. As shown below (click to enlarge), replace the Filter field's default true value with: !team_src Pipelines
  6. Click Save.

Note that we've used the simple JavaScript filter !team_src to filter out events that do not contain a team_src field, meaning that these events were not sent by one of the four enumerated teams.