Skip to main content

Timestamp Extraction

Timestamp extraction in Event Breakers allows for correct timestamp recognition inside events and during replay.

You can parse timestamps from events using regular expressions – or by defining the Timestamp field, if you are extracting fields from a JSON event.

Generally, an Event Breaker's autotimestamp functionality can correctly extract the timestamp. So let's look at how to work around certain scenarios where automatic timestamp extraction fails.

Overriding Default Timestamp Extraction Behavior​

Let's consider the following example of an AWS VPC Flow Log:

2 123456789010 eni-abc123de 172.31.16.139 172.31.16.21 20641 22 6 20 4249 1418530010 1418530070 ACCEPT OK
2 123456789010 eni-abc123de 172.31.9.69 172.31.9.12 49761 3389 6 20 4249 1418530010 1418530070 REJECT OK

vpc_flow_wrong_timestamp.png

Notice in the sample, that the second field in both events (123456789010) is the AWS account ID. However, the autotimestamper decided that the 8-digit string was sufficient for a timestamp. Because that timestamp falls outside the default allowed earliest and future timestamps, the event time is set to the Cribl Stream/Edge Worker's current time.

To override this behavior, you can specify the Timestamp Anchor field. The following regex demonstrates this. It tells the autotimestamper to start later in the event, to look for the correct timestamp.

\s(?=\d{10}\s\d{10}\s\w)

This regex says to start after the space between two fields that are each 10 digits long, separated by spaces.

vpc_flow_correct.png

You can see that the correct timestamp is highlighted now.

Defining the Timestamp Format​

Let's consider the following example:

20220301000000 <alert> I am the G.O.A.T.!

The timestamp at the beginning of this line comprises the year, month, day, hour, minute, and seconds, with zeros for padding. Unfortunately, while you can read this, the autotimestamper does not understand this format.

By using time string format variables, we can instruct the Event Breaker to properly extract the event's timestamp.

For this example, you could configure the following Manual Format:

%Y%m%d%H%M%S

Custom Timestamp​

If you want to try configuring a custom timestamp, follow this section:

note

If you're not already on the Event Breaker Rules page:

  1. Navigate to Processing > Knowledge.
  2. Click Event Breaker Rules in the left nav.

Now, open the Rule builder again. 3. Click the blue Add Ruleset button. 4. On the New Ruleset configuration modal, click the Add Rule button. 5. Try this Event Breaker by copying and pasting the sample event above into the input section. Click anywhere in the input pane, outside the field labeled Paste your events here or upload a sample file.

20220301000000 <alert> I am the G.O.A.T.!
  1. Under the Timestamp Settings section, select Manual Format.
  2. Copy and paste the time expression into the Manual Format field:
%Y%m%d%H%M%S
  1. Close all dialogs without saving this Event Breaker. It is not required later in the course.

Cribl uses the D3 Time Format library. The tokens available for extraction can be found at this link.

Conclusion​

In this module, we discussed how to modify an Event Breaker's timestamp configuration. In the next module, you'll begin the hands-on labs to configure Event Breakers in various scenarios.