Log File Source

Finally, one of the things that really makes Edge unique is log file autodiscovery - Edge is aware of every file opened for writing on its host system, and can monitor any or all of them. Let's take a look.

Autodiscovery Configuration

1. On the top tabs bar, click the Cribl tab to get back to our Edge UI.
2. From Edge's own top nav, click Manage. On the resulting page, click on default_fleet.
3. Click to open the More submenu, then select Sources.
4. From the resulting Manage Sources page, find and click on the File Monitor tile.
5. On the Manage Sources / File Monitor page, click on the in_file_auto Source to open its configuration modal.

On the modal's right side, you should now see something similar to this:

There are a couple things in this config modal that bear explaining. First, the Discovery mode toggle buttons. If you set these to Manual, you have to specify every file you want to monitor. But in the Auto mode, Edge finds every file that's open for writing on its host system, and you set a Filename allowlist of the file patterns you want to include. This list contains a set of wildcard strings.

In this config, it's set to look for anything that does not end in .journal (there's a separate Source for that), anything in a directory named log anywhere on the system, and any file that ends in log. This Source is already in use internally (which is how we earlier saw the files in the Explore Edge > Files section), but its data is not being sent anywhere. Now, we're going to take a look at the "manual" File Monitor Source that is set up by default.

Manual File Monitor Mode

1. Click on the X or the Cancel button to close the in_file_auto Source's config modal.
2. Back on the Manage Sources / File Monitor page, click the in_file_varlog Source below in_file_auto. This will open this Source's config modal.

You should see something like this:

Here, the discovery mode is set to Manual, which means we specify the path to search, in addition to setting a Filename allowlist. In this config, the Search path is set to /var/log, with an unlimited Max depth setting.

The Max depth setting determines how many directory "levels" the Source will traverse to find files that match the Filename allowlist. For example, consider the following excerpt from a typical Ubuntu /var/log:

if we have Max depth set to 1, with an allowlist of !\*.journal\*, !\*.gz, \*log, the Source will find only things that end in log in the top level of the directory - highlighted in green below:

By changing the Max depth to 2 (or unlimited in this case, since our tree has only 1 additional level below the top-level directory), you'll see it pick up more files:

Now, let's enable both of these File Monitors. They're both set up as QuickConnect Sources, so they're disabled until we connect them to a Destination. Let's do that.

Routing Log Events

1. Click on the X or the Cancel button to close the in_file_auto config modal.
2. Click Edge's Collect submenu. This will restore the QuickConnect UI.
3. Above the File Monitor Source's left tile, click the number 2 to expand the two File Monitor Sources.
4. From the in_file_auto Source, click the + button, drag a connector over to the Default tile, and drop it onto that tile.
5. In the resulting Connection Configuration modal, click Save to confirm that you want to configure the connection as Passthru.
6. From the in_file_varlog Source, click on the + button, drag a connector over to the Default tile, and drop it onto that tile.
7. In the resulting Connection Configuration modal, click Save to confirm that you want to configure the connection as Passthru.
8. Click the Commit & Deploy (or Commit) button at Edge's upper right.
9. In the resulting confirmation modal, enter a Commit message (if needed), then click on Commit and Deploy.

Let's see what the new File Monitor is seeing.

Checking File Monitor Status

1. Hover the mouse over the in_file_varlog source tile. From the resulting options, click on Configure to open this Source's configuration drawer.
2. Click on the drawer's Status tab. Expand either of the Hosts that you see there, and you should see a list of the files that Edge is monitoring. (If you don't see a list of files show up, the changes likely haven't finished deploying – wait a minute or so, and check again.)
3. You can also look at the Logs tab, where you should see an accordion for each file in /var/log. Expand a few accordions to fine add file monitor or start monitoring file entries.

That's everything for now! Let's finish things up.