Routing Data With Stream

Now, let's get the data to some tools for visualization and analysis. Stream makes this easy, with a simple drag-and-drop interface (QuickConnect) to connect sources to destinations.

We've preconfigured our outputs. We'll add a new AppScope source to send to these destinations.

important

Connect AppScope to Prometheus and Elasticsearch

  1. On the Stream tab, close the Live Data display and AppScope modal. (Click the "X" at upper right.)
  2. Click the "Manage" tab in Stream's top nav.
  3. Click the "Routing" submenu and click "QuickConnect".
  4. Click "+ Add" next to "Sources".
  5. Scroll down and click "+ Add New" on the "AppScope" tile.
  6. In the "Input ID" field, enter "desktop".
  7. In the "Address" field, enter "0.0.0.0".
  8. In the "Port" field, enter "10091".
  9. Click "Save" to save this source configuration.
  10. Click and drag from the "+" icon next to "AppScope" to "Prometheus".
  11. Click the second "Pipeline" button to select a pipeline to process the data.
  12. Click on the "metrics" pipeline to fill its radio button.
  13. Click "Save" at the bottom right to save this connection.
  14. Click and drag from the "+" icon next to "AppScope" to "Elasticsearch".
  15. Click the second "Pipeline" button to again select a pipeline to process the data.
  16. Select the "not-metrics" pipeline.
  17. Click "Select Pipeline" at the bottom right.
  18. Click "Save" at the bottom right to resave the whole configuration.

That's it! We've now connected AppScope to Prometheus and Elasticsearch. We created a new source for AppScope on port 10091. We connected that AppScope source to two destinations, each of which receives a full copy of the data.

The pipelines we've selected (both preconfigured) filter the dataset to just metrics for Prometheus, and to everything except metric data for Elasticsearch.

We're now ready to route data from AppScope to Prometheus and Elasticsearch. We'll use AppScope to instrument Firefox, which will show us all HTTP connections made, DNS requests made, and detailed performance metrics and events.

important

Scoping Firefox

  1. Click the "Desktop" upper tab.
  2. Click the terminal to give it focus.
  3. Type ^C to kill top if it's still running.
  4. Type export SCOPE_CRIBL=tcp://in-to-win:10091 in the Terminal, and press Enter/Return to run this command.
    • Remember that you can't copy/paste this, you'll need to type in the full configuration.
  5. Run scope firefox in the terminal.
  6. Navigate to any website in Firefox.

Data from Firefox will now be going through Stream to Prometheus and Elasticsearch. Let's verify that we see data flowing.

important

Monitoring Data Flow

  1. Click the "Stream" upper tab.
  2. On Stream's top nav, click "Monitoring".
  3. From the Monitoring submenu (second row), select "Data" > "Sources".

You should see data flowing through the "desktop" source, the "appscope" source, or both.

Next, let's go check out this data in Prometheus/Grafana and Elasticsearch/Kibana.

← Running An AppAnalyzing Data →