Skip to main content
Version: 7.13

All-in-One Quickstart for NetObserv Flow with Elasticsearch

Prerequisites​

  • Ubuntu 22 (or greater) Linux server should be provisioned with 16GB of RAM, 8 CPU cores, and 500 GB of disk space. This will allow you to store roughly 1 month of flow data at 500 FPS (Flows per second)
  • Root access to Ubuntu server. It should be internet connected, clean, unused, dedicated. It should not have any remnants of Elasticseach, Kibana, Opensearch, or ElastiFlow products on it.

Purpose​

This guide is for a specific, though common, scenario:

  • installing NetObserv Flow and Elasticsearch/Kibana (Docker based) on a single Ubuntu machine.
  • assuming around 500 FPS (flows per second)

It is assumed that you will adapt any steps from this guide to suit your specific needs.

While we are not affiliated with Elastic and do not maintain Elasticsearch, you can refer to Elastic’s official documentation for their specific guidance and best practices. This guide is provided to help you get started with NetObserv Flow as easily as possible.

For more complete and specific documentation regarding installation of NetObserv Flow itself, see either our Linux install guide or our docker install guide.

1) Installation​

You can run this single click installation command on your blank Ubuntu server.

sudo bash -c "$(wget -qLO - https://raw.githubusercontent.com/elastiflow/ElastiFlow-Tools/main/docker_install/install.sh)"

2) Send flow data​

Option 1: Most common - send flow data from your network infrastructure​

Send flow data to IP_of_your_host:9995. Refer to your network hardware vendor for how to configure Netflow 5,7,9 / IPFIX / sFlow / jFLow export.

Option 2: Send flow data from a host​

Generate flow data from one of your hosts (either the same machine running NetObserv or a different one).

  1. Install Pmacct on a machine somewhere.
sudo apt-get install pmacct
  1. Add the following Pmacct configuration to a new file located here /etc/pmacct/pmacctd.conf. Be sure to replace NETWORK_INTERFACE_TO_MONITOR with the name of an interface and ELASTIFLOW_NETOBSERV_FLOW_IP with the IP address of your ElastiFlow NetObserv Flow server.
daemonize: false
pcap_interface: NETWORK_INTERFACE_TO_MONITOR
aggregate: src_mac, dst_mac, src_host, dst_host, src_port, dst_port, proto, tos
plugins: nfprobe, print
nfprobe_receiver: ELASTIFLOW_NETOBSERV_FLOW_IP:9995
nfprobe_version: 9
nfprobe_timeouts: tcp=15:maxlife=1800
  1. Run pmacct: sudo pmacctd -f /etc/pmacct/pmacctd.conf

Option 3: Send sample flow records​

You can generate sample flow data with this approach. Be sure to replace ELASTIFLOW_NETOBSERV_FLOW_IP with the IP address of your ElastiFlow NetObserv Flow server. Do not use "localhost" here. Use an IP address.

sudo docker run -it --rm networkstatic/nflow-generator -t ELASTIFLOW_NETOBSERV_FLOW_IP -p 9995

3) Visualize your Flow Data​

Access ElastiFlow via In Kibana (https://IP_OF_YOUR_SERVER:5601) with username "elastic" / password "elastic". It may be a few minutes for flow records to populate as the system waits for flow templates to arrive. You can always access the "starting point" of ElastiFlow by doing a global search (at the top) for the dashboard "ElastiFlow (flow): Overview" and opening it.

More enrichments and functionality are available with a free basic license. You can also request a 30 day premium license which unlocks broader device support, much higher flow rates, and all of the NetIntel enrichments.

More Enrichments​

ElastiFlow NetObserv Flow is able to enrich flow records with many different pieces of data, adding business context and making those records even more valuable, from app id, to threat information, geolocation, DNS hostnames, and more. Please click here for information on how to enable various enrichments.

Notes​

  • If you need to make any ElastiFlow NetObserv Flow configuration changes (such as turning options on and off, adding your license information, or changing your password, etc), edit the .env file and / or elastiflow_flow_compose.yml and then run the following command:
sudo docker compose -f elastiflow_flow_compose.yml down && sudo docker compose -f elastiflow_flow_compose.yml up -d
  • After making configuration changes, or for troubleshooting, check the logs by doing
sudo docker logs flow-collector -f

  • If your server is has a different amount of RAM than 16GB, please view the .env file for guidance on the values for the following keys:

    • JVM_HEAP_SIZE

    • MEM_LIMIT_ELASTIC

    • MEM_LIMIT_KIBANA

  • Questions? You can find more helpful content in Community Forum and Community Slack Workspace.

  • Code in this folder may contain code from Elastic's Github Repo.