Autodiscovery of Devices

info

Device autodiscovery is currently a technology preview. The design and implementation are less mature than stable features and subject to change.

Overview

ElastiFlow can automatically discover SNMP-enabled devices. To do this, you will install and use a command-line tool called netobserv.

note

You will need to have NetObserv SNMP 7.12.0 (or later) already installed. That will have the latest version of the snmp definition files. If upgrading to 7.12.0, make sure you follow the manual upgrade steps for SNMP definition files.

You can install netobserv tool via our deb or rpm packages.

Assuming you use default settings for NetObserv SNMP, here's how you run autodiscovery:

1. Define a 'networks' input file at /etc/elastiflow/snmp/autodiscover/networks.yml, and specify the network ranges you want to scan.
   1. You can do this by copying the example file found at /etc/elastiflow/snmp/autodiscover/networks.yml.EXAMPLE and then editing it.
   2. Details are found below on what the file format looks like.
2. Run /usr/share/elastiflow/bin/netobserv autodiscover snmp.
   1. There are also additional flags to customize behavior if you want.
3. Trigger the apply-definitions API to reload device yml config.

Known Issues

Routers can have indirect duplicates sometimes. If a router does not have a non-local loopback interface, then, as of 7.12.0 release, then the device yml output will likely have all separate addresses of that router listed as separate device entries in its output

How Device Autodiscovery Works

Device autodiscovery will:

1. Receive a list of IP ranges you want to scan for devices.
2. Ping sweeps all addresses in the ranges you provided to find what devices exist.
3. Use SNMP to get the sysOID value (this specifies what device type it is).
4. Figure out what device_group name ElastiFlow associates with that sysOID.
5. Output yml file(s) that NetObserv SNMP can read to know what devices to poll. You have to tell NetObserv SNMP to reload config.

Concerning routers: Autodiscovery will query every router for its administrative loopback IP address and use that IP address instead. Whichever loopback interface appears first in the list will be the loopback IP used in autodiscovery.

You can safely rerun autodiscovery as many times as you like. If a defined 'group' of network blocks was already scanned, it will not rescan them. See the "Output" section for more details, including how to force a rescan.

Devices found in earlier network groups will be excluded from subsequent groups to avoid duplicate definitions.

The command will create the output directory if it doesn't exist.

For large networks, consider adjusting the concurrency setting to balance speed and network load.

Autodiscovery will ignore these 'local' address ranges:

• 127.0.0.0/8
• 169.254.0.0/16
• fe80::1/10
• ::1/128

Installation of Command Line Tool

You can download and install it on linux machines via our deb or rpm packages:

wget https://elastiflow-releases.s3.us-east-2.amazonaws.com/netobserv/netobserv_7.12.0_linux_amd64.debsudo apt install ./netobserv_7.12.0_linux_amd64.deb

wget https://elastiflow-releases.s3.us-east-2.amazonaws.com/netobserv/netobserv-7.12.0-1.x86_64.rpmsudo yum install -y netobserv-7.12.0-1.x86_64.rpm

If upgrading from a previously installed netobserv version, run the following:

sudo rpm -Uhv netobserv-7.12.0-1.x86_64.rpm

Running Device Autodiscovery

note

You will need to have NetObserv SNMP 7.12.0 (or later) already installed. That will have the latest version of the snmp definition files. If upgrading to 7.12.0, make sure you follow the manual upgrade steps for SNMP definition files.

1. Creating the "Networks" File Input

You must tell autodiscovery which network ranges to scan. This is done via a yml file. This file will contain:

• SNMP credentials to use for authentication
• Connection parameters (port, timeout, retries, etc.)
• Network ranges to scan

Example networks.yml:

# Note: the key name "public", in this example, can be replaced with any string. The keys here are arbitrary strings you want to use to identify each 'set' of network ranges to scan.
# When autodiscovery is done, it will create new yml files using this name. In this example, it will create a file called "public.yml"
# (the default location would be /etc/elastiflow/snmp/devices/public.yml)
public:
  credentials:
    - version: "1"
      community: public
    - version: "2c"
      community: public
  port: 161
  poll_interval: 60
  timeout: 2000
  retries: 2
  exponential_timeout: false
  networks:
    - 10.101.2.0/24
    - 10.101.3.0/24
    - 10.101.254.0/24

secure_network:
  credentials:
    version: "3"
    username: admin
    authentication_protocol: SHA
    authentication_passphrase: authpass
    privacy_protocol: AES
    privacy_passphrase: privpass
  networks:
    - 192.168.1.0/24

Details about this file format:

Option	Description
credentials	List of SNMP credential configurations to try when connecting to devices
port	SNMP port to connect to (default: 161)
poll_interval	Interval in seconds between SNMP polls (default: 60)
timeout	Timeout in milliseconds for SNMP requests (default: 3000)
retries	Number of retries for failed SNMP requests (default: 2)
exponential_timeout	Whether to use exponential backoff for retries (default: false)
networks	List of network ranges to scan. Either in CIDR notation (e.g., 10.0.0.0/24) or hyphenated rang (e.g., 10.0.200.0-10.0.203.0). You can also list individual IPs.

2. Running Autodiscovery Command

If you use all the default values, you can just run netobserv autodiscover snmp to do autodiscovery. Otherwise, use one of the below flags to configure how it runs.

Command Line Interface

netobserv autodiscover snmp [flags]

Flag	Shorthand	Default	Description
--networks	-n	/etc/elastiflow/snmp/autodiscover/networks.yml	YAML file containing the list of network groups to discover devices from.
--output	-o	/etc/elastiflow/snmp/devices	Output directory where discovered devices will be stored.
--mappings	-m	/etc/elastiflow/snmp/autodiscover/sysoids	YAML file containing sysObjectID to device group mappings. NetObserv (starting at 7.12.0) ships with mappings.
--concurrency	-c	100	Number of concurrent workers to use for device discovery.
--timeout	-t	3000	Default timeout in milliseconds for each SNMP request (can be overridden in the networks file).
--retries	-r	2	Default number of retries for each SNMP request (can be overridden in the networks file).
--poll-interval	-p	60	Default interval in seconds between each SNMP poll when running NetObserv SNMP (can be overridden in the networks file).
--verbose	-v	false	Enable verbose output (debug logging).
--overwrite			Normally, Autodiscovery will not rescan devices for network groups it already scaned. This flag will rescan, and overwrite output files.

Output

When autodiscovery finishes, you can run ls /etc/elastiflow/snmp/devices to see additional yml files created. Those files will have devices defined, which NetObserv SNMP will automatically load when you tell it to reapply definitions (next step).

Here's an example output file for the example input file from above

# /etc/elastiflow/snmp/devices/public.yml
router1_10.101.2.1:
  ip: 10.101.2.1
  port: 161
  version: 2c
  communities:
    - public
  device_groups:
    - generic

switch1_10.101.3.5:
  ip: 10.101.3.5
  port: 161
  version: 2c
  communities:
    - public
  device_groups:
    - cisco_cat_2960 # This was derived from the sysOID of the device.

# /etc/elastiflow/snmp/devices/secure_network.yml
re-router:
  ip: 192.168.1.1
  port: 161
  version: 3
  v3_credentials:
    - username: admin
      authentication_protocol: SHA
      authentication_passphrase: authpass
      privacy_protocol: AES
      privacy_passphrase: privpass
  device_groups:
    - generic

If an output file already exists, it is perserved and unchanged. Autodiscovery will skip scanning any addresses in that network 'group.'

You can use the --overwrite feature to forget perviously scanned results and rescan and recreate output again.

Explaining SysObjectID Mappings

The --mapping (-m) flag is optional. When using this flag, you can provide a YAML file that maps SNMP sysObjectIDs to device group names. This helps categorize discovered devices based on their system object identifiers.

If the -s flag is not provided or if a device's sysObjectID does not match any mapping in the file, the device definition will use the "generic" device group by default.

ElastiFlow's comprehensive set of sysObjectID to device group mappings is both publicly available for download and shipped with NetObserv SNMP product.

Example sysoid-mappings.yml:

.1.3.6.1.4.1.9.1.12: 'cisco_7000' # Cisco 7000 CISCO7010
.1.3.6.1.4.1.9.1.13: 'cisco_2500' # Cisco 2500 CISCO2500
.1.3.6.1.4.1.9.1.14: 'cisco_4000' # Cisco 4000 CISCO4500

Each entry maps a sysObjectID (OID) to a device group name. When a device is discovered with a matching sysObjectID, it will be associated with the specified device group.

3. Telling NetObserv SNMP to Re-Read Device Files

Trigger the apply-definitions API to reload device yml config.

POST localhost:8080/api/v1/snmp/apply-definitions should work, assuming all default settings for NetObserv SNMP.