ElastiFlow 6.4: Elasticsearch Output - support for TSDS

Elasticsearch Output: support for TSDS - TSDS output for Elasticsearch is now a fully supported feature and out of Technology Preview. Enabling Time Series Data Streams (TSDS), introduced in Elasticsearch 8.7, can result in storage savings of 50-70% depending on the content of flow records. Enabling TSDS does increase the ingest-related CPU load for Elasticsearch, which can be largely mitigated by the ingest CPU optimizations introduced in Elasticsearch 8.8. How to enable TSDS:

• In Kibana, delete the 3 existing ElastiFlow index templates, as new ones will automatically be created once TSDS is enabled.

• Stop your flow collector instance.

• Open flowcoll.conf and set EF_OUTPUT_ELASTICSEARCH_TSDS_ENABLE to true.

• Restart your flow collector instance.

Note: Enabling TSDS will not affect any existing data already in Elasticsearch. All dashboards will visualize data both before and after TSDS is enabled.