Elasticsearch Output: support for TSDS - TSDS output for Elasticsearch is now a fully supported feature and out of Technology Preview. Enabling Time Series Data Streams (TSDS), introduced in Elasticsearch 8.7, can result in storage savings of 50-70% depending on the content of flow records. Enabling TSDS does increase the ingest-related CPU load for Elasticsearch, which can be largely mitigated by the ingest CPU optimizations introduced in Elasticsearch 8.8. How to enable TSDS:
In Kibana, delete the 3 existing ElastiFlow index templates, as new ones will automatically be created once TSDS is enabled.
Stop your flow collector instance.
Open flowcoll.conf and set
EF_OUTPUT_ELASTICSEARCH_TSDS_ENABLEtotrue.Restart your flow collector instance.
Note: Enabling TSDS will not affect any existing data already in Elasticsearch. All dashboards will visualize data both before and after TSDS is enabled.