Migrate to Influx V3
A basic two-step pipeline that you can use to keep an InfluxDB V3 database synchronized with an InfluxDB V2 bucket. The pipeline periodically queries the InfluxDB V2 bucket for all measurements and data. When new records are detected, they are sent to a Kafka topic. A consumer process listens for new records and filters them for user-specified tags and fields. Finally, the filtered records are written as points to the corresponding InfluxDB V3 database.
Main project components
InfluxDB V2 Source
A service that queries your chosen V2 InfluxDB bucket at a configurable interval and writes all measurements and associated data to a Kafka topic.
InfluxDB V3 Sink
A service that continuously listens to the "influxV2-data" Kafka topic and writes new data to your chosen InfluxDB V3 database while only including your chosen tags and fields.
Technologies used
Using this template
This project could be easily adapted for use cases such as:
- Iteratively migrating data from InfluxDB V2 to V3
- Creating a V3 replica of an InfluxDB V2 database
