Interested in this use case?

If you'd like us to focus on building this template next, register your interest and let us know. You can also head over to the Quix Community Slack if you've got any questions.

Built on Quix with:

Project template

Use case

Code snippet

Sink high volumes of data to S3/Iceberg

This project is an example of how to transform Apache Kafka data before sinking it to Amazon S3 in Parquet format. By utilizing Apache Iceberg you can level up your data lakehouse and by combining Apache Kafka and Apache Iceberg with Quix Streams pure Python library you can win the game outright.

Use cases:

Data integration

ETL

Created by:

Quix

Architecture diagram showing the data integration template pipeline

Main project components

Data Ingestion

A websocket service that receives data from each game client.

Transformation

Normalize game data to enable easier downstream processing.

S3 Iceberg Sink

Sink raw game data and computed values to S3 for model training and analysis.

Technologies used

Using this template

Data integration. Data transformation and sinking.

Built on Quix with: