Quix Streams Release 3.17.0

Quix Streams 3.17.0: Interval Joins and Enhanced Debugging

New Feature: Interval Joins

Quix Streams 3.17.0 introduces `StreamingDataFrame.join_interval()`, enabling you to join records from two topics within a specified time window. This powerful feature is particularly useful for correlating events that occur within a defined temporal relationship.

from datetime import timedelta
from quixstreams import Application

app = Application(...)
sdf_measurements = app.dataframe(app.topic("measurements"))
sdf_events = app.dataframe(app.topic("events"))

# Join measurements with events occurring within ±5 minutes
sdf_joined = sdf_measurements.join_interval(
    right=sdf_events,
    how="inner",
    on_merge="keep-left",
    grace_ms=timedelta(days=7),
    backward_ms=timedelta(minutes=5),
    forward_ms=timedelta(minutes=5)
)

Key Benefits:

- Temporal Correlation: Match records based on time proximity rather than just key-based joins

- Flexible Windows: Define asymmetric time windows before and after each event

- Efficient Processing: Built-in state management with configurable grace periods

Enhanced Application Debugging

1. Improved `Application.run()` Behavior

The `count` parameter in `Application.run()` now refers to the number of output messages rather than input messages, making it more intuitive for testing and debugging:

# Process exactly one output message
result = app.run(count=1, collect=True)

# Process messages for up to 10 seconds
result = app.run(timeout=10, collect=True)

2. Output Collection

Easily capture and inspect processing results during development:

# Collect values only
values = app.run(count=5, collect=True)

# Collect values with metadata (offsets, timestamps, etc.)
messages = app.run(count=5, collect=True, metadata=True)

New Connector: InfluxDB 1.x Sink

Version 3.17.0 adds support for InfluxDB 1.x as a sink destination, expanding the range of time-series databases you can integrate with Quix Streams.

Other Improvements

- Better Recovery Logging: Progress updates during state store recovery

- PostgreSQL Sink: Fixed JSONB value handling

- BigQuery Sink: Added user agent identification for better request tracking

Upgrading

Update to the latest version using pip:

pip install --upgrade quixstreams

For detailed migration guidance, refer to the Quix Streams documentation and the full release notes.

Next Steps

- Explore the Interval Join documentation

- Check out the debugging guide for tips on testing your streaming applications

- Join our community Slack to discuss your use cases with other Quix users

‍