back
July 18, 2023
|
Ecosystem

Accelerating AI-ready application development: Quix and Confluent partnership

Teams can now build AI applications on Confluent’s data in motion, with Quix, the AI-ready event streaming application framework.

Quix ML model icons on black background.
Quix Streams combines an Apache Kafka client with a stream processing library, offering a feature-rich pure Python alternative to the kafka-python library. Deploy on your stack or on Quix Cloud for scalable, stateful and fault-tolerant ETL without the headache of cross-language debugging.

👉 Star the repo on GitHub and pip install quixstreams to get started.

Introduction

Confluent will go down in history as the company that set the world’s data in motion. We have long admired this mission having experienced the power of Apache Kafka at McLaren Racing where we used it to gain an advantage from real-time data processing.

At McLaren we ingested streaming data into Kafka fairly quickly, but struggled to roll out the technology across the business due to the language barriers and impedance gaps inherent in modern, multi-disciplinary technical organisations. Ultimately we spent three years building an internal developer platform around Kafka, and still burned hundreds of hours supporting developers across the team to build event streaming applications.

We founded Quix to solve these problems by building a tool that empowers AI teams to develop real-time applications that run directly on Kafka without complex infrastructure build-outs or high-touch support from platform teams.

Today, I’m delighted to announce the launch of a deep integration between Quix and Confluent. Together with today’s announcement that Quix is a launch partner in the Connect with Confluent programme, we fill a crucial gap in the AI ecosystem and take a giant leap on our mission to empower developers to unleash the full potential of real-time applications.

Why Quix and Confluent are better together

Confluent is the de facto standard in streaming data. However, until now, one critical capability has been missing from the ecosystem—the ability for teams to rapidly develop AI applications directly on Kafka. This is where Quix steps in, offering an event streaming application framework that empowers your teams to unlock the full potential of real-time data.

| | Confluent | Quix | Together | |-------------------|-------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| | Connect | ✅ 180+ ready-to-run Kafka Connect connectors with fully managed clusters. | ✅ 60+ open source code samples running in Fully managed Kubernetes containers. Compatible with any framework e.g. Airbyte, Debezium MQTT, OpenTelemetry etc. | 🔥 Quickly ingest and sink any data from any source to any destination | | API | ✅ Confluent REST Proxy API (producer/consumer) Confluent Cloud API Connect REST API ksqlDB REST API Metadata API Schema Registry API | ✅ REST producer/consumer API WebSocket producer/consumer API REST Query API REST Portal API | 🔥 Rapidly integrate to any external system e.g. Jupyter, MLFlow, Weights & Biases, Seldon etc. | | Streaming | ✅ Enterprise ready fully managed Kafka | 🔐 Bring Your Own Kafka with native TCP/IP integration | 🔥 Stream and process data with best-in-class tooling | | SQL Processing | ✅ Fully managed ksqlDB (and FlinkSQL coming Q4 2023) clusters | ❌ None | 🔥 Transform and join data in motion | | Python Processing | ❌ None | ✅ Fully managed Kubernetes clusters with Quix Streams open source Python library. | 🔥 Real-time machine learning feature computation and AI model serving | | Storage | ✅ Unlimited tiered storage for long-term Kafka log persistence | ✅ Queryable storage, replay service and data exporter for AI development and testing | 🔥 Access streaming data for exploratory data analysis and build traceable AI application pipelines | | KafkaOps tools | ✅ Management console. Cluster Replicator. Schema registry. Stream Lineage. Stream Catalog. | ❌ Application config and monitoring tools only | 🔥 Control Kafka in dev and production environments | | DevOps tools | ❌ ksqlDB pipeline builder only. | ✅ Pipeline builder. Samples library. Cloud IDE. Git integration, YAML IaC and CI/CD workflow | 🔥 Enable data science and ML teams to quickly build AI-ready applications |

Demo: see how the integration works

Conclusion

The partnership between Quix and Confluent represents a significant milestone in the world of real-time applications. By combining the power of Confluent's industry-leading Kafka platform with Quix's F1-derived event streaming application framework, organisations can now build AI-ready applications directly on data in motion in Confluent Cloud. Explore the new use cases you can unlock with Confluent and Quix or try it yourself here.

What’s a Rich Text element?

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

Static and dynamic content editing

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to customize formatting for each rich text

Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.

Related content

Featured image for the "How to fix common issues when using Spark Structured Streaming with PySpark and Kafka" article published on the Quix blog
Ecosystem

How to fix common issues when using Spark Structured Streaming with PySpark and Kafka

A look at five common issues you might face when working with Structured Streaming, PySpark, and Kafka, along with practical steps to help you overcome them.
Steve Rosam
Words by
Featured image for the "Quix Streams, a reliable Faust alternative for Python stream processing " article published on the Quix blog
Ecosystem

Quix Streams—a reliable Faust alternative for Python stream processing

A detailed comparison between Faust and Quix Streams covering criteria like performance, coding experience, features, integrations, and product maturity.
Steve Rosam
Words by
The logos of Flink and Python
Ecosystem

Debugging PyFlink import issues

Solutions to a common issue that Python developers face when setting up PyFlink to handle real-time data.
Steve Rosam
Words by