September 29, 2022

The Stream — September 2022 edition

The September 2022 edition of The Stream: covering this month in stream processing on the internet.

The Stream September 2022 banner.

Stream processing is defining a new world for organizations that want to be able to use and make sense of data in the moment it is created. Unfortunately, this leading-edge technology is not easily and broadly understood, and even less easily wrangled into a cohesive, performant data pipeline.

In the Gartner market survey, “Adopt Stream Data Integration to Meet Your Real-Time Data Integration and Analytics Requirements,” the research company found that “In our annual survey for the data integration tools market, 47% of organizations reported that they need streaming data to build a digital business platform, yet only 12% of those organizations reported that they currently integrate streaming data for their data and analytics requirements.”

That means that just 5.6% of companies are actually harnessing the power of streaming data to generate immediate insights, power ML capabilities or feed fresh data back into their products. Looking at this on the adoption curve, this represents just the beginning: the so-called “innovators.”

The next chunk of companies to use stream processing — the 41.4% of companies who would fit either into an “early adopters” or “early majority” category — are aware they want and need to be able to use this technology, but can’t easily do so.

It’s tricky. Combining Kafka, Kubernetes, and many other newer technologies is not for the faint of heart (or light of wallet). Large-scale companies that have already deeply invested in developer resources here such as John Deere, and Alibaba are leading the charge, but where does that leave midmarket and digital-first companies with fewer resources?

This month, we’re connecting with organizations of all sizes and industries by hosting community meetups in London, Berlin, Munich and Austin, speaking at events like Big Data London, and joining the Kafka user community at Current in Austin in early October. Our goal is to bring the power of stream processing to everyone by building shared knowledge, understanding and great tools to democratize data stream processing. Join us.

An image of a cat along tech company logos.

Streaming-first infrastructure for real-time machine learning

A streaming infrastructure can improve ML prediction latency and continual learning. Chip Huyen and Anthony Alford explain why an event-driven microservices architecture is a better choice for using continual learning than a REST-based architecture.

Learn more

Logos of four open source stream processing technologies.

Explore four popular open-source stream processing technologies

Gary Stafford offers a technical two-part post comparing these stream processing projects: Apache Spark Structured Streaming, Apache Kafka Streams, Apache Flink, and Apache Pinot.

Explore more →

Zhamak Dehghani Big Data conference.

Understanding the Data Mesh with Zhamak Dehghani

Dig into one of the most talked about trends in data, the Data Mesh. Dehghani offers “atomic steps to rewire the sociotechnical backbone of your organization,” and how to apply it to get value from data rapidly, sustainably and at scale.

Read more →

Colorful event banner saying Current.

Destination: Austin, Texas for Current

It’s not just the smell of brisket in the air — it’s the Kafka user community coming together at Current, the conference for data professionals to learn about data streaming, Oct. 4-5.

Register now →

More insights

  • According to Memgraph, the most common use case for stream processing is monitoring sensor or device data.
  • In the latest Redpanda University course, you’ll learn about stateless and stateful stream processing and work through an advanced end-to-end tutorial. Enroll for free.
  • Ververica offers a new video on streaming concepts for Flink: “Exactly Once Fault Tolerance Guarantees.” Watch on YouTube.
  • What’s next for streaming analytics on Spark and Flink? Mike Ridley discusses complex event processing with Macrometa on YouTube.

What’s a Rich Text element?

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

Static and dynamic content editing

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to customize formatting for each rich text

Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.

Related content

The Stream May 2023 banner.

The Stream — May 2023 edition

A monthly round-up of the most interesting news coming out of the stream processing ecosystem
Mike Rosam
Words by
The Stream April 2023 banner.

The Stream — April 2023 edition

A monthly round-up of the most interesting news coming out of the stream processing ecosystem
Mike Rosam
Words by
The Stream March 2023 banner.

The Stream — March 2023 edition

A monthly round-up of the most interesting news coming out of the stream processing ecosystem
Mike Rosam
Words by
The stream

Updates to your inbox

Get the data stream processing community's newsletter. It's for sharing insights, events and community-driven projects.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.