back
March 10, 2022
|
Industry insights

The Stream — March 2022 edition

The March 2022 edition of The Stream: covering this month in stream processing on the internet.

The Stream March 2022 banner.
Insert Quix boilerplate.

There’s no instruction manual for stream processing. With established technology, there are volumes of information about how to do something better — books, blogs, user groups, reference architectures. But with an emerging technology as complex and varied as data stream processing, it’s a brave new world.

That’s why Quix is sponsoring the first community dedicated to figuring this all out. We’re calling it The Stream. Whether you’ve been processing data on a message broker like Kafka for a while, or you’re simply stream-curious, you’re welcome. You’ll find friends and helpful allies here.

Our community connects on Slack and in person at meetups. Our first two live events are in Berlin this month. You should be a part of the community if you:

Work with data — whether as a software developer, data scientist, or engineer of another stripe. We’re having great conversations with mechanical and electrical engineers who focus on building IoT devices and need the data stream processing backbone to make them go.

Write code in Python — while many data-oriented technologies have centered on SQL, and streaming data has roots in Java and Scala for software engineering, we’re focusing on Python because it’s easy to use (making streaming accessible to newcomers), extensive (there’s a huge ecosystem of packages, like Pandas, that do much of the work for you) and incredibly flexible (making streaming useful to anyone from any industry).

Are interested in real-time data — you recognize the power and potential of streaming data, migrating workloads from batch to stream processing, or realizing the benefits of it such as lower latency, more efficient use of compute resources, and real-time automation or ML.

I believe passionately in streaming data. It will be the defining technology of this era. The 2010s focus on big data is giving way to faster architectures that enable you to build new product categories.

It’s early days, and innovation can be complicated. I believe this community will be coming together to better understand not only how to use stream processing, but also when not to. We’re not about technology for technology’s sake. We’re about finding new and better ways to handle the massive volume and velocity of data that’s streaming in from IoT devices and digital experiences every day.

Our community’s goals are learning, supporting each other, and helping to develop connected devices and data-driven products faster, easier and more efficiently.

You might use Quix for that. You might not. Either way, you’re welcome and we hope to make the community a valuable forum and connection for you.

Start today on the Slack channel and introduce yourself. Drop a meeting on my calendar to talk about your use case. I’m excited by the potential our community can collectively realize. Join us!

The stream on black background.

Meet The Stream in person

We’re gathering in Berlin on March 31, 2022 to talk about the ins and outs of stream processing.

Join us

Two people in a dessert.

Can better data help policy makers and communities build resilience?

How can streaming data help fight against food insecurity? “By using machine learning and other tools, high-frequency data collected by MIRA has greater potential to predict food insecurity outcomes and identify households likely to be most affected by shocks and stressors.”

Read more →

White background showing letters from A to H.

A glossary for stream processing

We’ve put together a long list of terms and definitions important to stream processing. You won’t find anything specifically about Quix in there; it’s as wide-reaching and inclusive as possible to help anyone navigate the world of real-time data.

Check it out →

More insights

  • Data Council offers a “100% no bullsh*t guarantee” event about data science, engineering and analytics. You’ll want to join for the easy networking and the abundance of speakers across six learning tracks. There’s something for everyone here.
  • The European Union has proposed an addition to its digital rulebook. Read about it on TechCrunch.
  • Niyi Odumosu, Associate Solutions Architect at Confluent, uses the Metrics API, Docker, Prometheus, Grafana, Splunk and Datadog to put together a full monitoring solution for your Confluent Cloud deployment.
  • To settle some confusion about the terms real time and stream processing, check out How stream processing goes beyond real-time.

What’s a Rich Text element?

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

Static and dynamic content editing

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to customize formatting for each rich text

Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.

Related content

Banner image for the article "Streaming ETL 101" published on the Quix blog
Industry insights

Streaming ETL 101

Read about the fundamentals of streaming ETL: what it is, how it works and how it compares to batch ETL. Discover streaming ETL technologies, architectures and use cases.
Tun Shwe
Words by
LLMOps: large language models in production with Quix
Industry insights

LLMOps: running large language models in production

LLMOps is a considered, well structured response to the hurdles that come with building, managing and scaling apps reliant on large language models. From data preparation, through model fine tuning, to finding ways to improve model performance, here is an overview of the LLM lifecycle and LLMOps best practices.
Tun Shwe
Words by
What is stream processing
Industry insights

What is stream processing?

An overview of stream processing: core concepts, use cases enabled, what challenges stream processing presents, and what the future looks like as AI starts playing a bigger role in how we process and analyze streaming data
Tun Shwe
Words by