Jun 6, 2024
Introduction to stream processing
This first episode in the course is an introduction to stream processing. You’ll learn some fundamental concepts and will have an understanding of why and when to use stream processing. There will be a practical section showing you how to get started with producing, consuming and transforming data.
Presenters:
Prerequisites
- 🐍 Python 3.8+
- 🐳 Docker
- ✍🏽 IDE (VS Code, PyCharm or GitHub Codespaces)
- ⭐️ Star the repo to follow/support the project https://github.com/quixio/quix-streams
Register to attend live:
Watch the recording:
What you'll learn
📖 Theory
- Introduction to stream processing
- When stream processing is needed
- Stream processing architecture
- Kafka fundamentals
- Quix Streams fundamentals
💻 Coding Session
- Create a local environment from scratch
- Create a demo data source
- Connect to a local Redpanda broker
- Produce messages
- Consume messages
- Build and deploy a data normalization data pipeline locally
Useful resources
💻 Code
- GitHut repo https://github.com/quixio/streaming-academy
📚 Docs
- Why use stream processing? https://quix.io/docs/kb/why-stream-processing.html
- Quix Streams introduction https://quix.io/docs/quix-streams/introduction.html
📝 Blog Posts
- A practical introduction to stream reprocessing in Python https://quix.io/blog/intro-stream-reprocessing-python
Understanding Kafka’s auto offset reset configuration: Use cases and pitfalls https://quix.io/blog/kafka-auto-offset-reset-use-cases-and-pitfalls
🎓 Exercises
- Join the Quix Community Slack and the #quix-academy channel https://stream-processing.slack.com/archives/C075MGFT3TN
- Share the course details with friends and colleagues.
Get notified about upcoming episodes
Leave your email and we'll drop a reminder in your inbox when future episodes of the course are ready to go live.
Thanks! Look out for reminders in your inbox 👀
Please check your email address is correct
