Gaming & ML: How Real-Time ML Enhances Player Experience

Introduction

Gaming is a powerhouse in the current entertainment landscape. The video game industry has experienced remarkable growth over the past decades, with steadily increasing revenue, an expanding player base, and broader global reach. According to Newzoo’s Global Games Market Report 2024, the number of video game players across the world is expected to reach a whopping 3.42 billion in 2024, while the global revenue generated by video games this year is forecasted at $187.7 billion. 

That’s significantly greater than the global movie box office revenue ($38.99 billion) and recorded music industry revenue ($28.6 billion) combined.

The success of the gaming industry is underpinned by a multitude of factors, such as the rise of Esports and the growing popularity of online multiplayer games, technological advancements in video game design (e.g., cutting-edge graphics and physics engines), cloud gaming, and the shift from the traditional one-time purchase model to live-service games.

The gaming industry is thriving in part due to the integration of data science into game development. This enables developers to create engaging, personalized, profitable, and long-lasting gaming experiences.

In this article, we will focus on a specific application of data science — real-time machine learning (ML) — and explore how it’s transforming the gaming industry. We’ll cover the following:

• The benefits and use cases of real-time machine learning in games
• Case studies to highlight how game developers are leveraging ML and real-time data
• The stages of real-time machine learning for games
• The challenges of implementing real-time ML in video games
• How platforms like Quix can help you simplify building gaming ML capabilities

The Importance of Real-Time Machine Learning in Games

In the context of gaming, real-time machine learning refers to the ability of a machine learning model to process and analyze game telemetry data on the fly, as soon as it becomes available, which enables immediate, data-driven decisions. This brings significant benefits:

• Enhanced user experience and player retention. Real-time ML allows for dynamic and personalized game experiences, adapted to individual player preferences and behaviors. This can lead to increased player engagement, satisfaction, and retention.
• Optimized game development process and resource management. Real-time ML can help automate tasks like game testing and content creation, improving development efficiency and reducing time-to-market. It also enables smarter allocation of resources, such as server capacity and network bandwidth, enhancing game performance and lowering costs.
• Improved game analytics: Real-time ML can provide valuable real-time insights into any player’s behavior and preferences. This data can be used to inform game design decisions, and it’s often an essential requirement for gaming live operations. 
• Increased revenue. Real-time ML enables targeted offers and personalized in-game purchases, which leads to increased monetization opportunities.

Going beyond these broad benefits, real-time ML unlocks a range of advanced gaming use cases. For example, real-time ML and deep learning techniques are used to create AI opponents that learn from the player's actions and adapt their strategies. As a result, the game becomes more challenging and immersive, and the player’s interactions with in-game characters are enhanced through deep reinforcement learning.

Similarly, real-time ML and artificial intelligence (such as neural networks) enable dynamic difficulty adjustment by analyzing player performance in real time. This helps keep gameplay both accessible and rewarding. Real-time ML also facilitates smarter matchmaking by pairing players based on skill level, playing history, and preferences, thus ensuring fair and competitive matches. 

Additionally, real-time ML underpins natural language processing, allowing players to interact with the game using intuitive language commands. It also powers dynamic and procedural content generation that creates or modifies the game world (such as events, quests, and environments) in real time, keeping gameplay fresh and exciting. 

Lastly, ML-driven chat moderation and anti-cheat mechanisms help maintain a positive gaming environment by swiftly identifying toxic language and cheating attempts, ensuring fairness and a safe, welcoming atmosphere for all players.

Case Studies: Real-World Examples of Real-Time Machine Learning for Games

Video game companies are using real-time machine learning techniques to drive key business objectives. We'll unpack how two industry leaders — Wildlife Studios and Activision Blizzard — are making use of real-time ML, and the benefits they are reaping. 

Wildlife Studios — Using Real-Time ML for Personalized Offers

Wildlife Studios is one of the largest mobile gaming companies worldwide. Their portfolio consists of 60+ games that have been downloaded 3 billion times by players around the world. 

Wildlife Studios uses real-time machine learning to personalize offers for players in their mobile games. The system is designed to find the right product to show to the right user at the right time, with the goal of maximizing revenue from in-app purchases. This is a critical use case for Wildlife Studios, as 60% of their revenue comes from in-app purchases, and personalized offers play a crucial role. 

Here’s a high-level overview of the ML-powered offer personalization system used by Wildlife Studios:

Whenever a user is close to a point in a game where they might be receptive to an offer (e.g., they’re struggling to advance to the next level, and buying an item might help them progress), the game requests an offer. This request is sent to the game backend server, which forwards it to an Offers Gateway. 

This Offers Gateway component communicates with the Experimentation Platform to determine which experiment group the player belongs to and what ML model to use for the offer recommendation. The Offers Gateway then sends a request to the appropriate Model API, which uses precomputed player features stored in a feature store (DynamoDB) to generate a personalized offer recommendation. These features include behavioral data such as average spending, purchase frequency, days since game installation, and number of matches played. 

The Model API returns a recommendation to the Offers Gateway, which is then forwarded to the game's backend servers and ultimately displayed to the player as a pop-up offer within the game. This entire process happens in real time. 

It’s worth noting that at each step of the workflow, data is stored in a data lake (S3) and forwarded to an analytics solution to monitor the health of the system and business performance. 

The Wildlife Studios team has developed an in-house library called da Vinci to help their data scientists create and deploy machine learning models. This library has several key features:

• It’s application-agnostic and able to train a variety of ML models.
• It integrates with MLflow to handle versioning datasets and models.
• It can be used locally (for experimentation) or on Databricks clusters for computationally intensive tasks.
• It streamlines and standardizes training and deployment, enabling faster iteration and automation.
• It provides automated reports that highlight business metrics, model errors, and feature importance, allowing data scientists to more easily evaluate model performance and identify areas for improvement.

Regarding the actual ML technique used to generate offer recommendations, the Wildlife Studios team uses a custom ML model type called  CompoundClassifier. It combines multiple binary classifiers to predict purchase probabilities for different offer types and selects the offer with the highest expected value.

All in all, it’s a pretty complex ML system, with many moving parts. However, it enables Wildlife Studios to analyze game telemetry data in real time via machine learning capabilities, and then serve personalized offers to players in-game. The company has achieved impressive results with its real-time ML system for personalized offers, attaining revenue increases ranging from 4% to 14% in their most popular games. 

Call of Duty — Using Real-Time ML to Personalize Player Engagement

Developed by Activision Blizzard, Call of Duty is one of the most popular FPS franchises in the world, attracting millions of players worldwide. Activision Blizzard uses real-time machine learning to personalize player engagement in Call of Duty. 

To be more exact, they use real-time ML to choose the best messages to deliver to players at any given time. These messages could be anything from a notification about a new map being available to a suggestion to buy a particular item based on a player's past preferences. The purpose is to encourage gamers to play more and try new features, content, and maps. 

Here’s an overview of the architecture used by Activision Blizzard:

The system continuously ingests a massive volume of real-time game telemetry data. This data is then processed and analyzed through a series of steps to offer personalized messages to players. First, data pipeline applications selectively extract and store relevant player information, such as platform and language preferences, in a data mart. 

Another subset of data is selected for real-time analysis to determine the player’s current state in the game. The state information is maintained within a graph database. 

This real-time player state data is combined with information stored in the data mart, and the combined result is served to an ML model powered by a Monte Carlo Tree Search algorithm. The ML model predicts the optimal engagement messages that should be sent to players. Messages are delivered through channels like in-game notifications, email, or SMS, creating highly personalized experiences. 

To ensure the system is improving its message selection over time, it employs an attribution process. This process analyzes the relationship between when a player sees a message and when their state changes, attributing shifts in player behavior to specific messages. The insights gained from this attribution process are used to update the effectiveness ratings of each message, allowing the system to prioritize messages that have proven successful at driving desired player actions.

Below is a summary of the technologies leveraged by Activision Blizzard to implement this real-time ML system:

Similar to the Wildlife Studios use case, we are dealing with a complex architecture with many moving parts. However, this architecture enables Activision Blizzard to leverage real-time ML and reap the benefits. Due to personalizing player engagement with real-time ML, the company noticed an increase in the number of hours gamers spend playing Call of Duty. 

The Types of Real-Time Machine Learning for Games

There are three different types (or maturity stages) of real-time ML in the gaming industry. Each subsequent type is harder to implement than the preceding one, but equally, it unlocks new opportunities. 

Type 1: Online Prediction with Batch Features

In this stage, predictions are made on live data as it comes in. However, ML model training is a batch process, and the features used for prediction are precomputed and updated periodically (e.g., every few hours, daily, or weekly). This approach is commonly used in scenarios where feature generation is computationally expensive, requires access to large datasets, and can’t be done in real time without impacting video game performance.

Online prediction with batch features is often used in game recommendation systems (game recsys). For instance, an MMO or a MOBA game might precompute recommendations for cosmetic items, quests, hero picks, and strategies based on a player's past preferences and recent gameplay. Precomputing these features can significantly reduce the latency of real-time predictions, ensuring a smooth and responsive gaming experience. These recommendations can then be served in real time when the player logs in, or as they progress through the game.

The Wildlife Studios use case I mentioned earlier is a good example of how Type 1 works in practice.

Type 2: Online Prediction with Real-Time (Online) Features

Similar to Type 1, in this stage, inference (prediction) is real-time. However, there are some notable differences when it comes to training and the way features are used. Training usually happens at regular intervals (much more frequent time frames than batch training). Feature computation can be done:

• In real time. Feature computation happens in real time, the instant a prediction request is generated.
• In near real time. Similar to batch features, these are precomputed. However, they are updated much more frequently (e.g., every 30 seconds).

Online prediction with real-time features is especially useful for game developers who must analyze player behavior to address use cases like real-time cheat detection. The ML system can predict whether a player's actions are legitimate or indicative of cheating based on real-time features such as player movement patterns, aim accuracy, resource acquisition rates, discrepancies in reaction times, and sudden skill level changes. Predictions can be used to trigger real-time actions, such as sending an alert to video game moderators, flagging the account for review, removing the player from the match, or even banning the player until further investigation. 

Going beyond cheat detection, the Call of Duty case study previously discussed in this article is a great example of how Type 2 can be implemented to improve the gaming experience. 

Type 3: Online Prediction with Real-Time Features and Continual (Online) Learning

Type 3 represents the purest form of real-time ML. Similar to Type 2, inference and features are real-time. The notable difference is around training. Specifically, with Type 3, training is stateful and incremental. The ML model learns from a stream of data continually over time, adapting and updating its knowledge, without forgetting previously learned information.

While it represents the Holy Grail of real-time ML, at the time of writing (November 2024), online prediction with real-time features and continual learning is largely theoretical or experimental in nature, and not yet a common or standard practice. That’s because it poses significant technical challenges related to infrastructure, scaling, and real-time performance, making it impractical or inaccessible for most gaming companies at this time. 

Challenges of Implementing Real-Time Machine Learning in Games

Implementing real-time machine learning capabilities that can benefit gamers and game development studios alike is far from a walk in the park. Let's dive into some of the main hurdles involved.

Ingesting and Processing Game Telemetry Data in Real Time

Popular online games can generate massive amounts of data, up to millions of events per minute during peak times. Handling this data efficiently requires the underlying infrastructure to exhibit certain characteristics:

• Reliability, availability, and fault tolerance
• Predictable, sub-second latencies
• The ability to scale to meet fluctuating demand
• Data quality and integrity guarantees (e.g., exactly-once processing)

It’s true that technologies like Apache Kafka and Apache Flink make it possible to collect and process high-frequency, high-velocity data game telemetry data. However, just because it’s possible doesn’t mean it’s easy. Configuring and managing Kafka and Flink (especially in-house) is often difficult, time-consuming, and expensive, requiring one or even more specialized teams. 

For reference, check out the following resources to get a taste of the complexity of using Flink, and some of the hurdles you’ll likely encounter:

• PyFlink - A deep dive into Flink’s Python API
• Common issues when using Spark Structured Streaming with Pyspark and Kafka   

Real-Time Inference and ML Model Development

Once game data is collected and processed, we move to the crucial real-time analysis stage. This introduces even more complexity. Here are but a few of the questions and challenges that need to be answered and dealt with:

• What are the main gaming use cases where real-time ML would be most impactful?
• What real-time ML approach should be used (Type 1, 2, or 3)?
• What features are critical to leverage for real-time predictions, and how can they be extracted?
• What machine learning tools and models are best suited for the identified use cases?
• Can ML models make predictions fast enough to support real-time decisions without affecting gameplay and adding excessive latency?

Additional factors to consider are the frequency of model retraining to keep up with evolving player behavior, strategies for monitoring model accuracy and detecting shifts in player patterns, the cost-benefit analysis of maintaining real-time ML systems, and the mechanisms for deploying, versioning, and rolling back models across different game builds and updates. Handling player privacy concerns around data collection for ML model training is also a crucial consideration.

Deploying Real-Time ML Systems to Production

Releasing ML capabilities into production is a multi-faceted challenge. First of all, for those accustomed to batch ML, the shift to real-time machine learning requires some getting used to. New technologies to learn, new workflows and testing methodologies to adopt, new operational concerns to manage, new error handling strategies to develop, and so on. It can be a lot to take in, and time is needed to make the transition.

Secondly, strong, multi-disciplinary collaboration is required to release real-time ML capabilities into production. Several distinct roles are usually involved in real-time ML pipelines, such as:

• Software engineers are crucial for data ingestion (making new data points available to data teams) and for getting the results of the inference back into the game engine to influence gameplay.
• Data scientists are responsible for identifying the right features to use and for experimenting with different machine learning algorithms and models.
• Data/ML engineers are tasked with deploying the ML model into production and setting up and monitoring the whole ML infrastructure.

These roles have different skill sets, which can be a barrier to efficient collaboration and may lead to convoluted workflows and even occasional friction. For example,  tech used by data/ ML engineers (such as Apache Kafka and Apache Flink) is primarily Java and Scala-based. Meanwhile, data scientists usually work in Python. 

This language barrier is a problem that becomes obvious, for instance, in the case of feature engineering. Data scientists often define their feature computations and build ML models in Python. Data engineers must then rewrite this logic in Java or Scala to put the models online. This refactoring ultimately slows down the release of real-time ML capabilities into production.

Simplifying Real-Time Machine Learning for Games with Quix

Implementing real-time machine learning capabilities can be a journey fraught with challenges. However, the integration of game development & machine learning paves the way to enhance gameplay, improve player retention, and increase revenue, giving game development studios a cutting edge in a highly competitive market.

Are you a data professional in the gaming industry looking to capitalize on real-time ML while keeping complexity to a minimum? Take a look at Quix, a managed, Kafka-based stream processing platform trusted by leading gaming companies like Miniclip, King, and Codemasters.

Quix makes it straightforward and convenient to operationalize real-time data and ML and switch from offline ML:

• Serverless environment, eliminating the need for you to manage infrastructure and allowing you to focus on building and deploying ML pipelines.
• Scalability, reliability and fault tolerance, so you can manipulate high-volume streams of real-time game telemetry data with consistently low latencies.
• Pure Python and easy integration with your favorite Python libraries, so you can comfortably work with real-time data and ML using the tools you’re already familiar with.
• Intuitive GUI with built-in code editor, so it’s easy to code your logic, test apps, deploy them to production and monitor them in one place.
• Streaming DataFrame API (similar to pandas DataFrame) for rich processing and transformation operations on streaming data.
• Ready-made sink and source connectors so you can conveniently integrate your external data sources and destinations with Quix. You can also build custom sink and source connectors via the Quix Source and Sink APIs.
• 50% lower total cost of ownership compared to alternatives for ingesting and processing game telemetry data.

For more information on how you can leverage real-time data and ML for gaming use cases with Quix, check out this game telemetry demo project or book a demo with the Quix CEO.