Predicting crowd density for public transport in real time

CK Delta, a data innovation unit within CK Hutchison Holdings, develops AI products that help internal and external teams operate more efficiently. They rely on agile methodologies and design thinking as driving forces, offering their hands-on expertise as a service to external companies. Like M1, a subsidiary of Keppel Corporation, and Singapore's first digital network operator, that provides a suite of communications services, including mobile, fixed line and fibre offerings, to over two million customers.

The opportunity

Prevent overcrowding at train stations in Singapore

M1 won public funding to provide Wi-Fi routers throughout Singapore’s transportation network (LTA). One condition of this funding was that the city’s citizens should in some way benefit from the data being collected by the Wi-Fi routers. To meet this condition, M1 opted for a predictive analytics solution that could forecast crowd density at train stations.This data would be used to power a mobile app that provided citizens with information about anticipated waiting times and crowd congestion at each station. By providing this information, the app could in turn help prevent overcrowding at stations. This is especially appealing in a post-COVID world, when social distancing is still important to many.

The solution

A production-ready event streaming system for ML

CK Delta had to make a quick decision: they could create an in-house streaming system or find a production-ready solution. The former involved upskilling their team and introducing a learning curve that would be certain death for the deadline. It wasn’t even a choice. The only option was to find and implement a production-ready solution that met their requirements. They eventually chose Quix, the only solution in the market that offers a full-stack yet modular stream processing platform that provided the assurance needed to complete the project on time. It would enable data scientists to process data in Python and autonomously deploy their ML models into production without re-engineering. At the same time it allowed their platform team to seamlessly integrate with the current IT architecture of M1 and LTA.

Since they were dealing with public data, the system also had to be extremely secure. The Quix platform was installed on a dedicated AWS cloud account, accessible only by a limited number of pre-approved engineers.

The outcome

ML & platform teams cooperate to safely launch and scale the predictive app

The key to success was the smooth collaboration between the Quix and CK Delta teams. To speed up the delivery date, a data team of three data scientists and a platform team of two engineers worked in parallel in two separate streams: while the data scientists developed algorithms, the platform team worked on integrating Quix into M1's infrastructure. ML models created offline by data scientists in Python were easily copied to Quix without refactoring the code into Java or Scala. This skipped a standard step in data engineering of re-engineering code, usually required by stream processing systems, saved time and made it easier to test the application’s functionality and ensure that there were no failures in production. The system went live within the deadline, allowing time to test the results against the old system.

To meet security standards, the Quix environment was locked down, meaning no resources could be accessed from the internet, so VPNs were used to access any part of the platform. Application resources were isolated for development, staging, and production, with strict access control at network and resource levels (VPN and platform user accounts per environment). The team also ran regular security scans and penetration tests on all nodes to identify vulnerabilities and perform software patches as needed.

When they were ready to launch the system, CK Delta had planned for a daily processing capacity of 10GB, but ultimately found themselves processing nearly 40GB per day—four times the anticipated data. Quix consistently demonstrated its resiliency and scalability, ensuring everything ran without a hitch.

Given the system’s success, CK Delta now plans to roll it out to additional cities, regions, and locations.