View and store the data
With Quix it's easy to visualize your data in a powerful and flexible way, you can see the data in real time, as well as viewing historical data.
Quix was designed for real-time data, so if you want to see data-at-rest for any topic you must turn on data persistence for that specific topic. You'll do this in the historical data section.
Follow these steps to view real-time data as it arrives in your topics:
Pipelinepage, click on the arrow coming out of the
New York Bikesservice. If there is data being emitted this arrow is green, otherwise it is gray.
Explore live dataon the context menu.
Select a stream from the streams listed under
If there are no streams under
SELECT STREAMS, wait a few moments, the New York CitiBike API is queried every few seconds.
Select the parameters in the
SELECT PARAMETERS OR EVENTSlist.
Tabletab in the top middle of the page.
After a few moments you will see data being shown in the table.
If you don't see any
parameters or data, just wait a moment or two. The next time data arrives these will be populated automatically.
Now you know how to observe data arriving into your topics. You can also explore the
Waveform tab to see numeric data in a graphical form and the
Messages tab to see the raw, JSON format, messages.
In order to train a machine learning model on historical data, the live real-time data being ingested needs to be stored. However, topics are real time and therefore not designed for data storage. To solve this, Quix enables you to store the data going through a topic to an efficient real-time database if you need it.
Enable persistence on your topics:
Navigate to the
Topicspage using the left-hand navigation.
Locate the topic(s) you want to store data for (in this case
For each topic, click the toggle in the
Finally, go back to the
Now, to ensure there is some historical data stored in the weather topic, stop and then start the
VisualCrossing Weatherservice. This will force the service to collect and publish fresh data to the
weather-topicwithout waiting for 30 minutes.
You will need this historical data in the next section, where you will learn how to retrieve data for training a model.