No code sentiment analysis

This tutorial shows how to build a data processing pipeline without code. You’ll analyze tweets that contain information about Bitcoin and stream both raw and transformed data into Snowflake, a storage platform, using the Twitter, HuggingFace and Snowflake connectors.

I’ve made a video of this tutorial if you prefer watching to reading.

Step one: create a database

Sign in to your Snowflake account to create the Snowflake database which will receive your data. Call this "demodata" and click "Create."

Step two: get your data

In Quix, click into the library and search for the Twitter source connector.

Click "Add new." This adds the source to your pipeline and brings you back to the library.

Fill in the necessary fields:

  • Name: Twitter Data - Source

  • Output: twitter-data

  • Twitter bearer token: paste your Twitter Dev token here

  • Twitter_search_paramaters: (#BTC OR btc #btc OR BTC)

Use search parameters to obtain Tweets on a subject that interests you! e.g. (#dolphins OR dolphins)

Click "Deploy"

image

Step three: transformation for sentiment analysis

  • Click the "Add transformation" button

  • In the library, search for "HuggingFace"

  • Click "Set up and deploy" on the HuggingFace connector

  • Choose "Twitter data" as the input topic

  • The output field should be set to "hugging-face-output" by default, leave this as it is or enter this if it’s not pre-populated.

  • Leave all other values with their defaults

  • Click "Deploy"

Find out more about the hugging face model and the other models you could use at huggingface.co

image

Step five: delivery to your Snowflake database

  • Click the "Add destination" button on the home screen

  • Search the library for the Snowflake connector

  • Click "Set up and deploy" on the connector

  • Fill in the necessary fields:

  • Choose the hugging-face-output output topic

  • The "Broker__TopicName" field should be set to "hugging-face-output". This means it will receive the data being output by the sentiment analysis model.

To fill in the Snowflake locator and region (these are similar to a unique ID for your Snowflake instance), navigate to your Snowflake account. Copy the locator and region from the URL and paste them into the corresponding fields in the connector setup in Quix. Lastly, input your username and password.

image

Click "Deploy" on the Snowflake connector. If the credentials and connection details are correct, you’ll see the "Connected" status in the log and will be redirected to your workspace.

image

Congratulations! You built a no-code pipeline that filters and collects data from Twitter, transforms it with a HuggingFace model and delivers it to a Snowflake database.

You can now go back over to Snowflake and find the "Databases" menu. Expand the "demodata" database and then find the tables under "public".

image

If you need help with this tutorial or want to chat about anything related to Quix or stream processing in general please come and say hi on The Stream, our slack community.