Chromia
Start Building

07 / 15Module 2: Running the data pipelineAdvanced

Module 2: Running the data pipeline

With your environment set up, you can move on to preparing the movie data and getting it onto the chain.

This module walks you through downloading the dataset, cleaning and normalizing its text, and creating embeddings with the model you picked earlier. You then upload the vectors, together with the complete movie metadata, to the Chromia chain you deployed.

When you finish, your on-chain database will contain the data that semantic search needs.

Make sure vector_demo_env is active

From inside the rell/ folder, run:

bash
cd ../python

From the project root, run this instead:

bash
cd python

Next, activate the environment:

bash
source vector_demo_env/bin/activate

In this lesson

  1. Preprocess movie data
  2. Generate sentence embeddings
  3. Store vectors on the blockchain
Begin lesson →
Written for this site from Chromia’s public course. Code samples are the course’s own.