Generate sentence embeddings

Here you turn the plot summaries in the cleaned movie dataset into vector embeddings, using the model you chose earlier in python/.env.

The script keeps each embedding together with its metadata, so the data is ready to be written to the blockchain afterward.

Filter by box office revenue (optional)

Movies with low box office revenue are dropped by the script to keep processing manageable.

By default, the cutoff is $100,000,000, which leaves roughly 1,000 movies. To change it, edit this line in python/.env:

BOX_OFFICE_THRESHOLD=100_000_000

• A lower value keeps more movies. For example, 0 gives you the full dataset of about 42,000.
• A higher value keeps fewer movies and makes processing faster. For example, 1_000_000_000 leaves only a few.

Run the script

From the python/ folder, run:

python vectorize.py

The script reads data/movie_data.csv, removes rows below the box office cutoff, converts each plot into a vector with the model set in .env, and writes the output to a JSONL file.

Output file

You'll find the results here:

data/movie_vectors.jsonl

Every line is a JSON object with:

• The full movie metadata, such as title, plot, and release date
• A "vector" field holding the plot embedding

You'll use this file to store the vectors and metadata on-chain in the following step.

What’s next?

Next, you upload the vectors and movie metadata to your Chromia chain so you can run semantic searches against them.