# Generate sentence embeddings · Semantic search on Chromia using Vector DB · Learn Chromia

> Here you turn the plot summaries in the cleaned movie dataset into vector embeddings, using the model you chose earlier in python/.env.

Source: https://chromia.com/learn/vector-db-movie-demo/data-pipeline/generate-embeddings/

Semantic search on Chromia using Vector DBLesson 9 of 15

1. [01Searching movies semantically on Chromia](https://chromia.com/learn/vector-db-movie-demo/introduction/)

Module 1: Project setup

1. [02Overview](https://chromia.com/learn/vector-db-movie-demo/setup/)
2. [03Choose an embedding model](https://chromia.com/learn/vector-db-movie-demo/setup/embedding-model/)
3. [04Configure your Rell module](https://chromia.com/learn/vector-db-movie-demo/setup/configure.rell/)
4. [05Deploy your Rell module](https://chromia.com/learn/vector-db-movie-demo/setup/deploy-rell-module/)
5. [06Finalize your Python environment](https://chromia.com/learn/vector-db-movie-demo/setup/finalize-python-env/)

Module 2: Running the data pipeline

1. [07Overview](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/)
2. [08Preprocess movie data](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/preprocess/)
3. [09Generate sentence embeddings](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/generate-embeddings/)
4. [10Store vectors on the blockchain](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/upload-vectors/)

1. [11Search the vector database](https://chromia.com/learn/vector-db-movie-demo/search/)

Module 4: A deep dive into the code

1. [12Overview](https://chromia.com/learn/vector-db-movie-demo/code-deep-dive/)
2. [13Link your data to the Vector DB with Rell](https://chromia.com/learn/vector-db-movie-demo/code-deep-dive/rell-interface/)
3. [14Work with Chromia through the Python client](https://chromia.com/learn/vector-db-movie-demo/code-deep-dive/python-client/)

1. [15Use cases and extensions](https://chromia.com/learn/vector-db-movie-demo/use-cases/)

[Learn](https://chromia.com/learn/)[Semantic search on Chromia using Vector DB](https://chromia.com/learn/vector-db-movie-demo/)

09 / 15Module 2: Running the data pipelineAdvanced

# Generate sentence embeddings

Here you turn the plot summaries in the cleaned movie dataset into vector embeddings, using the model you chose earlier in `python/.env`.

The script keeps each embedding together with its metadata, so the data is ready to be written to the blockchain afterward.

## Filter by box office revenue (optional)

Movies with low box office revenue are dropped by the script to keep processing manageable.

By default, the cutoff is $100,000,000, which leaves roughly 1,000 movies. To change it, edit this line in `python/.env`:

> BOX\_OFFICE\_THRESHOLD=100\_000\_000

* A lower value keeps **more** movies. For example, `0` gives you the full dataset of about 42,000.
* A higher value keeps **fewer** movies and makes processing faster. For example, `1_000_000_000` leaves only a few.

## Run the script

From the `python/` folder, run:

bash

```nolines
python vectorize.py
```

The script reads `data/movie_data.csv`, removes rows below the box office cutoff, converts each `plot` into a vector with the model set in `.env`, and writes the output to a `JSONL` file.

## Output file

You'll find the results here:

> `data/movie_vectors.jsonl`

Every line is a JSON object with:

* The full movie metadata, such as title, plot, and release date
* A `"vector"` field holding the plot embedding

You'll use this file to store the vectors and metadata on-chain in the following step.

## What’s next?

Next, you upload the vectors and movie metadata to your Chromia chain so you can run semantic searches against them.

Written for this site from Chromia’s public course. Code samples are the course’s own.

[← Previous**Preprocess movie data**](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/preprocess/)[Next →**Store vectors on the blockchain**](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/upload-vectors/)

On this page

* [Filter by box office revenue (optional)](#filter-by-box-office-revenue-optional)
* [Run the script](#run-the-script)
* [Output file](#output-file)
* [What’s next?](#whats-next)