# Preprocess movie data · Semantic search on Chromia using Vector DB · Learn Chromia

> You will download a real-world movie dataset and turn it into a clean, structured CSV file. The next step embeds the plot summaries from this file as…

Source: https://chromia.com/learn/vector-db-movie-demo/data-pipeline/preprocess/

Semantic search on Chromia using Vector DBLesson 8 of 15

1. [01Searching movies semantically on Chromia](https://chromia.com/learn/vector-db-movie-demo/introduction/)

Module 1: Project setup

1. [02Overview](https://chromia.com/learn/vector-db-movie-demo/setup/)
2. [03Choose an embedding model](https://chromia.com/learn/vector-db-movie-demo/setup/embedding-model/)
3. [04Configure your Rell module](https://chromia.com/learn/vector-db-movie-demo/setup/configure.rell/)
4. [05Deploy your Rell module](https://chromia.com/learn/vector-db-movie-demo/setup/deploy-rell-module/)
5. [06Finalize your Python environment](https://chromia.com/learn/vector-db-movie-demo/setup/finalize-python-env/)

Module 2: Running the data pipeline

1. [07Overview](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/)
2. [08Preprocess movie data](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/preprocess/)
3. [09Generate sentence embeddings](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/generate-embeddings/)
4. [10Store vectors on the blockchain](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/upload-vectors/)

1. [11Search the vector database](https://chromia.com/learn/vector-db-movie-demo/search/)

Module 4: A deep dive into the code

1. [12Overview](https://chromia.com/learn/vector-db-movie-demo/code-deep-dive/)
2. [13Link your data to the Vector DB with Rell](https://chromia.com/learn/vector-db-movie-demo/code-deep-dive/rell-interface/)
3. [14Work with Chromia through the Python client](https://chromia.com/learn/vector-db-movie-demo/code-deep-dive/python-client/)

1. [15Use cases and extensions](https://chromia.com/learn/vector-db-movie-demo/use-cases/)

[Learn](https://chromia.com/learn/)[Semantic search on Chromia using Vector DB](https://chromia.com/learn/vector-db-movie-demo/)

08 / 15Module 2: Running the data pipelineAdvanced

# Preprocess movie data

You will download a real-world movie dataset and turn it into a clean, structured CSV file. The next step embeds the plot summaries from this file as vectors. The remaining metadata is used to display search results.

## 1. Download the dataset

From the `python/` folder, run:

bash

```nolines
python download_data.py
```

The script downloads the [CMU Movie Summary Corpus](https://www.cs.cmu.edu/~ark/personas/) and unpacks it into a `MovieSummaries/` folder at the **root of your project**.

## 2. Preprocess the data

Once the download finishes, run:

bash

```nolines
python preprocess.py
```

The script loads the raw plot summaries and the movie metadata, joins them, and writes the cleaned result to:

* `data/movie_data.csv`

## What’s in the dataset?

Every row of the cleaned CSV has these fields:

* `wiki_id`: unique movie identifier that ties the data together
* `title`: name of the movie
* `plot`: complete plot summary, which is the input for vector embedding
* `release_date`
* `box_office`
* `runtime`
* `languages`
* `countries`
* `genres`

The `plot` field is converted to a vector and stored on-chain together with the movie metadata. This gives you a decentralized system that supports semantic search.

## What’s next?

Next, you will create sentence embeddings from the movie plots with the model you chose earlier.

Written for this site from Chromia’s public course. Code samples are the course’s own.

[← Previous**Module 2: Running the data pipeline**](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/)[Next →**Generate sentence embeddings**](https://chromia.com/learn/vector-db-movie-demo/data-pipeline/generate-embeddings/)

On this page

* [1. Download the dataset](#1-download-the-dataset)
* [2. Preprocess the data](#2-preprocess-the-data)
* [What’s in the dataset?](#whats-in-the-dataset)
* [What’s next?](#whats-next)