Skip to content
This site documents the development version of Orcapod (the main branch), which may differ from the latest release. For the releases, see PyPI and the changelog.

Part 0: Setup

About 10 minutes. At the end you have Orcapod installed, a check that it works, and the workshop dataset on disk.

Orcapod needs Python 3.11 or newer. Create a fresh environment for the workshop so that nothing else on your machine is affected.

With uv, the recommended way (uv init starts a project in the new directory, and uv add adds Orcapod to it):

Terminal window
mkdir orcapod-workshop && cd orcapod-workshop
uv init
uv add git+https://github.com/metamorphic-labs/orcapod-python

Without uv, with pip in a virtual environment:

Terminal window
mkdir orcapod-workshop && cd orcapod-workshop
python -m venv .venv
source .venv/bin/activate # on Windows: .venv\Scripts\activate
pip install git+https://github.com/metamorphic-labs/orcapod-python

Both install the development version from the repository: until 0.4.0 is released, PyPI has no current stable release (0.3.0 was withdrawn), so a plain uv add orcapod installs 0.1.4, which this tutorial does not cover. If your workshop organiser asked you to install a release candidate, name it instead: uv add "orcapod==0.4.0rc1". Once 0.4.0 is out, uv add orcapod (or pip install orcapod) installs the latest stable release.

The tutorial needs nothing beyond this. Orcapod already depends on pandas, polars and pyarrow, which the samples use.

Save this as check_install.py and run it with uv run python check_install.py (with pip, in the activated environment: python check_install.py):

from orcapod import PipelineJob, function_pod
from orcapod.databases import InMemoryArrowDatabase
from orcapod.sources import DictSource
@function_pod(output_keys="doubled")
def double(value: int) -> int:
return 2 * value
job = PipelineJob(name="install_check", database=InMemoryArrowDatabase())
with job:
numbers = DictSource(
data=[{"name": "a", "value": 1}, {"name": "b", "value": 2}],
tag_columns=["name"], tag_only_columns=True,
)
double.pod(numbers, label="double")
job.run()
print(job.nodes["double"].as_table().sort_by("name").to_pylist())
# [{'name': 'a', 'doubled': 2}, {'name': 'b', 'doubled': 4}]
print("Orcapod is working.")
# Orcapod is working.

If you see the two lines above, you are ready. You do not need to understand the script yet; every piece of it is introduced in the parts that follow.

Depending on what else is installed, you may see these on your screen. They are harmless.

  • UserWarning: Protobuf gencode version ... is exactly one major version older than the runtime version ..., several times, when Orcapod is imported.
  • WARN deltalake_core::writer::stats] Skipping column ... because it's a binary field., many times, whenever results are written to an on-disk database (from Part 5 on).

The dataset is generated by one short script that uses only the Python standard library. Download workshop_data.py into your working directory, or fetch it from the command line:

Terminal window
curl -fO https://raw.githubusercontent.com/metamorphic-labs/orcapod-python/main/docs/tutorial/workshop_data.py

Then run it:

Terminal window
python workshop_data.py

It prints Wrote 27 files (... kB) to .../workshop_data and creates the workshop_data/ directory described on the tutorial front page. The data is the same on every machine and every run, so your output will match these pages exactly.

Keep workshop_data.py in your working directory. Every part of the tutorial starts with these two lines, which generate the data if it is missing and give you its location:

from workshop_data import make_dataset
data_dir = make_dataset()
print(data_dir)
# workshop_data
print(sorted(p.name for p in data_dir.iterdir()))
# ['sessions', 'sessions.csv', 'subjects.csv', 'units.csv']

1. Open workshop_data/subjects.csv and workshop_data/sessions.csv in a text editor or with pandas.read_csv. Which column or columns identify a row in each file?

Solution

In subjects.csv, subject_id alone identifies a row. In sessions.csv you need both subject_id and session: every mouse has a session called s1.

import pandas as pd
sessions = pd.read_csv(data_dir / "sessions.csv")
print(sessions.head(3))
# subject_id session date duration_s
# 0 mouse_01 s1 2026-10-05 60.0
# 1 mouse_01 s2 2026-10-12 120.0
# 2 mouse_02 s1 2026-10-05 90.0
print(sessions["subject_id"].is_unique, sessions[["subject_id", "session"]].duplicated().any())
# False False

Hold on to this idea. Orcapod calls these identifying columns tags, and Part 1 starts from them.

2. How many spikes are in workshop_data/sessions/mouse_01_s1/spikes_imec0.csv?

Solution
import pandas as pd
spikes = pd.read_csv(data_dir / "sessions" / "mouse_01_s1" / "spikes_imec0.csv")
print(len(spikes))
# 1736

Next: Part 1: Your first stream.