Skip to content
This site documents the development version of Orcapod (the main branch), which may differ from the latest release. For the releases, see PyPI and the changelog.

Types

Core type definitions used throughout Orcapod.

class · orcapod.types.Schema

class Schema(
fields: Mapping[str, DataType] | None = None,
optional_fields: Collection[str] | None = None,
**kwargs: type,
)

Bases: Mapping[str, DataType]

Immutable schema representing a mapping of field names to Python types.

Serves as the canonical internal schema representation in OrcaPod, with interop to/from Arrow schemas. Hashable and suitable for use in content-addressable contexts.

Parameters:

  • fields (Mapping[str, DataType] | None) default None: An optional mapping of field names to their data types.
  • **kwargs (type) default {}: Additional field name / type pairs. These are merged with fields; keyword arguments take precedence on conflict.

Example::

schema = Schema({"x": int, "y": float})
schema = Schema(x=int, y=float)

property · orcapod.types.Schema.optional_fields

Type: frozenset[str]

Field names that are optional (have a default value in the source function).

property · orcapod.types.Schema.required_fields

Type: frozenset[str]

Field names that must be present in an incoming data.

method · orcapod.types.Schema.is_required

def is_required(field: str) -> bool

Return True if field must be present (has no default).

method · orcapod.types.Schema.merge

def merge(other: Mapping[str, type]) -> Schema

Return a new Schema that is the union of self and other.

Parameters:

  • other (Mapping[str, type]): A mapping of field names to types to merge in.

Returns:

  • Schema: A new Schema containing all fields from both schemas.

Raises:

  • ValueError: If any shared field has a different type in other.

method · orcapod.types.Schema.with_values

def with_values(other: dict[str, type] | None, **kwargs: type) -> Schema

Return a new Schema with the specified fields added or overridden.

Unlike merge, this method silently overrides existing fields when a key already exists.

Parameters:

  • other (dict[str, type] | None): An optional mapping of field names to types.
  • **kwargs (type) default {}: Additional field name / type pairs.

Returns:

  • Schema: A new Schema with the updated fields.

method · orcapod.types.Schema.select

def select(*fields: str) -> Schema

Return a new Schema containing only the specified fields.

Parameters:

  • *fields (str) default (): Names of the fields to keep.

Returns:

  • Schema: A new Schema with only the requested fields.

Raises:

  • KeyError: If any of the requested fields are not present.

method · orcapod.types.Schema.drop

def drop(*fields: str) -> Schema

Return a new Schema with the specified fields removed.

Parameters:

  • *fields (str) default (): Names of the fields to drop. Fields not present in the schema are silently ignored.

Returns:

  • Schema: A new Schema without the dropped fields.

method · orcapod.types.Schema.is_compatible_with

def is_compatible_with(other: Schema) -> bool

Check whether other is a superset of this schema.

Parameters:

  • other (Schema): The schema to compare against.

Returns:

  • bool: True if other contains every field in self with a
  • bool: matching type.

method · orcapod.types.Schema.as_required

def as_required() -> Schema

Return a copy of this schema with all fields marked as required.

Strips optional-field metadata so the result reflects the structural (Arrow-level) schema, where every field is unconditionally present.

Returns:

  • Schema: A new Schema containing the same fields and types but with
  • Schema: optional_fields set to the empty frozenset.

classmethod · orcapod.types.Schema.empty

def empty() -> Schema

Create an empty schema with no fields.

Returns:

  • Schema: A new Schema containing zero fields.

class · orcapod.types.ColumnConfig

class ColumnConfig(
meta: bool | Collection[str] = False,
context: bool = False,
source: bool = False,
system_tags: bool = False,
content_hash: bool | str = False,
sort_by_tags: bool = False,
all_info: bool = False,
)

Configuration for column inclusion in DatagramProtocol/DataProtocol/TagProtocol operations.

Controls which column types to include when converting to tables, dicts, or querying keys/types.

Attributes:

  • meta (bool | Collection[str]): Include meta columns (with '__' prefix).
    • False: exclude all meta columns (default)
    • True: include all meta columns
    • Collection[str]: include specific meta columns by name (prefix '__' is added automatically if not present)
  • context (bool): Include context column
  • source (bool): Include source info columns (DataProtocol only, ignored for others)
  • system_tags (bool): Include system tag columns (TagProtocol only, ignored for others)
  • all_info (bool): Include all available columns (overrides other settings)

Examples:

>>> # Data columns only (default)
>>> ColumnConfig()
>>> # Everything
>>> ColumnConfig(all_info=True)
>>> # Or use convenience method:
>>> ColumnConfig.all()
>>> # Specific combinations
>>> ColumnConfig(meta=True, context=True)
>>> ColumnConfig(meta=["pipeline", "processed"], source=True)
>>> # As dict (alternative syntax)
>>> {"meta": True, "source": True}

classmethod · orcapod.types.ColumnConfig.all

def all() -> Self

Convenience: include all available columns

classmethod · orcapod.types.ColumnConfig.data_only

def data_only() -> Self

Convenience: include only data columns (default)

classmethod · orcapod.types.ColumnConfig.handle_config

def handle_config(config: Self | dict[str, Any] | None, all_info: bool = False) -> Self

Normalize column configuration input.

Parameters:

  • config (Self | dict[str, Any] | None): ColumnConfig instance or dict to normalize.
  • all_info (bool) default False: If True, override config to include all columns.

Returns:

  • Self: Normalized ColumnConfig instance. When all_info is set, by the
  • Self: keyword or inside config ({"all_info": True} or
  • Self: ColumnConfig(all_info=True)), every group is on, whatever else
  • Self: config says; an embedded all_info=True is not cancelled by a
  • Self: False keyword.

class · orcapod.types.ContentHash

class ContentHash(method: str, digest: bytes)

Content-addressable hash pairing a hashing method with a raw digest.

ContentHash is the standard way to represent hashes throughout OrcaPod. It is immutable (frozen dataclass) and provides convenience methods to convert the digest into hex strings, integers, UUIDs, base64, and human-friendly display names.

Attributes:

  • method (str): Identifier for the hashing algorithm / strategy used (e.g. "arrow_v2.1").
  • digest (bytes): The raw hash bytes.

method · orcapod.types.ContentHash.to_hex

def to_hex(char_count: int | None = None) -> str

Convert the digest to a hexadecimal string.

Parameters:

  • char_count (int | None) default None: If given, truncate the hex string to this many characters.

Returns:

  • str: The full (or truncated) hex representation of the digest.

method · orcapod.types.ContentHash.to_int

def to_int(hexdigits: int | None = None) -> int

Convert the digest to an integer.

Parameters:

  • hexdigits (int | None) default None: Number of hex digits to use. If provided, the hex string is truncated before conversion.

Returns:

  • int: Integer representation of the (optionally truncated) digest.

method · orcapod.types.ContentHash.to_uuid

def to_uuid(namespace: uuid.UUID = uuid.NAMESPACE_OID) -> uuid.UUID

Derive a deterministic UUID from the digest.

Uses uuid5 with the full hex string to ensure deterministic output.

Parameters:

  • namespace (uuid.UUID) default uuid.NAMESPACE_OID: UUID namespace for uuid5 generation. Defaults to uuid.NAMESPACE_OID.

Returns:

  • uuid.UUID: A UUID derived from this hash.

method · orcapod.types.ContentHash.to_base64

def to_base64() -> str

Convert the digest to a base64-encoded ASCII string.

Returns:

  • str: Base64 representation of the digest.

method · orcapod.types.ContentHash.to_string

def to_string(prefix_method: bool = True, hexdigits: int | None = None) -> str

Convert the digest to a human-readable string.

Parameters:

  • prefix_method (bool) default True: If True (the default), prepend the method name followed by a colon (e.g. "sha256:abcd1234").
  • hexdigits (int | None) default None: Optional number of hex digits to include.

Returns:

  • str: String representation of the hash.

method · orcapod.types.ContentHash.to_prefixed_digest

def to_prefixed_digest() -> bytes

Return the hash as method-prefixed raw bytes: b"{method}:{digest}".

Encodes the method name as ASCII, appends a literal : byte, then appends the raw digest bytes. The result is suitable for storage in a pa.large_binary() column and preserves the hash method for introspection without the hex-encoding overhead of to_string().

Example::

h = ContentHash("arrow_v2.1", b"\x00" * 32)
h.to_prefixed_digest()
# b"arrow_v2.1:\x00\x00...\x00"

Returns:

  • bytes: Raw bytes in the form b"{method}:{raw_digest}".

classmethod · orcapod.types.ContentHash.from_prefixed_digest

def from_prefixed_digest(data: bytes) -> 'ContentHash'

Parse method-prefixed raw bytes back into a ContentHash.

Inverse of to_prefixed_digest().

Parameters:

  • data (bytes): Bytes in the form b"{method}:{raw_digest}", as produced by to_prefixed_digest().

Returns:

  • 'ContentHash': A new ContentHash instance.

Raises:

  • ValueError: If data does not contain a colon separator.

classmethod · orcapod.types.ContentHash.from_string

def from_string(hash_string: str) -> 'ContentHash'

Parse a "method:hex_digest" string into a ContentHash.

Parameters:

  • hash_string (str): A string in the format "method:hex_digest".

Returns:

  • 'ContentHash': A new ContentHash instance.

method · orcapod.types.ContentHash.display_name

def display_name(length: int = 8) -> str

Return a short, human-friendly label for this hash.

Parameters:

  • length (int) default 8: Number of hex characters to include after the method prefix. Defaults to 8.

Returns:

  • str: A string like "arrow_v2.1:1a2b3c4d".