Skip to content

colnade_polars

Polars backend adapter, I/O functions, and dtype mapping.

PolarsBackend

PolarsBackend

Colnade backend adapter for Polars.

translate_expr(expr)

Recursively translate a Colnade AST node to a Polars expression.

validate_schema(source, schema)

Validate that a Polars DataFrame/LazyFrame matches the schema.

validate_field_constraints(source, schema)

Validate value-level constraints (Field(), @schema_check) on data.

to_arrow_batches(source, batch_size)

Convert a Polars DataFrame to an iterator of Arrow RecordBatches.

from_arrow_batches(batches, schema)

Reconstruct a Polars DataFrame from Arrow RecordBatches.

from_dict(data, schema)

Create a Polars DataFrame from a columnar dict with schema-driven dtypes.

Construction Functions

from_dict(schema, data)

Create a typed DataFrame from a columnar dict.

The schema drives dtype coercion — plain Python values ([1, 2, 3]) are cast to the correct native types (e.g. UInt64).

from_rows(schema, rows)

Create a typed DataFrame from Row[S] instances.

The type checker verifies that rows match the schema — passing Orders.Row where Users.Row is expected is a static error.

Example::

df = from_rows(Users, [
    Users.Row(id=1, name="Alice", age=30, score=85.0),
    Users.Row(id=2, name="Bob", age=25, score=92.5),
])

I/O Functions

read_parquet(path, schema)

Read a Parquet file into a typed DataFrame.

scan_parquet(path, schema)

Lazily scan a Parquet file into a typed LazyFrame.

read_csv(path, schema, **kwargs)

Read a CSV file into a typed DataFrame.

scan_csv(path, schema, **kwargs)

Lazily scan a CSV file into a typed LazyFrame.

write_parquet(df, path)

Write a DataFrame to a Parquet file.

write_csv(df, path, **kwargs)

Write a DataFrame to a CSV file.

Dtype Mapping

map_colnade_dtype(colnade_type)

Map a Colnade dtype annotation to a Polars DataType.

Handles: - Concrete types (UInt64 → pl.UInt64) - Nullable unions (UInt64 | None → pl.UInt64) — Polars handles nulls natively - Struct[S] → pl.Struct with recursively mapped fields - List[T] → pl.List with recursively mapped element type

map_polars_dtype(pl_dtype)

Map a Polars DataType instance to a Colnade dtype class.