colnade_polars

Polars backend adapter, I/O functions, and dtype mapping.

PolarsBackend

PolarsBackend

Colnade backend adapter for Polars.

translate_expr(expr)

Recursively translate a Colnade AST node to a Polars expression.

validate_schema(source, schema)

Validate that a Polars DataFrame/LazyFrame matches the schema.

to_arrow_batches(source, batch_size)

Convert a Polars DataFrame to an iterator of Arrow RecordBatches.

from_arrow_batches(batches, schema)

Reconstruct a Polars DataFrame from Arrow RecordBatches.

I/O Functions

read_parquet(path, schema)

Read a Parquet file into a typed DataFrame.

scan_parquet(path, schema)

Lazily scan a Parquet file into a typed LazyFrame.

read_csv(path, schema, **kwargs)

Read a CSV file into a typed DataFrame.

scan_csv(path, schema, **kwargs)

Lazily scan a CSV file into a typed LazyFrame.

write_parquet(df, path)

Write a DataFrame to a Parquet file.

write_csv(df, path, **kwargs)

Write a DataFrame to a CSV file.

Dtype Mapping

map_colnade_dtype(colnade_type)

Map a Colnade dtype annotation to a Polars DataType.

Handles: - Concrete types (UInt64 → pl.UInt64) - Nullable unions (UInt64 | None → pl.UInt64) — Polars handles nulls natively - Struct[S] → pl.Struct with recursively mapped fields - List[T] → pl.List with recursively mapped element type

map_polars_dtype(pl_dtype)

Map a Polars DataType instance to a Colnade dtype class.