colnade_pandas

Pandas backend adapter, I/O functions, and dtype mapping.

PandasBackend

PandasBackend

Colnade backend adapter for Pandas.

Expression translation produces callables (df) -> Series | scalar since Pandas has no standalone lazy expression API.

translate_expr(expr)

Recursively translate a Colnade AST node to a callable (df -> result).

validate_schema(source, schema)

Validate that a Pandas DataFrame matches the schema.

validate_field_constraints(source, schema)

Validate value-level constraints (Field(), @schema_check) on data.

to_arrow_batches(source, batch_size)

Convert a Pandas DataFrame to an iterator of Arrow RecordBatches.

from_arrow_batches(batches, schema)

Reconstruct a Pandas DataFrame from Arrow RecordBatches.

from_dict(data, schema)

Create a Pandas DataFrame from a columnar dict with schema-driven dtypes.

Construction Functions

from_dict(schema, data)

Create a typed DataFrame from a columnar dict.

The schema drives dtype coercion — plain Python values ([1, 2, 3]) are cast to the correct native types (e.g. UInt64).

from_rows(schema, rows)

Create a typed DataFrame from Row[S] instances.

The type checker verifies that rows match the schema — passing Orders.Row where Users.Row is expected is a static error.

I/O Functions

read_parquet(path, schema, **kwargs)

Read a Parquet file into a typed DataFrame.

read_csv(path, schema, **kwargs)

Read a CSV file into a typed DataFrame.

Applies the schema's dtype mapping to ensure correct column types.

write_parquet(df, path, **kwargs)

Write a DataFrame to a Parquet file.

write_csv(df, path, **kwargs)

Write a DataFrame to a CSV file.

Dtype Mapping

map_colnade_dtype(colnade_type)

Map a Colnade dtype annotation to a Pandas dtype.

Handles: - Concrete types (UInt64 → pd.UInt64Dtype()) - Nullable unions (UInt64 | None → pd.UInt64Dtype()) — nullable extension types - List[T] / Struct[S] → object

map_pandas_dtype(pd_dtype)

Map a Pandas dtype to a Colnade dtype class.