SQE: Sovereign Query Engine

SQE is a Rust-based distributed SQL query engine for Apache Iceberg tables. It replaces a patched Trino fork with a purpose-built engine based on Apache DataFusion and iceberg-rust.

graph LR
    Client["JDBC / Flight SQL Client"] --> Coordinator
    Coordinator --> Worker1["Worker 1"]
    Coordinator --> Worker2["Worker 2"]
    Coordinator --> WorkerN["Worker N"]
    Worker1 --> S3["S3 / MinIO"]
    Worker2 --> S3
    WorkerN --> S3
    Coordinator --> Polaris["Polaris Catalog"]
    Coordinator --> Keycloak["Keycloak OIDC"]

Key Properties

• No service account. Every query runs as the authenticated user. The user’s bearer token passes through to the Polaris catalog (metadata) and on the write path. Read-path S3 access can use per-user, table-scoped credentials vended by the catalog ([catalog] access_delegation = "vended-credentials"); without it, reads use the configured [storage] credentials (see S3 credential vending).
• Arrow-native — columnar data flows from Parquet files through the entire query pipeline to the client. No row-based serialization anywhere.
• Iceberg-native — built on iceberg-rust, not a connector bolted onto a generic engine. Partition pruning, metadata caching, and Iceberg v3 support are first-class.
• Fine-grained security — row filters and column masks enforced at the logical plan level, before the optimizer runs. Invisible columns, transparent row filtering, no information leakage.
• MCP for AI clients. A Model Context Protocol endpoint ships in the coordinator binary, off by default. It acts as an OAuth resource server rather than a credential proxy: the assistant’s user token becomes the same per-user session Flight SQL uses, so a model reads exactly what its human can read. See Model Context Protocol.
• Rust performance — single binary, no JVM, no GC pauses, predictable memory usage, fast startup.

Quick Start (embedded, no server)

cargo install --path crates/sqe-cli
sqe-cli --embedded                    # ~/.sqe/warehouse persistent Iceberg catalog

sqe> SELECT * FROM '/data/sales.parquet' LIMIT 5;
sqe> SELECT * FROM read_csv('s3://bucket/orders.tsv.gz');
sqe> SELECT * FROM 'hf://datasets/squad/plain_text/train-00000-of-00001.parquet';

Full embedded reference: Using the CLI. DuckDB comparison: getsqe.com/compare/duckdb.

Quick Start (cluster mode)

# Build
cargo build --release --bin sqe-server --bin sqe-cli

# Start coordinator (sqe-server runs as coordinator by default)
SQE_CONFIG=sqe.toml ./target/release/sqe-server

# Connect
./target/release/sqe-cli --host localhost --port 50051

Project Status

SQE is production-ready for the data path against Apache Iceberg; the coordinator currently runs as a single replica (a single point of failure), and coordinator HA is on the roadmap (see Kubernetes & Helm). The cluster mode runs distributed (coordinator + stateless workers) with OIDC bearer-token passthrough, Polaris / Nessie / Glue / HMS / S3 Tables / JDBC / Hadoop catalogs, and 147/162 (90.7%) on the public Iceberg matrix rubric. The embedded mode (V8 through V12.1) adds DuckDB-style file-format TVFs (read_csv, read_json), HuggingFace hf:// URLs, and a single-binary CLI for laptop analytics.