SQE vs Trino, DuckDB, Spark

SQE is not trying to win every workload. It is built for one shape: query Apache Iceberg tables as the authenticated user, on your own hardware, with a single Rust binary and no JVM. This page is a decision aid. Where another engine is the better fit, it says so.

The detailed function-by-function matrices live on getsqe.com: compare/trino and compare/duckdb. The per-page comparison columns in the SQL Reference carry the same data at function granularity.

SQE vs Trino

Trino is the industry-standard SQL engine for lakehouses, and it is what SQE replaces in our own platform. The reasons we moved off it are specific:

Choose Trino when you need its breadth: a wide connector catalog beyond Iceberg, a mature Ranger-based security deployment you already run, or features SQE does not have. Trino is the stronger general-purpose engine.

Choose SQE when per-user identity to the catalog and storage matters, when JVM start time and GC pauses fight your autoscaling, or when maintaining a patched Trino fork for token passthrough is a cost you want to drop. See Why SQE for the full account of that migration, and GRANT and REVOKE for the security comparison.

A note on performance: SQE is faster than Trino 481 on all eight benchmark suites at SF1 (2026-10-10) and on six of seven at SF10 (2026-07-09), on identical Iceberg tables and S3 storage, with every query differentially validated. The exception is the bank suite at SF10, where spill degraded in the July run. SSB is at parity at SF10. The full table and method are on the benchmark page. Looking further out, SF100 is the open frontier (Roadmap, Phase 10). Measure on your workload.

SQE vs DuckDB

DuckDB is an in-process analytical engine. It is excellent on a single machine: a laptop, a notebook, an ETL step inside one process. SQE’s embedded mode borrows the same feeling (sqe-cli --embedded, quoted-string file reads, read_csv / read_json, hf:// URLs), so the file-level ergonomics overlap.

The split is about deployment and identity. DuckDB is a library you embed; there is no server, no per-user auth, no distributed execution, and no GRANT model (see the grant comparison, where DuckDB is “no” across the security row). SQE runs as a server with OIDC auth, distributes across workers, and is Iceberg-native with a Polaris-style REST catalog.

Choose DuckDB for single-machine analytics, embedding into an application, or fast local exploration where a server is overhead. It is the stronger tool there.

Choose SQE when you need multi-user access with per-user identity, a shared Iceberg catalog, distributed scans on data larger than one box, or a server other clients connect to. For the file-reading ergonomics on one machine, SQE’s embedded mode is the bridge.

SQE vs Spark

Spark SQL is the reference for large-scale batch and the breadth of the Spark ecosystem. It runs on the JVM, uses a service-account model for catalog and storage like Trino, and pushes fine-grained security to Ranger or Lake Formation at the connector boundary (see the grant comparison).

Choose Spark for very large batch transformation, when you are already invested in the Spark ecosystem (MLlib, structured streaming, the broad connector set), or for scale beyond what SQE has been validated against. Spark is the stronger heavy-batch engine.

Choose SQE for interactive and analytical SQL on Iceberg where per-user identity and a small operational footprint matter, and where a single Rust binary beats standing up a Spark cluster. SQE distributes (coordinator plus stateless workers) but is positioned for query serving, not as a general batch framework.

The through-line

SQE’s distinguishing property is sovereignty: every query runs with the identity and permissions of the user who submitted it, no service-account intermediary, on hardware you control. Trino, DuckDB, and Spark each beat SQE on some axis (breadth, single-machine simplicity, batch scale). None of them pass a per-user bearer token through to both the catalog and storage. That is the trade SQE is built around. See Why SQE.