Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

DDL

Data Definition Language: schemas, tables, views, columns, partitions, branches, tags. Most statements parse via sqlparser-rs; SQE adds branch / tag / partition-evolution syntax that sqlparser-rs does not natively understand.

Source: sqe-sql/src/classifier.rs, sqe-sql/src/ddl.rs, sqe-sql/src/partition.rs, sqe-sql/src/partition_evolution.rs. Coordinator handlers in crates/sqe-coordinator/src/catalog_ops.rs.

Schema

StatementOriginNotesTrinoSnowflakeSpark SQLDuckDB
CREATE SCHEMA [IF NOT EXISTS] cat.nssqlparser-rs + sqe-coordinatorCreates a namespace in the catalog.yesyesyesyes
CREATE SCHEMA [IF NOT EXISTS] cat.ns LOCATION 's3://...'sqlparser-rs + sqe-coordinatorOverride default location. Only on catalogs that accept location at namespace level (Polaris, S3 Tables).yesyesyes-
DROP SCHEMA [IF EXISTS] cat.ns [CASCADE|RESTRICT]sqlparser-rs + sqe-coordinatorCASCADE drops contained tables.yesyesyesyes
ALTER SCHEMA cat.ns RENAME TO new_namesqlparser-rs + sqe-coordinatorCatalog must support namespace rename.partialyesyesyes
CREATE SCHEMA IF NOT EXISTS analytics.staging;
CREATE SCHEMA marketing LOCATION 's3://my-warehouse/marketing/';
DROP SCHEMA staging CASCADE;

Table creation

StatementOriginNotesTrinoSnowflakeSpark SQLDuckDB
CREATE TABLE t (col TYPE [DEFAULT expr], ...)sqlparser-rs + sqe-sql + sqe-coordinatorIceberg V3 column defaults supported.yesyesyesyes
CREATE TABLE t (...) PARTITIONED BY (transform(col), ...)sqe-sql/partition.rsPartition transforms: bucket(N, col), truncate(N, col), year(col), month(col), day(col), hour(col), identity (just col).partial-yes-
CREATE TABLE t AS SELECT ... (CTAS)sqlparser-rs + sqe-coordinatorInferred schema; partitioning via WITH (partitioning = ARRAY['day(ts)']).yesyesyesyes
CREATE OR REPLACE TABLE t AS SELECT ...sqlparser-rs + sqe-coordinatorAtomic replace. New snapshot replaces the table; old data files retained until expire_snapshots.yesyespartialyes
CREATE TABLE [IF NOT EXISTS] t LIKE other_tablesqlparser-rs + sqe-coordinatorCopy schema only, no data.yesyesyesyes
CREATE TABLE analytics.events (
    id          BIGINT,
    user_id     BIGINT,
    event_type  VARCHAR,
    occurred_at TIMESTAMP(6),
    payload     JSON,
    region      VARCHAR DEFAULT 'unknown'
)
PARTITIONED BY (day(occurred_at), bucket(16, user_id));

CREATE TABLE analytics.daily_events AS
SELECT day(occurred_at) AS d, count(*) AS n
FROM analytics.events GROUP BY 1;

Schema evolution

StatementOriginNotesTrinoSnowflakeSpark SQLDuckDB
ALTER TABLE t ADD COLUMN c TYPE [DEFAULT expr]sqlparser-rs + sqe-coordinatorNew column. Existing rows get the default (V3) or NULL (V2).yesyesyesyes
ALTER TABLE t DROP COLUMN [IF EXISTS] csqlparser-rs + sqe-coordinatorLogical drop. Field id retained in old data files.yesyesyesyes
ALTER TABLE t RENAME COLUMN old TO newsqlparser-rs + sqe-coordinatorIceberg field id stays the same; only the name changes.yesyesyesyes
ALTER TABLE t ALTER COLUMN c SET NOT NULLsqlparser-rs + sqe-coordinatorTighten nullability. Fails if existing rows have NULL.yesyesyesyes
ALTER TABLE t ALTER COLUMN c DROP NOT NULLsqlparser-rs + sqe-coordinatorLoosen nullability.yesyesyesyes
ALTER TABLE t ALTER COLUMN c SET DEFAULT exprsqlparser-rs + sqe-coordinatorIceberg V3 column default.partialyesyesyes
ALTER TABLE t ALTER COLUMN c TYPE new_typesqlparser-rs + sqe-coordinatorType promotion only (e.g. INT -> BIGINT). Lossy changes rejected.partialpartialpartialpartial
ALTER TABLE t RENAME TO new_tsqlparser-rs + sqe-coordinatorCatalog rename. Different catalog support varies.yesyesyesyes
ALTER TABLE t SET TBLPROPERTIES (...)sqlparser-rs + sqe-coordinatorSet Iceberg properties (e.g. write.delete.mode).yesyesyes-
COMMENT ON TABLE t IS 'description'sqlparser-rs + sqe-coordinatorStored in Iceberg properties.yesyesyesyes
COMMENT ON COLUMN t.c IS 'description'sqlparser-rs + sqe-coordinatorStored on the column metadata.yesyesyesyes
ALTER TABLE analytics.events ADD COLUMN device VARCHAR DEFAULT 'unknown';
ALTER TABLE analytics.events DROP COLUMN IF EXISTS deprecated_field;
ALTER TABLE analytics.events RENAME COLUMN payload TO body;
ALTER TABLE analytics.events ALTER COLUMN region SET NOT NULL;
ALTER TABLE analytics.events SET TBLPROPERTIES (
    'write.delete.mode' = 'merge-on-read',
    'write.parquet.bloom-filter-columns' = 'user_id,event_id'
);

Partition evolution (SQE / Iceberg-specific)

Iceberg lets you change partition spec without rewriting data. SQE parses these in sqe-sql/src/partition_evolution.rs because sqlparser-rs only knows Hive-style PARTITION (col = val).

StatementNotesTrinoSnowflakeSpark SQL
ALTER TABLE t ADD PARTITION FIELD transform(col)Add a new partition field. Existing data stays in the old spec.partial-yes
ALTER TABLE t ADD PARTITION FIELD transform(col) AS aliasSame with explicit name for the partition column.--yes
ALTER TABLE t DROP PARTITION FIELD transform(col)Remove a partition field from the current spec.partial-yes
ALTER TABLE t REPLACE PARTITION FIELD old_transform(col) WITH new_transform(col)Replace one transform with another.--yes
-- Originally partitioned by day(ts); switch to hour() for finer granularity.
ALTER TABLE events REPLACE PARTITION FIELD day(occurred_at) WITH hour(occurred_at);

-- Add a bucketing field on top of existing daily partitions.
ALTER TABLE events ADD PARTITION FIELD bucket(64, user_id);

Branches and tags (SQE / Iceberg-specific)

Iceberg branches are named pointers to a snapshot, like git branches. Tags are immutable named pointers. SQE parses these in sqe-sql/src/ddl.rs.

StatementNotes
ALTER TABLE t CREATE BRANCH nameNew branch from current snapshot.
ALTER TABLE t CREATE BRANCH name AS OF VERSION snapshot_idNew branch from a specific snapshot.
ALTER TABLE t CREATE BRANCH name WITH RETENTION (max_ref_age_ms = N)Auto-expire branch after N ms of inactivity.
ALTER TABLE t CREATE [OR REPLACE] TAG nameNew tag pointing at current snapshot. OR REPLACE is allowed because tags are not strictly immutable in iceberg-rust.
ALTER TABLE t CREATE TAG name AS OF VERSION snapshot_idTag a specific snapshot.
ALTER TABLE t DROP BRANCH [IF EXISTS] nameRemove a branch.
ALTER TABLE t DROP TAG [IF EXISTS] nameRemove a tag.
-- Branch a snapshot for development work
ALTER TABLE analytics.events CREATE BRANCH dev_2026_05;

-- Pin a known-good snapshot as a tag
ALTER TABLE analytics.events CREATE TAG release_2026_q2 AS OF VERSION 8472810294831234567;

-- Query the branch
SELECT * FROM analytics.events FOR VERSION AS OF 'dev_2026_05';

Views

StatementOriginNotesTrinoSnowflakeSpark SQLDuckDB
CREATE [OR REPLACE] VIEW v AS SELECT ...sqlparser-rs + sqe-coordinatorStandard SQL view. Iceberg views format-version 1.yesyesyesyes
CREATE [OR REPLACE] VIEW v (col1, col2) AS SELECT ...sqlparser-rs + sqe-coordinatorExplicit column list.yesyesyesyes
DROP VIEW [IF EXISTS] vsqlparser-rs + sqe-coordinatorRemove a view.yesyesyesyes
CREATE VIEW v COMMENT '<text>' AS ...sqe-sql view_compat + sqe-coordinatorTrino’s comment spelling, without =. Stored as the view’s comment property.yesnonono
CREATE VIEW v SECURITY {DEFINER | INVOKER} AS ...sqe-sql view_compat + sqe-coordinatorAccepted and recorded. See below: DEFINER is not enforced.yesnonono
CREATE OR REPLACE VIEW analytics.recent_events AS
SELECT * FROM analytics.events
WHERE occurred_at >= now() - INTERVAL '7' DAY;

DROP VIEW IF EXISTS analytics.recent_events;

SECURITY DEFINER is recorded, not enforced

Trino’s SECURITY DEFINER runs a view with its creator’s privileges, so a reader needs access to the view but not to its base tables. SECURITY INVOKER checks the querying user against the base tables instead.

SQE always behaves as INVOKER, and that is architectural rather than unfinished. Every query runs as the authenticated user via bearer-token passthrough, with no service account anywhere in the engine, so there is no credential to run a view as its definer with. Honouring DEFINER would mean storing one.

The clause is accepted because rejecting it blocks every dbt model that uses the default view materialization, and because the direction of the difference is safe: INVOKER is stricter than DEFINER asks for, so a reader who was meant to be shielded from base-table grants is denied rather than allowed. It fails closed.

What SQE does with it:

Recorded asview properties sqe.view-security and, for definer, sqe.view-definer
EnforcementINVOKER semantics: the base table is authorized as the querying user
On DEFINERone warning at creation, naming the reader-denial consequence

The properties live in the Iceberg view metadata, so the intent travels with the object and a later implementation, or another engine, can honour it.

Drop

StatementOriginNotes
DROP TABLE [IF EXISTS] t [PURGE]sqlparser-rs + sqe-coordinatorPURGE deletes data files immediately; default keeps the metadata so system.remove_orphan_files can clean later.
DROP VIEW [IF EXISTS] vsqlparser-rs + sqe-coordinatorStandard.
DROP SCHEMA [IF EXISTS] s [CASCADE|RESTRICT]sqlparser-rs + sqe-coordinatorCASCADE drops contained tables.

Iceberg V3 type system

These types only exist in format-version 3. Adding one to a CREATE TABLE auto-bumps the table to V3.

TypeNotes
TIMESTAMP_NS, TIMESTAMP_NS WITH TIME ZONENanosecond precision timestamps. Arrow Timestamp(Nanosecond, ...).
GEOMETRY, GEOGRAPHYStub types in V3; SQE accepts them in CREATE but does not yet provide spatial functions.
Default values via DEFAULT exprExisting rows in older snapshots inherit the default at read time.

What CREATE / ALTER does NOT cover

  • CREATE INDEX. Iceberg has no equivalent. Bloom filter columns and partition fields cover the same ground; configure via SET TBLPROPERTIES and ADD PARTITION FIELD.
  • CREATE FUNCTION / CREATE PROCEDURE. UDFs are Rust-side. SQL-defined functions and procedures are not supported.
  • CREATE SEQUENCE. no auto-increment / sequence support today. Use row_number() over a deterministic ordering for synthetic keys.
  • CREATE TYPE. no user-defined types. Use STRUCT<...> or MAP<...>.

These are tracked but not on the immediate roadmap.