Top 12 Apache Iceberg Interview Questions

Top 12 Apache Iceberg Interview Questions & Answers (2026) — Complete Guide

📅 Updated: June 15, 2026 ⏱ 10 min read 🎯 Beginner to Senior

📋 Table of Contents

Iceberg Fundamentals
Architecture & File Formats
Table Operations & ACID
Catalog & REST API
Performance & Optimization
Multi-Engine Integration
Advanced & Senior-Level Questions

1. Iceberg Fundamentals

Q1 What is Apache Iceberg and how does it differ from Hive tables?

Apache Iceberg is an open, high-performance table format for huge analytic datasets designed to work with compute engines like Spark, Trino, Flink, and DuckDB. Unlike traditional Hive tables — which rely on directory-level partitioning and a central metastore that becomes a bottleneck — Iceberg tracks every individual data file in a metadata tree. This enables:

Snapshot isolation: readers never block writers
Schema evolution: add, rename, or drop columns without rewriting data
Hidden partitioning: the engine handles partition transforms; queries need no partition predicate
Time travel: query any historical snapshot by timestamp or snapshot ID

In a Hive table, renaming a column breaks downstream queries. In Iceberg, columns have stable numeric IDs — a rename is a metadata-only operation and existing files are unaffected.

Q2 Explain the Iceberg metadata layer — what are manifest files and manifest lists?

Iceberg's metadata is a three-level hierarchy:

Metadata file (metadata.json) — the root. It records schema, partition spec, sort orders, and a list of all snapshots. The catalog stores only a pointer to the current metadata file.
Manifest list (one per snapshot) — a list of manifest files that belong to that snapshot, plus partition-level statistics so the engine can prune whole manifests without opening them.
Manifest file — lists individual data files (Parquet, ORC, Avro) with per-file statistics: null counts, lower/upper bounds per column. The engine uses these for data-skipping.

This hierarchy means a full table scan requires reading only the relevant manifest files rather than listing all S3 prefixes — a massive improvement over Hive's MSCK REPAIR TABLE pattern.

2. Architecture & File Formats

Q3 What file formats does Iceberg support, and which should you choose?

Iceberg is format-agnostic at the data layer but most production deployments use Apache Parquet. Here is a quick comparison:

Format	Best For	Typical Use
Parquet	Analytical reads, wide tables	Default for most lakehouses
ORC	Hive-heavy pipelines	Legacy Hive migration
Avro	Row-level streaming writes	Flink streaming ingest

Choose Parquet for OLAP workloads (Spark batch, Trino ad-hoc), Avro for high-throughput Flink streaming, and ORC if migrating from an existing Hive/ORC estate.

Q4 What are Iceberg's partition transforms and why are they important?

Iceberg supports hidden partitioning via transform functions applied to column values at write time. Users never write partition predicates in queries — the engine derives them automatically.

Common transforms:

-- Create a table partitioned by month extracted from event_time
CREATE TABLE warehouse.events (
    event_id  BIGINT,
    user_id   BIGINT,
    event_time TIMESTAMP,
    payload   STRING
)
USING iceberg
PARTITIONED BY (months(event_time));

-- Query without partition filter — engine auto-prunes
SELECT COUNT(*) FROM warehouse.events
WHERE event_time BETWEEN '2026-05-01' AND '2026-05-31';

Available transforms: identity, bucket(N), truncate(W), year, month, day, hour. Because transforms are stored in the spec, you can evolve partitioning without rewriting old data — a new spec is added for future writes while the old spec handles older files transparently.

3. Table Operations & ACID

Q5 How does Iceberg guarantee ACID transactions on object storage?

Object stores like S3 are not natively transactional, but Iceberg achieves ACID guarantees through optimistic concurrency control:

A writer reads the current metadata pointer from the catalog.
It creates a new snapshot (new manifest list + manifests + data files) in isolation.
It attempts a compare-and-swap (CAS) atomic update of the metadata pointer in the catalog. If another writer succeeded first, the CAS fails and the writer retries.
Readers always see a consistent snapshot — they are never exposed to partial writes.

This gives snapshot isolation (SI) — the strongest guarantee needed for analytical workloads — without any distributed lock manager.

Q6 Walk me through how you would perform a row-level DELETE or UPDATE in Iceberg.

Iceberg supports row-level mutations via two write modes:

Copy-on-Write (CoW): On any UPDATE/DELETE, affected data files are rewritten in full. Simple and produces clean Parquet files, but expensive for high-update workloads.
Merge-on-Read (MoR): Iceberg writes small delete files (positional or equality deletes) that record which rows to exclude. Readers merge data files with delete files at query time. Fast writes, slightly slower reads.

# PySpark example — GDPR right-to-erasure row deletion
from pyspark.sql import SparkSession

spark = SparkSession.builder \
    .config("spark.sql.extensions",
            "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions") \
    .config("spark.sql.catalog.warehouse", "org.apache.iceberg.spark.SparkCatalog") \
    .config("spark.sql.catalog.warehouse.type", "hadoop") \
    .config("spark.sql.catalog.warehouse.warehouse", "s3://my-bucket/warehouse") \
    .getOrCreate()

# Delete rows for a specific user (GDPR erasure)
spark.sql("""
    DELETE FROM warehouse.events
    WHERE user_id = 98765
""")

# Verify deletion via time travel — old snapshot still has the rows
spark.sql("""
    SELECT COUNT(*) FROM warehouse.events
    VERSION AS OF 5   -- previous snapshot ID
    WHERE user_id = 98765
""").show()

4. Catalog & REST API

Q7 What is the Iceberg REST Catalog and why has it become the standard in 2026?

The Iceberg REST Catalog specification defines a vendor-neutral HTTP API for catalog operations: create/list/load/drop namespaces and tables, commit transactions, and fetch credentials. Any engine that speaks the REST spec — Spark, Trino, Flink, DuckDB, StarRocks, PyIceberg — can connect to any compliant catalog (Polaris, Unity Catalog, AWS Glue, Nessie, Tabular) without engine-specific drivers.

Before the REST spec, each engine needed a custom Hive Metastore client or a vendor SDK. Now one URL is enough:

# Configure PyIceberg to use a REST catalog (e.g. Polaris / Snowflake Open Catalog)
from pyiceberg.catalog import load_catalog

catalog = load_catalog(
    "prod",
    **{
        "type": "rest",
        "uri": "https://my-catalog.example.com/iceberg",
        "credential": "client_id:client_secret",
        "warehouse": "s3://my-bucket/warehouse",
    }
)

# List namespaces
print(catalog.list_namespaces())

# Load table and scan with predicate pushdown
table = catalog.load_table("analytics.events")
scan = table.scan(row_filter="event_time >= '2026-06-01'")
df = scan.to_arrow()          # returns a PyArrow Table
print(df.schema)

The REST Catalog is the reason Iceberg can serve as a universal "data sharing" layer — one table, many engines, no data duplication.

Q8 How does Iceberg handle time travel and what are its practical uses?

Every write to an Iceberg table produces an immutable snapshot with a unique ID and timestamp. You can query any snapshot by ID or timestamp:

-- Query data as of a specific timestamp (audit / debugging)
SELECT * FROM prod.warehouse.orders
TIMESTAMP AS OF '2026-06-14 08:00:00';

-- Query by explicit snapshot ID
SELECT * FROM prod.warehouse.orders
VERSION AS OF 3891042;

-- Compare yesterday vs today to detect data drift
SELECT 'today'    AS period, COUNT(*) FROM prod.warehouse.orders
UNION ALL
SELECT 'yesterday', COUNT(*) FROM prod.warehouse.orders
TIMESTAMP AS OF CURRENT_TIMESTAMP - INTERVAL '1' DAY;

Practical uses: regulatory audits, ML training reproducibility (always train on the same snapshot), debugging pipelines that wrote incorrect data, and rollback via CALL catalog.system.rollback_to_snapshot().

5. Performance & Optimization

Q9 What is compaction in Iceberg and how do you run it?

Streaming ingest (Flink, Kafka) produces thousands of tiny files, degrading query performance. Compaction (also called rewriting data files) merges small files into optimally sized Parquet files (typically 128–512 MB) without changing table data or breaking ongoing readers.

-- Spark SQL: compact files smaller than 128 MB into 256 MB target files
CALL prod.system.rewrite_data_files(
    table => 'analytics.events',
    strategy => 'binpack',
    options => map(
        'target-file-size-bytes', '268435456',   -- 256 MB
        'min-file-size-bytes',    '134217728'    -- 128 MB threshold
    )
);

-- Also compact the manifests themselves for faster planning
CALL prod.system.rewrite_manifests('analytics.events');

Run compaction on a schedule (nightly or hourly for high-frequency tables). In 2026, managed services like Snowflake Open Catalog and Databricks Unity Catalog auto-compact transparently.

Q10 How does Iceberg's data skipping work, and how can you maximise it?

Each manifest file stores column-level statistics for every data file it references: null count, lower bound, and upper bound. The query engine uses these to skip entire files before reading any bytes from S3.

To maximise skipping:

Sort data before writing: ORDER BY (event_time, user_id) groups similar values into the same files, tightening bounds.
Use Z-ordering: for multi-dimensional filtering (user_id AND country), Z-order interleaves sort keys so skipping works across both dimensions.
Choose low-cardinality partitions: daily or hourly partitioning creates fine-grained manifest-level pruning before file-level pruning even kicks in.

-- Write sorted for optimal data skipping
CALL prod.system.rewrite_data_files(
    table   => 'analytics.events',
    strategy => 'sort',
    sort_order => 'event_time ASC NULLS LAST, user_id ASC NULLS LAST'
);

6. Multi-Engine Integration

Q11 How would you set up a production pipeline where Flink writes to Iceberg and Trino reads from it?

This is a classic streaming-to-lakehouse pattern. Both engines connect to the same REST Catalog — Flink as the writer, Trino as the reader:

# Flink SQL — stream Kafka events into Iceberg
CREATE TABLE kafka_source (
    event_id   BIGINT,
    user_id    BIGINT,
    event_time TIMESTAMP(3),
    payload    STRING,
    WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND
) WITH (
    'connector' = 'kafka',
    'topic'     = 'user-events',
    'properties.bootstrap.servers' = 'broker:9092',
    'format'    = 'json'
);

CREATE TABLE iceberg_sink (
    event_id   BIGINT,
    user_id    BIGINT,
    event_time TIMESTAMP(3),
    payload    STRING
) WITH (
    'connector'               = 'iceberg',
    'catalog-name'            = 'prod',
    'catalog-type'            = 'rest',
    'uri'                     = 'https://catalog.example.com/iceberg',
    'warehouse'               = 's3://my-bucket/warehouse',
    'database-name'           = 'analytics',
    'table-name'              = 'events',
    'write.format.default'    = 'parquet',
    'write.target-file-size-bytes' = '134217728'
);

INSERT INTO iceberg_sink SELECT * FROM kafka_source;

-- Trino reads the same table (zero duplication)
SELECT
    date_trunc('hour', event_time) AS hour,
    COUNT(DISTINCT user_id)        AS dau
FROM iceberg.analytics.events
WHERE event_time >= NOW() - INTERVAL '24' HOUR
GROUP BY 1
ORDER BY 1;

Because both engines use the REST Catalog, Trino automatically sees newly committed Flink snapshots. Flink's MoR write mode keeps write latency low; Trino's vectorised reader handles merge-on-read efficiently at query time.

7. Advanced & Senior-Level Questions

Q12 How would you design a lakehouse for both ML training and real-time analytics on the same data?

This is the canonical multi-modal lakehouse design that dominates 2026 architectures:

Single Iceberg table as the source of truth. All ingest paths — batch ETL, Flink streaming, CDC from Postgres/MySQL — write to the same Iceberg table via the REST Catalog.
ML training uses snapshot IDs. Training jobs pin to a specific snapshot (VERSION AS OF <id>) so experiments are reproducible. MLflow or DVC can log the snapshot ID as a dataset artifact.
Real-time analytics use the latest snapshot. Trino or StarRocks read CURRENT snapshot, benefiting from continuous Flink commits.
Branching for staging/experimentation. Iceberg branches (introduced in the 0.14+ API) let data teams write experimental data to a branch without touching production snapshots — like git for data.
Agentic AI querying via REST. In 2026, AI agents use PyIceberg + the REST Catalog to autonomously discover schemas, run predicate-pushed scans, and feed embeddings into vector stores — all without copying data.

This architecture eliminates the classic "lambda architecture" complexity of maintaining a separate speed layer and batch layer — Iceberg's snapshot model unifies both.

✅ Key Takeaways

Iceberg's three-level metadata hierarchy (metadata → manifest list → manifest) enables fast file-level pruning and data skipping.
Hidden partitioning with transform functions means queries never need partition predicates — the engine handles it.
ACID is achieved via optimistic concurrency control and compare-and-swap catalog updates — no locks required.
Copy-on-Write vs. Merge-on-Read is the key write mode trade-off: CoW for read-heavy, MoR for write-heavy workloads.
The Iceberg REST Catalog is now the universal interface — one table, many engines, zero duplication.
Time travel enables ML reproducibility, regulatory auditing, and pipeline debugging out of the box.
Compaction and Z-ordering are production essentials — run them on a schedule for large, frequently updated tables.

Top 100 Interview Questions and Answers (2026) | With Explanations

🚀 Free Technical Interview Prep — 700+ Questions