Big Data Storage

Parquet vs ORC vs Avro: Big Data File Formats Interview Guide (2026)

One of the most common interview topics for data engineers. Know when to use each format, the trade-offs, and how to choose for any scenario.

📅 Updated April 2026  |  ⏱ 10 min read  |  🎯 All Levels

📋 Formats Covered

1. Apache Parquet — The Analytics Champion
2. Apache ORC — Hive's Native Format
3. Apache Avro — The Streaming Choice
4. CSV — When to Avoid It
5. Quick Comparison Table
6. Scenario-Based Questions

Choosing the wrong file format can mean 10x slower queries and 5x higher storage costs. Interviewers ask this to see if you understand the fundamentals of how data is physically stored.

Parquet Columnar Storage — The Default Choice for Analytics Spark · Athena · BigQuery · Presto

Parquet stores data column by column rather than row by row. When you run SELECT revenue, date FROM sales on a 500-column table, Parquet reads only those 2 column files — ignoring all other 498 columns entirely. This makes analytical queries dramatically faster.

Key features:
• Predicate pushdown — Stores min/max statistics per row group. Spark can skip entire chunks without reading them based on your WHERE clause.
• Dictionary encoding — Repeated values (like country codes, status flags) are stored as integer lookups, not repeated strings.
• Nested data support — Natively handles arrays and nested structs (Dremel encoding).
• Schema embedded in footer — File is self-describing.

# Writing Parquet in PySpark
df.write.mode('overwrite') \
    .partitionBy('year', 'month') \
    .parquet('s3://my-bucket/sales/')

# Reading with column pruning (only reads 2 columns from disk)
spark.read.parquet('s3://my-bucket/sales/') \
    .select('revenue', 'date') \
    .filter('year = 2026')

✅ Use Parquet When

• Analytical workloads (OLAP)
• Reading a few columns from wide tables
• Spark, Presto, Athena, BigQuery
• Storing data lake tables

❌ Avoid Parquet When

• Streaming (Kafka) — row-by-row writes are expensive
• Frequent small appends
• Schema changes very often

ORC Optimized Row Columnar — Hive's Preferred Format Hive · Spark · Impala

ORC (Optimized Row Columnar) is also a columnar format, similar to Parquet. It was designed specifically for Hive and offers slightly better compression ratios and faster Hive queries, but Parquet is more universally supported across tools.

ORC vs Parquet key differences:
• ORC uses ACID transactions natively — supports UPDATE and DELETE in Hive (Parquet doesn't in Hive).
• ORC stores Bloom filters by default — faster equality lookups.
• Parquet has better ecosystem support — Spark, Arrow, Pandas, BigQuery all prefer Parquet.
• ORC has better Hive performance — if your stack is Hive-heavy, ORC is the native choice.

⚠️ Interview Answer "Which is better, Parquet or ORC?" — Answer: "It depends on the ecosystem. Pure Spark/cloud → Parquet. Hive-heavy or need ACID transactions → ORC." Interviewers don't want a "one size fits all" answer.

Avro Row-Based with Schema — The Streaming Champion Kafka · Schema Registry · Event Streaming

Avro is a row-based serialization format with an embedded JSON schema. It's the dominant format for Kafka messages because it writes one record at a time efficiently and supports robust schema evolution.

Key features:
• Schema evolution — Forward and backward compatible changes (add optional fields, remove fields with defaults).
• Schema Registry — In Kafka pipelines, Avro schemas are registered in Confluent Schema Registry so all producers/consumers share the same schema.
• Compact binary encoding — No field names repeated in each record (stored once in schema), making messages small.
• Row storage — Writing one record at a time is efficient (unlike Parquet which needs to buffer column data).

# Avro schema example (JSON format)
{
  "type": "record",
  "name": "OrderEvent",
  "fields": [
    {"name": "order_id", "type": "string"},
    {"name": "amount",   "type": "double"},
    {"name": "status",   "type": "string"},
    {"name": "discount", "type": ["null", "double"], "default": null}  ← optional field
  ]
}

✅ Use Avro When

• Kafka event streaming
• Schema evolution needed
• Write-heavy workloads
• Serializing Java/Python objects

❌ Avoid Avro When

• Analytical queries on specific columns
• Storing in a data lake for analytics
• Need human-readable format

Quick Comparison Table

Feature	Parquet	ORC	Avro	CSV
Storage type	Columnar	Columnar	Row	Row
Best for	Analytics (OLAP)	Hive / ACID	Streaming	Interchange only
Compression	Excellent	Best of all	Good	None built-in
Schema evolution	Limited	Limited	Excellent	None
Splittable	✅	✅	✅ (with sync marks)	✅ (if uncompressed)
Self-describing	✅ (footer)	✅ (footer)	✅ (header)	❌
Human readable	❌	❌	❌	✅
Kafka use	❌ Poor	❌ Poor	✅ Ideal	⚠️ Possible but inefficient

Scenario-Based Questions

Scenario 1 You're building a data lake on S3 for Athena. What format do you choose?

Answer: Parquet with Snappy compression, partitioned by date.

Athena is a columnar query engine — Parquet aligns perfectly. Partition by year/month/day so Athena can skip entire partitions for date-range queries. Snappy compression for a balance of speed and size. Expected cost savings vs CSV: 70–90% reduction in bytes scanned = direct cost reduction since Athena charges per TB scanned.

Scenario 2 Your team uses Kafka for order events and the schema changes frequently. What format for Kafka messages?

Answer: Avro with Confluent Schema Registry.

Avro's forward/backward compatible schema evolution means producers can add new optional fields without breaking existing consumers. Schema Registry enforces schema compatibility rules (BACKWARD, FORWARD, FULL) centrally — preventing incompatible schema changes from reaching production.

🗂️ Master More Big Data Interview Topics

Practice free interactive quizzes on Hadoop, Spark, SQL, Networking and DSA — and get the 300Q PDF bundle for your deepest prep.

Visit the Blog → Get the PDF Bundle