Tag:data engineering
Articles tagged "data engineering".
- 31 MIN READ•Sep 10, 2026
Migrating Into Iceberg Without Moving Data
add_files, snapshot, and migrate compared: the three in-place paths into Iceberg, the reconciliation each requires, the layout traps, and the rollback story.
Apache Icebergmigrationmetadata - 31 MIN READ•Sep 10, 2026
What Iceberg Table Maintenance Actually Costs
A cost model for compaction, snapshot expiry, orphan cleanup, and manifest rewriting: what each operation spends, on which meter, and how to set a schedule.
Apache Icebergtable maintenancecompaction - 24 MIN READ•Sep 2, 2026
Will AI Replace Data Engineers?
What the evidence shows about whether AI replaces data engineers, which parts of the job compress, and which parts do not.
AIData EngineeringCareers - 31 MIN READ•Aug 24, 2026
Iceberg Is Becoming a Library, Not Just a Table Format
Iceberg is turning from a JVM table format into a library other systems embed. What that shift changes for engines, catalogs, and the spec itself.
Apache Iceberglibrariesecosystem - 31 MIN READ•Aug 19, 2026
Zero-Copy Warehouse Modernization: Moving to Apache Iceberg Without Downtime
A practical guide to modernizing a data warehouse to Apache Iceberg without downtime, using federation first, then redirecting new data, then.
Apache Icebergwarehouse migrationzero-copy - 31 MIN READ•Aug 2, 2026
Apache Polaris 1.7.0 and the Quiet Work of Making a Catalog Trustworthy
Apache Polaris 1.7.0 deep dive: idempotent writes, semantic models, stricter credential vending, orphan cleanup, and what the upgrade asks of you.
Apache PolarisApache IcebergCatalog - 31 MIN READ•Aug 2, 2026
Designing Batch Pipelines That Write Well Into Apache Iceberg
How to design batch pipelines that write well into Apache Iceberg: commit strategy, partitioning, sort order, write-audit-publish, and maintenance done right.
Apache IcebergData EngineeringBatch Pipelines - 31 MIN READ•Jul 28, 2026
Running an Apache Iceberg Lakehouse With No Internet Connection
A practical guide to deploying an Iceberg lakehouse in air-gapped environments: component choices, artifact pipelines, identity without a cloud.
Apache IcebergAir-GappedData Engineering - 31 MIN READ•Jul 28, 2026
Why Iceberg V4 Wants to Retire Equality Deletes, and What Streaming Teams Should Do About It
Equality deletes made streaming upserts into Iceberg practical at the cost of read performance.
Apache IcebergStreamingData Engineering - 31 MIN READ•Jul 28, 2026
Surviving Commit Conflicts When Dozens of Writers Hit the Same Iceberg Table
Commit conflicts multiply with writer count, and AI agents introduce unpredictable write patterns.
Apache IcebergConcurrencyData Engineering - 31 MIN READ•Jul 28, 2026
The Parquet Versioning Problem, and Why Iceberg Cares About It
Parquet files have a version field that doesn't reliably signal feature requirements. A new versioning discipline is coming, borrowing from Iceberg's.
ParquetApache IcebergData Engineering - 31 MIN READ•Jul 25, 2026
The Apache Iceberg Market in the Middle of 2026
A survey of the Apache Iceberg market in July 2026: the state of the specification, platform support, the acquisition wave, the catalog contest, and how.
apache iceberglakehousedata engineering