2026
- Exactly-once from Kafka into ClickHouse
Kafka transactions stop at the topic. Getting the same guarantee into ClickHouse means making inserts repeatable and letting the database throw away retries.
- Iceberg maintenance: four jobs, in order
Compaction gets the attention, but snapshot expiry and orphan cleanup are what keep the storage bill flat. The order we run them in, and the settings we actually touch.
- Late data in Flink: allowed lateness or a side output?
Both keep late events from vanishing. One keeps window state around and re-fires results; the other hands the problem to you. Choose by who reads the output.
- ClickHouse materialized views are insert triggers
They run on each inserted block, not on the table. Once that clicks, the surprises around backfills, joins and deletes stop being surprises.
2025
- Partition for deletion, sort for reading
In ClickHouse and Iceberg alike, the partition key decides what you can drop cheaply and the sort key decides what you can read cheaply. Mixing them up is expensive.