Public preview: Log compaction support in Azure Event Hubs

Azure Updates

Discussionpowered by zap.ms

Published

Dec 12, 2022

Status

In preview

Timeline

  • December 2022
    Preview

Tag

Microsoft Ignite

Description

Log compaction is a way of retaining events in Event Hubs. Rather using time based retention, you can use key-based retention mechanism where Event Hubs retrains the last known value for each event key of an event hub or Kafka topic. Event Hubs service runs a compaction job internally and purges old events in a compacted event hub. The partition key that you set with each event is used as the compaction key and users can also mark events that needs to be deleted from the event log by publishing event with a key and null payload. 

To learn more about log compaction, please check out Log Compaction documentation.