> The only downside with what I have proposed is that there is no garbage collection or cleanup possible.
If you are using Kafka, you can use log compaction[0] to achieve this. It's also possible to delete individual entries with tombstone records, assuming your objects have a "key" attribute.
If you are using Kafka, you can use log compaction[0] to achieve this. It's also possible to delete individual entries with tombstone records, assuming your objects have a "key" attribute.
[0]https://kafka.apache.org/20/documentation.html#compaction