What’s the best way to work with message sequence numbers to ensure order and completeness?

Our recommendation for handling/processing with message sequence is as follows: 

On beginning of day (one system event messages): 

Reset variable to 0; (processMessageNumber=0) 

On every other message: If(messageSequnceNumber <= processMessageNumber) { Ignore message; } 

Else If (messageSequnceNumber > processMessageNumber+1) { LOG WARN (GAP DATA); }

Else { processMessageNumber = messageSequnceNumber ProcessMessage(BusinessLogic) } 

We recommend this approach as Kafka (or any distributed system) can have dupes. We are working on reducing this from happening, but it can happen given the distributed nature of Kafka. 

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.