If you are a software engineer, you would have heard about deduplication hundreds of times. Every system that wants to scale or focus on scaling should be interested in filtering out duplicate processing as early as and in the shortest possible time in the event processing pipeline. At Walmart Global Tech, we are no different for our enterprise rule engine platform that was built to optimize a specific event (we call it as override on rules) where we needed processing that was created and updated in many different ways by the user (e.g., UI/API/spread-sheets via REST or Kafka).

Assumptions:

  • If…

Trinath Patri

I’m a Sr. Software Engineer at Walmart Global Tech. I enjoy solving complex problems using data structures, algorithms and distributed systems.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store