Real-time anything is the bees knees, but it is not always appropriate. Teams appreciate this, but in working with stakeholders there is always a bit of a dance between what they are looking for and what the systems and data required actually are capable of supporting. With these situations, solutions typically stay at the bookends:
- A very batch, time-bound set of solutions
- A data streaming solution
Instead of thinking ‘Real-time or bust’, teams should consider support of just-in-time. You may think these are just words and in the end don’t mean anything, but really what using this nomenclature does is set the stage for the proper solutions mindset. Your solutions should promote future-proofing and active listen/response.
Accessibility and availability are different and make sure your users know that
Accessibility is the ability to access data when you need it, on-demand. Availability is the ability to share new data. Your application must be able to be able to support both along with be able to notify consumers that data has changed.
Knowing your stale is just as important
Staleness is something that can be combatted in a couple of ways. First, if your sources of data don’t follow these principles, your application can ‘watch’ for source data changes and run accordingly. It is also important to publish a ‘usability’ data point (e.g. score, TTL) that tells your consumers when using this data becomes not valuable, or even worse, harmful.
In the end, don’t make anything wait for you
Bring it all together, your data application needs to respond as soon as it knows it needs to and provides its output just the same; think of it as hot potato. And as you design you applications, it is easier to slow down than it is to speed up. Make sure your technical choices support the greatest common factor from inside out and when you hit constraints with your dependencies, you can provide alternative interfaces.
Applications support just-in-time by making sure that as data moves around your environment quicker, they will always be ready to accept and process what is made available and provide any number of interfaces to provide output as needed. By leveraging these principles, you can stay away from the conversations about real-time being an all-or-nothing proposition and investment.