What are the key trends in data in 2023?
As we move into 2023, I am very excited to see all of the predictions for data and analytics and what they mean to Kensu.
I looked at different publications and spoke with various industry experts and analysts to see if there were any conclusions we could draw.
Almost all publications reference data observability in some shape or form as becoming increasingly important in 2023. Forbes states “As more and more organizations are becoming data-centric, the principles of Agile for application development and monitoring the operational health of applications will be relevant to data as well…”, this really underlines the importance of data observability and, in particular, the approach, first described by Andy Petrella in the O’Reilly publication “What is Data Observability?” and blog. VentureBeat states explicitly that “Data observability will become a critical industry” - to quote Shadi Rostami, SVP of engineering at Amplitude “...I predict we’ll increasingly see engineers and data teams becoming facilitators of enabling companies to make data-driven decisions…”
So data observability itself is to become a critical pillar of organizations' data architectures. Which does beg the question why?
This is more interesting and nuanced. If we look at the top four predictions from VentureBeat
1. CIOs will look to consolidate data and simplify the architecture.
From a Kensu perspective, If you are consolidating your architecture, that implies moving data around, which in turn implies more data pipelines, which means an increased need to maintain the quality of the data and hence an increased need for observability.2. Broader adoption of data contracts.
Data contracts are an interesting concept, often spoken about. I recently came across an article in data science central “Data Contracts — ensure robustness in your data mesh architecture” by Piethein Strengholt from Microsoft. In this article, he touches on the importance of the contract in relation to the data mesh but also on the role of the contract in setting standards for managing your pipelines with confidence. The flip side to this is what happens if the contract is broken, how and when do you find out? You can wait for an issue to occur and cause problems, by observing the downstream data, or you can observe your data pipelines as they execute, and put in place preventative measures, for example, a circuit breaker to mitigate risks and issues with failed contracts.3. Availability will be the key to winning in 2023.
This is pretty obvious, and you need to make sure that all aspects of your data landscape are supported and available. This is not just about resilience and redundancy in your databases or making sure you have coverage across multiple AWS regions. Availability is also about the end-to-end data pipelines. The Kensu data observability platform uniquely places agents into the data applications and monitors them for the health of both the application and the data helping you maintain and improve data availability.4. 2023 will be the year of the data app.
“In the past 10 years, we’ve seen the rise of the web app and the phone app, but 2023 is the year of the data app. Reliable, high-performing data applications will be a critical tool for success… “ - Dhruba Borthakur, cofounder and CTO at Rockset. From a Kensu perspective, this underlines some of the hard decisions we have taken in terms of our product strategy and how the Kensu platform functions. The way that Kensu implements data observability and in particular the method we deploy agents, allows organizations to place data observability directly into the data applications rather than the inputs or outputs. This means that Kensu can predict, monitor, and alert on the health not only of the data but the data app itself.To conclude here at Kensu we are very excited by the prospects of 2023 and the direction the data industry is taking. We have worked hard to get to where we are to be able to provide a unique and strong data observability platform that focuses on the areas that matter.
Roll on more success in 2023!