This Article is first appeared in Computerworld
There’s no dearth of data but it is vital to convert that data to action in a timely manner to drive desired business benefits. Despite colossal investments in big data management and advanced analytics, things have not reached too far from where it all started. The past actions can be measured with relative accuracy and hence future can be altered accordingly. But the real challenge begins when it comes to gathering deep, impactful, and timely insights – implementing them instantly, and learning from them at every step taken.
The technology that bought all the data is also acting as the biggest roadblock in achieving analytical excellence. This is where an insight platform comes in. An insight platform beautifully overcomes the challenges posed by technology that block the users from turning data into action.
How is an insight platform different?
BI practices today work in the silos of data management and analytics. Data management focuses on developing a central data repository or data lakes that hold a variety of data and then analytical tools work over that data in another layer. Translation of insights into actions happens outside the tools and is a time consuming and challenging process.
“An insight platform removes many of the hurdles that retailers faced with data and analytics and offers them an easier solution for their complex analytics needs.”
According to Forrester’s Global Business Technographics Data And Analytics Survey, 2015: “53 percent of data and analytics decision-makers say that it takes months for IT to make new customer data sets ready for self-service analytics or data science, severely limiting the new capabilities.”
An insight platform is very different from the current BI practices. Most importantly, it builds closed-loop systems that are directly embedded into business operations. This simplifies the process of connecting data to insights, and then connecting the insights with execution. It also makes it possible to create action and then link the results of that action back into the system to initiate continuous optimization. What it means for your business is that your users can get consistent and instant insights for making everyday decisions with confidence. Their actions are tracked and results are measured to optimize future decisions.
What does an insight platform contain?
Data ingestion systems to collect data continuously: An insight platform has pipelines to capture data from multiple applications and devices. The system absorbs data from varied sources, frequencies and formats. The platform captures data in real time, processes it, examines it, and generates insights. For example, inventory transactions are fed into the system in real-time and the on-hand-stock quantity is computed instantly for supporting the omnichannel functions.
Compute engines to generate insights: A system of insight has the computation engine (including machine learning, artificial intelligence, and statistical engine) that can run analytical computations to generate insights, and store them in a manner suitable for further analysis and execution. Computation engines also support real-time scoring of the models.
Integration with execution engines to deliver insights: An insight platform reduces the time between the insight generation to execution by delivering insights to the application through the execution framework. Execution framework also collects the feedback data from instrumented applications for measurement and future refinement.
Big data framework to scale: An insight platform has a dig data framework to handle the varied data sources automatically, manage the growth of the data, and run computations in a distributed manner at the foundation level. This framework also has the ability to add services like in-memory caching, data virtualization, or metadata cataloging as they evolve from datalakes.
Let’s now look at a real-life scenario of a retailer leveraging a system of insights. A grocery retailer wants to personalize customer experience and make instant recommendations to its visitors on the website. The transaction data, the product catalog, and the customer repository are readily available in the system of insight. Using the big data framework, the insight platform builds recommendations for customers and products. These results are made available in No-SQL database and the API layer is available for integration to any customer interaction channel, be it mobile, POS, or e-commerce.
How does it work? As the first step the customer’s online behavior is streamed into the system and recommendations are sent to the e-commerce site. Execution framework, which embeds a powerful rule engine, delivers product and promotion recommendations by correlating the online behavior, customer details, and product details. Customer response behavior is taken into the system and that helps in evaluating options and optimizing the recommendations during the future visits. Retailers utilizing a system of insight could easily scale and optimize with this additional knowledge and generate higher average basket value and website conversions.
There are innumerable ways where a system of insight can prove to be a game changer for retailers. An insight platform removes many of the hurdles that retailers faced with data and analytics and offers them an easier solution for their complex analytics needs.