Business Intelligence for Retail

Problem

Today, in retail, assortment modeling, pricing, demand forecasting, advertising campaigns and personalization of product recommendations (from the formation of individual offers to loyalty programs) are absolutely impossible without the use of developments in the field of BIG DATA. Meanwhile, the automated and streamlined work of these areas is a guarantee of profit for any retail chain.

However, retail is an area where a large amount of disordered data is accumulating at a tremendous speed: the range of products on the shelves can reach thousands of items, tens of thousands of transactions go through the system every minute, and the retail chains themselves serve millions of customers a day. Thus, on a daily basis, any retail chain must collect, analyze and generate reports on the growing volumes of data that it receives from CRM-, ERP-systems, and points of sale.

The processing of such data is becoming the biggest problem of the retail business — the storage and analytics of such an array of information requires more and more computing resources, scaling ground-based DWHs and expanding staff, which constantly increases the cost of working with data and forces retailers to include these costs in the price of goods. As a result, instead of profit, retailers often get a drop in sales due to regular price increases and, as a result, are forced to reduce the assortment due to lack of sales.

Moreover, constant competition forces retailers to look for new and more effective ways of selling, and making any commercially effective decisions is impossible without high-quality analytics. Such analysis cannot be carried out by conventional means: Excel analytics can only cover a small part of business needs, while most of the data cannot be processed using such tools, as a result — most of the questions will remain unanswered.

Solution for retail

Let’s look at the classic scheme of the functioning of a retail network.

Company data is stored on local SQL servers (ground-based DWH), processed in an OLAP cube (On-Line Analytical Processing) — a structured multidimensional data array that allows multivariate analysis in various sections using SQL Server Analysis Services, and on Based on this consolidated information, daily reports are generated.

Since terrestrial DWH and typical analytics tools do not allow reporting to be displayed in stream mode, the data is updated a maximum of several times a day, which does not allow businesses to make timely decisions, moreover, to automate the process of making them.

For example, a company’s analysts will be able to see a losing trend and react to it only the next day, and by that time it can bring a considerable loss.

Since working with BIG DATA every day requires more and more resources and expanding data warehouses, accordingly, it also becomes more complicated and the amount of required reporting increases. In the case of large retail chains, this leads to a complete impossibility to conduct high-quality analytics and understand what measures should be taken, for example, to determine the optimal assortment, the number of outlets, favorable prices, profit maximization, increase in market share, not to mention what kind of — any forecasting without a thorough and complete analysis of the data is completely impossible.

The solution to this problem, which is the most relevant for retail, is the use of hybrid data warehouses or a complete migration to the Microsoft Azure cloud platform, namely, the use of its Microsoft Azure Synapse Analytics analytical system, which contains modules for connecting, processing, transforming, analytics and visualizing data in live mode.

Microsoft Azure Synapse Analytics allows you to transfer data from DWH (Microsoft Azure Data Factory) and automate BIG DATA analytics (Microsoft Power BI), that is, analytical reports can be transferred to Azure Synapse Analytics and linked to the Power BI data visualization platform, which will provide practically 24/7 data monitoring.

In the future, based on this descriptive and diagnostic analytics (descriptive and predictive analytics), you can build predictive (predictive, advanced analytics) using the Microsoft Data Bricks module, which will allow you to develop AI models for classifying and segmenting visitors, forecasting demand, pricing, automating commodity stocks.

The resulting forecasts can be used for assortment planning, launching advertising campaigns, developing loyalty programs, and much more.

Advantages:

1. The transition to Microsoft Azure Synapse Analytics optimizes infrastructure costs and allows you to “infinitely” scale data warehouses — the company pays only for the power consumed and the platform modules used, and can expand the service package at any time.
2. Daily reports can be updated every 15-45 minutes, instead of once a day, which will allow the business to quickly respond to changes
3. Usually, data is collected in different ways and systems, often so technologically different that it is difficult to aggregate them and even more so to conduct a qualitative analysis. Azure Synapse Analytics allows you to get a single automated solution for data storage and analytics within a single platform.
4. Maintaining the company’s IT infrastructure and ensuring its uninterrupted operation becomes the responsibility of Microsoft — the company guarantees at least 99.5% availability of its services according to the SLA (Service Level Agreement).
5. Microsoft guarantees data security — the platform allows you to set access levels to reports and monitor user activity.
6. Today, modern data solutions are developed on cloud platforms, and therefore it will be difficult for a company that ignores this trend to gain competitive advantages in the near future.