Data Enrichment

Data Enrichment – the nail in the coffin for legacy analytics

The Big Data phenomenon has simply reinforced the importance of transforming business through data, with organisations looking to enrich internal data sets with the addition of new, increasingly open, data sources.

From weather to crime, smart meters and traffic, the diversity of open data sources is incredibly exciting. So how can organisations experiment with this data quickly and efficiently without incurring the untenable costs associated with traditional data analytics? Our director Laurence Armiger explains.

The concept of data enrichment is not new – leading retailers, for example, have integrated demographic data sets to enhance customer data not only to improve customer understanding but also inform key strategic decisions such as the location of new stores. However, the sheer volume and innovation associated with the new generation of open data sources is radically transforming the opportunity and opening it up to organisations of all sizes.

Yet, organisations need to be able to experiment with these data sets quickly, effectively and cheaply. The more companies think about potential uses for these open data sources and combining diverse data sets, the more chance they have of discovering a killer application or piece of data that is going to deliver massive business value. Open source data has the power to transform business – but it might take 100 different data sources before a company hits on the right one. Trying to do that with an old style data warehouse is impossible.

The latest generation of analytics database technologies have been designed not only to manage vast data quantities but also to compress that data into manageable – and affordable – volumes and provide the business with the insight required. Exploiting innovation in areas such as data compression and pattern matching, these solutions require not only minimal infrastructure – and hence cost – but deliver a new way of locating information within the mass of data to enable rapid exploration of each new open source dataset. For example, a retailer can use smart information feeds from utilities regarding planned and emergency works that may affect customer access to stores and automatically contact customers due to use click & collect that day and suggest an alternative location. This is a simple but effective way of not only ensuring the click & collect service is unaffected but also improving overall customer perception.

Companies have begun to recognise that the old fashioned goliath databases have no place in the big data era. Many, however, have been understandably reluctant to walk away from multi million pound investments in customer databases to embrace the new breed of analytics databases despite the clear benefits of cost, speed and responsiveness.

The rise and rise of open data sources is the final nail in the coffin for the legacy approach.

Data enrichment is fast becoming an essential sidebar to every big data strategy – isn’t it time for every business to accept the need for a new analytics model that enables fast, effective and affordable data experimentation?