Data; let’s get personal

Keeping in contact with your customers is a vital ingredient to ensuring that the hard, expensive work of acquiring customers and then delivering a quality product is translated into brand loyalty, repeat orders and ultimately high lifetime value.

Mobile and browser notifications and direct social media messages are the hot channels these days, but good old email which, despite its relative long-in-the-tooth-ness, is still an extremely powerful communications medium. So building an effective communications strategy that maximises the weapons at your disposal makes the difference between a great and a mediocre strategy; one that works, and one that does not.

Data sits at the core of many of the tools at the disposal to marketeers. Measuring effectiveness is of course hugely important, but data can also be used much more tactically than that. This blog post discusses one of the ways data can be used to develop content that is personal, targeted and relevant to each individual and how you can expect this to increase your campaign click-through and key interactions by around 25%.

The right message

Content maybe king, but it has to be relevant to each individual. Sending a broad brush/generic message to the whole of your customer base may struggle to accomplish. Amazon founder Jeff Bezos famously said “If we have 6.2 million customers, we should have 6.2 million stores”, meaning of course that the customer experience should adapt to suit each individual visitor; highlighting things of interest to the individual and suppressing those that are irrelevant.

Of course doing this manually on any kind of scale quickly becomes impossible and this is where use of data comes in. A first example of use of data to build personalised content is to look at a customer’s purchasing behaviour and then recommend to them products that are often purchased together. For example, those that purchase a top-end product, such as a ride-on-lawnmower, may also be interested in other top-end products that can be used to furnish their ample gardens. Product-to-product relationships can be discovered using a technique called Association Rules (or Basket Analysis) which find products that tend to be purchased together, the famous example of this being nappies and beer which are apparently often found in the same basket at supermarket checkouts due to tired new dads being sent on the shopping run; though this may just be a slightly sexist data-driven old wives’ tale.

While use of this technique is a good first step down the personalisation road – those using it will garner no criticism from me – it does still tend to create generalised buckets. If person purchases Product A, we will always try to sell them Product B and C no matter what else we know about them. In order to get much more granular personalisation; one that really gets inside the mind of each individual customer it is worth considering another technique, collaborative filtering.

Collaborative filtering is in concept an extension of Association Rules but can be applied to far larger datasets. By combining myriad of data-sources available to you such as order history, each click, touch and scroll points website and mobile app, all your views, opens and clicks from your marketing campaigns (email and mobile push notifications) as well as pulling in social-networking queues, it becomes possible to develop algorithms that start to develop a deep understanding of the intents of individuals as they interact with your business. Combining these datasets in effect harnesses the wisdom of the crowd and gives your communication strategy a layer of collective intelligence upon which you can draw strong inferences about individuals and what they are likely to be interested in and from this derive content specifically for each individual you are talking to.

From a data-engineering perspective working with data of this nature can be extremely challenging. It has the verbose, high-velocity and schema free properties of typical big data workloads. However, the majority of the value is contained in the dense and complex relationships between customers and the ‘things’ that they might be associated with, be they your products, or their likes and dislikes gleaned from social media.

There are a number of data technologies that can help here, but at BiG we have found that a slightly esoteric but increasingly popular off-shoot of the NoSQL ecosystem forms a highly effective backbone to this type of work – the Graph database. This type of database, typified by market leaders OrientDB, contains no tables, rows or even cells. Instead all data is held as either atomic units called Nodes (or Vertex) or Relationships (Edge) between two Nodes. Thus you may be represented by one Node and a Product maybe by another Node. Your interest, viewing history and purchasing history will be represented by the relationship between those two nodes.

Graph database products, like OrientDB, allow this structure to contain many billions of records and enables you to form an extremely semantically rich picture of your data domain, be that for an individual, product or even product category. This forms the core of a highly scalable recommendation system that against which you can develop queries that give you truly personalised content.

Is it worth it?

The short answer is yes. Personalisation is huge driver in positive customer engagement; for example, we routinely see CTR of around 25%. At BiG we work closely with the world’s leading equity crowdfunding platform Via this and other data-centric hyper-personalisation marketing techniques we have driven significant increases in money invested into businesses raising on the Crowdcube platform. In some one recent case, the difference between hyper-personalisation vs control has been an over 56% increase; that equates to £8.26 vs. £0.22 investment made per email recipient. Great news for entrepreneurs seeking to take their businesses to the next level and also the investor membership, who have been connected directly with businesses they are keen to support.

About the companies in this article is the world’s leading equity based crowdfunding platform; with an investor membership of over 350,000 they have so far raised over £200 million for 500 growing businesses.

OrientDB is the world’s leading (and fastest) 2nd generation NoSQL Distributed Graph database offering a blazingly fast and scalable multi-model document and graph platform; benchmarked at 10× faster than rivals.

Big Consultancy – Gerry McNicol is co-founder at We are a team of data engineers, analysts and consultants who specialise helping companies successfully deploy data strategies and technologies. Whether you are dealing with real-time and high-volume streaming data, want to leverage the new wave of machine learning technologies or want to transform your business into a data-driven enterprise we want to solve your problems. Tell us your data ideas, hopes and data dreams. Connect on LinkedIn or via