Data is everywhere in today’s society. Whether it’s statistics in a government file, confidential information at the doctors, mobile phone calls, e-mails, Facebook pages or twitter posts, there is no escaping it. Without doubt, data today is ‘big’.
So big in fact, that experts are suggesting we could currently be creating as much data every 10 minutes as we created from the dawn of time up to 2008. We are talking petabytes or brontobytes of data (imagine your 1 terabyte (~1000GB) hard-disk at home, then add more than 20 zeros to the storage capacity and you’ll get an idea of the scale we’re talking about).
As a result, the phrase ‘big data,’ once confined to technical geeks and IT departments, is now creeping into everyday life and is commonplace in business terminology. Its technical definition is: A collection of large data sets that are difficult to process.
Traditionally data has been structured, meaning it has defined relationships that enable users to trace it and cross-check it against other data, as well as being historic or static e.g. geographical or address-related. By comparison, newer, social media-related data, such as messages, photos or mobile phone calls, tends to have little structure and often its value can only be obtained if it’s analysed in real-time. It is thought that as much as 80 per cent of big data is unstructured. Hence the use of the term ‘difficult’ in the above definition.
Big data is commonly described as having five characteristics, often referred to as ‘the five V’s’:
So how does big data apply to financial services?
Banks have been slow to recognise the potential of big data, mainly because of the high number of legacy systems in existence and the recent focus on compliance. However, as the cost of hardware falls and more off-the-shelf solutions become available from third-party vendors, financial institutions are starting to realise that the potential outweighs the risks. After all, research has shown that banks who apply analytics to both internal and external customer data have a 4 percentage-point greater share of the market. Still, half of all banks don’t use this resource.
Customer Analysis
Being at the centre of the flow of payments in most circumstances puts banks in a unique position. The account balance and general volatility of a customer are useful insights into their credit-worthiness, but a detailed drill-down into their recent transaction history and spending habits, combined with their demographic and personal details, improve the accuracy of many types of predictions. If the results are further combined with external sources such as social networking data, the accuracy can be improved even further.
In fact, one European bank built an analytical model that used over 40 variables to predict the likelihood of customers investing in savings products. The data was then used by the sales teams for cross-selling and, during the pilot, the area targeted saw a ten-fold increase in sales over a two-month period. Predictive analytical models have also been used with great success by banks to calculate customer retention potential and the probability of forming long-term customer relationships.
Compliance and Fraud
Banks are also now using big data techniques to monitor the activities of traders. Traditional methods included looking at key-words from internal communications and trade data, but this is being increasingly combined with more sophisticated recognition technology. Fonetic, a Spanish software company, has recently released software that can search for particular topics within a conversation. For example, if a search was being done to find traders who are talking about bananas, then the software might look for yellow, monkey, banana, fruit, etc. This software is used by a number of their banking clients.
Market Prediction
Furthermore, Goldman Sachs has recently invested $15 million in Kensho, a start-up company that provides a large database of timelines associated with world events. It could, for example, tell you everything that happened when Apple launched its last iPad, or about the market environment during a meeting of world leaders. Combine this with a sophisticated user interface and the fact that the data is provided in real-time, and analysts have the potential to make predictions that normally take hours within minutes.
To conclude, it appears that big data is not just another buzz word or a passing trend, and that the changes it’s brought are here to stay. Financial institutions are at last starting to realise its potential and are beginning to invest in some very useful technologies.
However, increased monitoring does prompt the question of whether rogue traders will simply be driven ‘underground’, making them even harder to track down and resulting in expensive battles of surveillance.
There are also privacy laws to consider. Google, Facebook and other social networks sit on a veritable gold-mine of potential customer data, but there have often been outcries in the past when they have tried to make use of it. So could it bring about a change in the principles of social networking and what we know as ‘private’, or could it generate a new raft of law-suits for the banks?
Analysts making market predictions in minutes rather than hours is a holy-grail for investment banks, but the potential for algorithmic or computer error gives credence to ‘the bigger the data the bigger the risk’. After all, one of the underlying concepts of big data is greater automation in the decision making process.
So, while the benefits of big data are very clear, like most leaps in the use of technology its progress needs to be taken with an element of caution and planning. After all, it is rare to get something for nothing.