Big Data is becoming a widely accepted term in the I.T. industry, but what does it actually mean, and how is it relevant to traditional ERP systems such as  E-Business Suite?

Before Big Data, there was… Data
To understand what we mean by Big Data, let's roll things right back to basics - what is data?

Data is the stuff that information comes from - raw data is pretty useless if there is no capability to interpret it. So - if we apply some logic and rules to data this then makes it valuable - information that we can use to help us make better informed decisions.

For example, if an organisation has a set of financial data that consists of a bunch of dates and sales figures this doesn't really help them make any kind of judgement or informed opinion on how the business is performing. But - if they apply some rules to the data such as ordering the transactions by date, or value - they can then start to see patterns and trends that will help them make better informed decisions. Maybe the business is seasonal, so they see a spike in sales during summer months - so this will help them to plan increases in stock during the early part of the year.

If the business also captures customer data with the sales transactions, more information can be derived - such as who buys the most from them, and when. Again, this can be used to make informed decisions to help them with marketing or promotional activity.

Traditionally, ERP systems such as E-Business Suite have been the home for this kind of data, which is usually stored in a relational database. The business applications that form the backbone of ERP systems allow the capture of this transactional data (customer information, invoices, supplier bills etc.) and the reporting tools provide insight and analysis of the data to allow the organisation to make informed decisions.

So what’s changed?
Whilst this is great - and has been the accepted standard for at least 20 years, things have changed in the last few years mainly with the emergence of social media and the instant effect this can have on organisations - either in a good or bad way.

Take the organisation above as an example. We have a great ERP system and we can find out lots of interesting things from it such as who are our top customers, when we make the most sales, what products are our best sellers, worst sellers, how much profit we make etc.

Again, this is good stuff and does help us make decisions. But what if we don't have the full picture. Say we notice a drop in sales for a particular product - how do we find out why this has happened? The ERP reporting tools can't help us find the root cause, they only tell us that the sales have dropped - not why they dropped.

Traditionally, to find out why our sales have dropped we would need to go out to our customers, maybe do some surveys and market research to establish the cause. Eventually we would find out the reason for the reduction in sales and be able to put an action plan in place - but this could take weeks or even months. By that time, our competitors are likely to have taken significant market share away from us.

OK, so what is Big Data and how can it help?
We've established that traditionally 'relational' data is used within organisations to help them make informed choices - but what about all of the data outside the organisation that people are creating - such as product reviews, Twitter comments, Facebook comments. Surely if the organisation can build this in to their analytics this will give them a huge advantage.

Take the case of sales figures dropping - we use the ERP information to identify that sales have dropped - but we could use the social media information to identify exactly WHY the sales have dropped. Maybe it's because people are putting out bad product reviews, maybe it's because there's a new product we weren't aware of, maybe it's because a competitor is selling the same product cheaper, or offering customers a better service.

Wouldn't it be good if we could have all of this information in one place so we can have a dashboard that tells us when sales have dropped, then allows us to drill in to not only sales and financial information but also other dimensions such as product reviews and feedback. We wouldn't then need to spend vast amounts of money and time on market research and customer surveys to get the answer.

Big Data - and how we can make use of it.
Big Data then, in simple terms, is the social and other data that you can find in every corner of the internet, through review sites, places such as eBay and Amazon, Facebook, Twitter. The problem with it though is that on the whole it is unstructured, so doesn't lend itself to being analysed by traditional 'relational' methods that we have used for many years to analyse ERP data.
So - a mechanism is needed to allow access to unstructured data, and for it to be interpreted and available alongside ERP data. There are now a plethora of tools available that allow access to this unstructured data - predominantly Apache Hadoop and Oracle’s Big Data Appliance.

To make sense of the data and provide analytical insight, 'Big Data' analytics tools becoming available. One of these tools is Oracle Endeca, which provides access to social and product review information allowing you to search for key information relating to your products or organisation. Endeca also has pre-built extensions that integrate with E-Business Suite so that you can use this information in conjunction with you existing ERP data to give you a real 360 view of not only how the organisation is performing - but what your customers are thinking about you, and why.

The result is that E-Business Suite users now have the opportunity to really get under the covers and gain a real insight in to not only what goes on inside their organisation but what their customers, potential customers and even competitors are thinking about them – and why.