Introduction
Big Data Analytics refers to the process of collecting and analyzing large volumes of data to identify patterns, uncover hidden correlations, recognize market trends and customer preferences, and gain other useful insights. It has emerged into a highly sought after and specialized discipline, with data scientists, predictive modelers, and other analytics professionals aiding modern businesses in making more informed business decisions.
The big question that arises is, what caused such a spike in the volume of consumer data being generated. The answer can be found if one considers the unprecedented changes in the digital world, which caused big shifts in the financial and economic fields, which, compounded by a burgeoning middle c
lass, drastically changed the prevalent business environment as well.
According to www.internetlivestats.com , around 40% of the world has an active internet connection today. In 1995, it was less than 1%. Increased Internet penetration, emergence of e-commerce and social media, have brought consumers closer than ever, to the products and services they seek to avail, and changes in their behaviors manifest themselves into changed internet usage patterns.
The above mentioned factors have spurred drastic changes in consumer expectations and demands, over the last half century. Consumer behavior study, too, is a new, interdisciplinary, and emerging science, developing since the 1960’s, the same time as organizations began employing analytics to better gauge incoming customer data.
The concept of Big Data has evolved slowly since the latter half of the 21st century, when organizations began to comprehend the benefits of tapping into the large volume of incoming customer data, streaming into their businesses.
Initially, companies had begun to apply basic analytics to such incoming data, to understand and recognize visible trends and insights, before anyone had thought of uttering the term “Big Data”. The term Big Data Analytics is generally applied to identify the present scenario, arrived at by over five decades of evolution in Data Analytics.
Big Data Analytics can find application in various industries, like the medical field, military and security field, and marketing field. In the marketing field, applying data-driven metrics to form decisions, can transform into better productivity and increased operational efficiency. Strong application of Big Data-Behavioral Analytics, helps marketers reach the right customer, with the right products, at the right time. The most successful examples of implementation include Amazon’s vaunted recommendation engines, and micro-targeting in political campaigns.
Thus Big Data Analytics has sparked off an intense knowledge revolution, that is sweeping through business, academia, healthcare, military conventions, and everyday life.
Purchase decision, and evolution of consumer behavior
Nicolaus Bernoulli, a Swiss mathematician, sparked the investigation into trying to determine what influences purchasing decisions (he introduced the terms expected and marginal utilities in economic theory), followed by John von Neumann, and Oskar Morgenstern. They considered people as purely rational beings, and created mathematical models to determine utility and consumption of a particular good/product.
Such models worked well to understand and predict consumption patterns till the 19th century. However, the economic and population boom in the latter half of the previous century rendered them inadequate. The surge in populations, as well as increased life expectancy, meant corresponding proliferation of needs and desires to be satisfied, and consequently more products and their variants. Travel became more accessible, communication between communities increased, which led to even further diversification of demands, and development of more complex behavioral patterns.
When it became apparent that social, cognitive and emotional factors influence purchase decisions as well, a new and complex interdisciplinary science appeared in the 1960’s, namely Consumer Behavior,
Over this past half century, people gradually developed an extensive behavior, buying more products, and increasingly more diverse. The rise of Internet, and e-commerce, further exaggerated the shift in behavioral pattern. Then arrived the widespread economic & financial crisis of 2008, which turned people into defensive buyers. The Internet began to be used on an even larger scale to look for products and services, and for comparing prices and specifications/attributes. The proverbial balance in the market had shifted. Control had passed over from the vendors and suppliers, firmly into the hands of the buyers, both in terms of influence and bargaining power. Buyers today are more informed (57% of purchasing process is completed before first contact with sales), socially networked (more than 50% customers declined an in-store purchase due to negative online feedback), and less loyal (59% customers are willing to try new brands to receive better customer service).
Organizations forecast that online marketing would begin play a decisive role in trying to gauge and influence purchase decision, so new techniques were needed to be developed predict customer behavior, one of them being Big Data Analytics.
Evolution of market research
Big Data- the newest instrument to predict consumer behavior
AMA (American Marketing Association) defines market research as the set of process or processes that forms the chain which links the consumer and the customer to the marketer, through information. Marketing research designs the method for collecting information related to consumer demands and prevalent market conditions, manages and implements data collection processes, analyzes the results, and communicates to the product development team the findings and their implications.
The marketing research, as a discipline, has had to adapt to changing consumer behavior and economic environments. Two most successful and widely applied methods of market research in the recent past, used extensively by multi-national organizations all over the world, are:
- TORA (Theory of Reasoned Action), developed by Martin Fishbein and Icek Ajzen during 1975-80.
- NPS (Net Promoter Score), a customer loyalty metric, developed by Frederick Reichheld, Bain & Company and Satmetrix, and introduced by Reichheld in 2003 in an HBR article titled “One number you need to grow”.
These processes inherently tried to elicit behavioral intention, instead of trying to capture actual behavior. However, marketers today are more interested in what percentages of a purchase intent has transformed into an actual purchase. This is where Big Data Analytics unveils its true prowess. When it comes to understanding and predicting actual consumer behavior, Big Data Scientists and analysts have better, and more accurate answers than traditional market research models and techniques.
The emergence of RFID at PoS (Point of Sales), the Internet, social media, and e-commerce, has radically altered the way consumer behavioral data is gathered. Digitization and increased internet usage has enabled extraction of information from a multitude of places, out of Internet searches, Facebook, smartphones, anywhere a user leaves behind a trace, even when he/she is not making a purchase. The terabytes of data thus generated is free to be measured and analyzed by data scientists, who in turn can present statistical models of actual consumer behavior.
Big Data is usually defined as the combination of volume (quantity of data), variety (multiple layers of data), velocity (speed at which data is created), and veracity (accuracy of inaccuracy of collected data). Due to the huge volumes, and complexity involved in authentication of real-time data being generated rapidly, Big Data can become a little tricky to master, and be molded into something useful.
| Advantages of Big Data | Disadvantages of Big Data |
| Improves predictive accuracy, and makes customer’s life easier | The more data is registered; more problems the analyst has to solve |
| Can identify hidden patterns and correlations, which can be used to propose better informed solution. | Customers feel their privacy is being increasingly breached. |
| Real-time data is collected, hence inferences and conclusions gathered are also real-time. |
Ownership of the data being collected can be questioned.
|
| Useful and accurate data can be immensely useful to market research teams. | “unclean”, or inaccurate date, can lead to incorrect results, leading to errors in decision making further down the chain. |
The practicality and utility of Big Data has led to its adoption by most major industries, world-wide: healthcare (increasingly personalized treatments and predictive analysis), sports (improving performance via use of data generated from sensors), weather, logistics, manufacturing, etc.
In this era of Digitization, Big Data has thus become one of the three channels of transformation (the other two being cloud and mobile). However, the path to implementing Big Data is still riddled with challenges:
- Digital proficiency
- Legacy systems
- Security
- Jobs becoming obsolete
Key technologies involving Big Data
- Data management and Data Mining
- Hadoop (an open source software framework to store large amounts of data and run application on clusters of commodity hardware)
- In memory analytics
- Predictive analytics
- Text mining (for analyzing text data from the web)
The perfect instrument to study consumer behavior
Since the Second World War, consumers and customers
have steadily displayed more and more open minded and extensive behaviors, which in turn has made marketer’s jobs more difficult. Big Data is one the various tools being employed to understand the nuances and anticipate the vagaries of customer behavior. Scientist and analysts dealing in Big Data try and glean information off that can aid marketers and manager in this quest, for instance customer loyalty/customer migration, cross sales, sales according season, customer segmentation, customer approach, promotion analysis, channel of distribution analysis, etc. The most popular analysis among these are basket analysis and cross selling analysis, which can help correlate and form associations between different products in a customer’s basket. This in turn can help companies formulate strategies on how to increase sales of products of same or complementary categories.
One of the most important marketing research tools to use Big Data in its operations is SAP hybris Marketing. It is then integrated into other SAP solutions like ERP and CRM. SAP hybris Marketing collects data about users’ present and historical purchases and usage (from in-stores or e-commerce sites, websites, complaints registered, social media feed, etc), which is then analyzed to create a customer profile. Customer profiling helps companies draw an image of customers’ needs and expectation, which then helps such companies to provide individualistic shopping experiences to its customers.
Big Data has become the perfect instrument to study modern consumer behavior. Businesses study and sift through real time data before making business decisions, and then analyze the feedback generated to gauge the effectiveness of their decisions. In other words, Big Data Analysis and business strategies form an endless loop, which helps today’s organizations survive and sustain themselves in the uncertain economic environment.
Big Data and Machine Learning to enhance customer experience
Big Data and machine learning both evolved from the need to study pattern recognition. Big Data mainly involves large data sets, while machine learning involves forming and testing algorithms. One of such algorithms, artificial neural networks, gave rise to a new science called neuro-marketing, in order to facilitate research into consumer behavior. Basically, neuro-marketing applies neural techniques to apprehend consumer behavior involving brands and marketing.
The innovation and added value gained from neuro-marketing, has already enticed big firms like Pepsi Co, Coca Cola, Google, P&G into trying to implement it.
Research shows that market has shifted from products and features, to user experience, which again can be measured through 3 key metrics: people, business and technology. Organizations must consider all these metrics before launching a product line.
Enhanced user experience benefits an organization in various ways, monetary as well as non-monetary, for instance increased customer loyalty, productivity gain, reduced alterations/exchange requests, et cetera. Big Data analysts study real time data to decrease firms’ response times, which is one of the main KPI’s of better customer experience.
Conclusion
Application of Big Data Analytics to understand and anticipate customer behavior is more than applying rear-view mirror tools to review past purchases and trends. It provides organizations with insights and knowledge to improve their operational efficiency, optimize sales and market research, prevent fraudulent purchases, etc. However, handling huge amounts of data and complexity of analytical tools often poses a steep learning curve or high implementation barriers, which often makes application of Big Data Analytics a daunting task for most major companies. However, companies need to overcome such hurdles, and determine the optimal strategy to collect and analyze such data in a scalable and accessible way, to better understand and anticipate customer behavior.
In the 21st century era of digital marketing platforms and e-commerce, it is important for ambitious firms to deploy such strategies and establish Big Data Analysis capabilities. Its more than likely market leaders find themselves at their positions today, because they could gauge which way winds are blowing some time ago, and had prioritized on employing Big Data to understand customer behavior long back.
References: –
- teradata.com
- optimove.com
- techrepublic.com
- Publication by Cristina Stoicescu on Big Data
- emarketer.com