Making sense of data in an era of data abundance
We all have smartphones, but have you ever considered how much data they generate? Data is continually generated these days whether you open an app, perform a Google search, or just travel to any location with your smartphone. It is not just the data that you upload and download, but also the useful information that many organizations generate in the background to display, analyse, and profile you or your usage.
The sheer visual presentation of per minute counts of activity across many everyday applications illustrates the vast volume of data we generate on a minute-to-minute basis.
So, How Big is Big Data?
Here are a few of the most significant Big Data statistics to get us started:
- The global Big Data and Analytics market is worth $274 billion
- Big Data analytics for the healthcare industry could reach $79.23 billion by 2028
- There are currently over 44 zettabytes of data in the entire digital universe
- 70% of the worldâs data is user-generated
- Cloud computing generates nearly $400 billion in revenue
We have now officially and irreversibly entered the Data Age. From cookies to social media profiles, everything we do online and even offline leaves data footprints. So, how much info is there really? How much data do we process each day? Internet traffic is simply one component of overall data storage, which includes all personal and commercial devices. Estimates for total data storage capacity available in 2019 vary but are already in the 10â50 zettabyte range. This is expected to increase to 150â200 zettabytes by 2025.
Clearly, data generation will accelerate in the coming years and while you are reading this, you are probably thinking, âIs there a limit to data storage? Well, there is, but we are not approaching them anytime soon.
Now, consider the preceding and add to it common business proverb, “Data is the new oil.” And you will be able to relate why making sense of information in this age of information abundance, has become more crucial than ever.
What is Big Data?
Big data refers to massively complex organised and unstructured data sets that are rapidly generated and sent from several sources. These characteristics include the three V’s of Big Data.
| The three Vâs of Big Data | ||
| Volume | Velocity | Variety |
 |  |  |
| The large volume of data stored in multiple environments | The velocity at which the data is generated and processed | The wide variety of sources or forms from which data is collected |
| Storage of logs of a website, data by Gmail | Number of requests received by Facebook or Google per second | Â CCTV video files across the city Customer feedback received from an ecommerce website |
Big data analysis is the process of analysing this data to discover hidden patterns, correlations, and other relevant information. Amazon and Netflix, for example, rely totally on Big Data analytics to develop and expand their businesses.
How is Big Data helping organizations stay ahead?
- Consumers, premium customers, and what motivates them to buy products are all being analysed by firms using Big Data.
- To construct market strategies based on client projections, it is incorporated with Machine Learning technology. When businesses embrace Big Data, they can be customer centric.
- To keep up with changing consumer tastes, companies can employ both historical and real-time data. Translation – actionable marketing insights and revamped sales strategies.
Demystifying the Gigabyte, Terabyte, PetabyteâŠ
As average computer users, we do not consider much beyond a few terabytes of storage capacity. As storage becomes more affordable and data becomes larger, the term “petabyte” will become more widespread. With the advent of the Internet of Things (IoT), physical objects will begin gathering data via various sensors to process better insights and harness and utilise the information. This will generate so much data that it will be necessary to upgrade data storage technologies to construct systems capable of processing massive amounts of ever-increasing data.
The Future is Data as Code
 Data is fetched from databases or filesystems to where the code resides in the traditional data processing paradigm. The massive amount of data that must be handled in Big Data makes this nearly impossible. Because the data volume is so large, a single database server cannot handle it. Big Data is often partitioned and stored over many physical database server computers. Application servers, on the other hand, must be added to boost the processing capability of Big Data. However, as the number of application and database servers for storing and processing Big Data grows, more data must be transferred back and forth over the network during the processing cycle, until the network becomes a major bottleneck.
A new computer paradigm was required to overcome the network constraint. Rather of moving the code to the data, we move the code to the data and do the processing where the data is stored. The software sends code to the database server. Apache Spark is a popular open-source unified analytics engine for large-scale data processing. The volume of data transferred over the network is greatly decreased by relocating the code to the data. This represents a significant paradigm shift in large data processing.
Big Data Use Cases from the Travel Industry
Fun Facts on Big Data
Cited Sources
- https://builtin.com/big-data
- https://www.techtarget.com/searchdatamanagement/definition/big-data
- https://cloudxlab.com/blog/introduction-to-big-data-and-distributed-computing/
- https://dzone.com/articles/big-data-processing-spark
- https://findstack.com/big-data-statistics/
- https://techjury.net/blog/big-data-statistics/#gref
- https://towardsdatascience.com/how-big-is-big-data-3fb14d5351ba
- https://explodingtopics.com/blog/big-data-stats
Do keep an eye on this space for more blogs and information on Big Data.
Authorsâ Bio
This article was authored by Cyril George, an Associate Product Manager at Tavisca who believes in asking the right questions to build the right product. Cyril is passionate about travel technologies as much as he likes to travel. He is currently working with engineering teams to build platform level products to help internal development teams.
