Curiosity is a great thing. One thing leading to the next in a domino effect makes a lot of noise and sometimes fantastic music. Having come to a point of developing expertise in Big Data, I am attempting to tell a story.
Let me frame why everyone thinks about big data. Since we started converting our thoughts, ideas and experiences into digital form, information and therefore capacity has grown in leaps and bounds. If you don't believe the very nice graphic here, look at the capacity of any electronic device you have.
The average costs per Gigabyte (10003) of storage keeps on going down from over $6 million in 1980 to just under $6 in 2013. Now a Gigabyte can store over 600,000 pages of text. An exabyte is a giga of a gigabyte or (10006) and can fit about 18,000 times all the books ever written (by 2006) according to an IDC paper sponsored by EMC Corporation in 2006.
Now that we have some context of what we have and what we can store, It does not take a math wizard to tell that there is a lot of duplication/noise in the data that we need to separate from the information/signal. I could not resist the electronic reference, but to separate the signal from the noise we have been building techniques mentally and scientifically over a long long time as humans have evolved to create such capacities.
All of this has just touched on one aspect of big data the Volume- the sheer amount of data- separate from information see the best definition, i have found here. Add on the growth as seen in the chart above leading us to the second factor Velocity- its cheaper so we will just store more; in different ways, ergo: Variety- literally as a picture is worth a 1000 words, 10 second movie 100,000 words in a language best available at the time; we have parroted the 2001 research report[20] .
When data is processed, organized, structured or presented in a given context so as to make it useful, it is called Information. This is the promise of Big Data- over the course of my exploration, I will write about what this means
Let me frame why everyone thinks about big data. Since we started converting our thoughts, ideas and experiences into digital form, information and therefore capacity has grown in leaps and bounds. If you don't believe the very nice graphic here, look at the capacity of any electronic device you have.
Now that we have some context of what we have and what we can store, It does not take a math wizard to tell that there is a lot of duplication/noise in the data that we need to separate from the information/signal. I could not resist the electronic reference, but to separate the signal from the noise we have been building techniques mentally and scientifically over a long long time as humans have evolved to create such capacities.
All of this has just touched on one aspect of big data the Volume- the sheer amount of data- separate from information see the best definition, i have found here. Add on the growth as seen in the chart above leading us to the second factor Velocity- its cheaper so we will just store more; in different ways, ergo: Variety- literally as a picture is worth a 1000 words, 10 second movie 100,000 words in a language best available at the time; we have parroted the 2001 research report[20] .
When data is processed, organized, structured or presented in a given context so as to make it useful, it is called Information. This is the promise of Big Data- over the course of my exploration, I will write about what this means
Nice start Ashvin. Looking forward to more.
ReplyDelete