What Lies Beyond Big Data

by Achraf Bennadi

Today, billions of dollars are being invested by companies and organizations into big data. It has become very easy for them to get too focused on quantitative data and lose sight of the fundamental qualitative information out there. Additionally, whenever possible there is always an attempt to quantify and measure and predict human behavior but fail to do so accurately. Tricia Wang, a global tech ethnographer, raises and sheds a rather disturbing perspective about the dangers of relying only on bid data to drive decisions.

According to her research “73% of big data projects are not profitable”, and with these odds at hand should companies be investing billions of dollars? Tricia Wang explains further about how she believes that these big data project tend to have value in “neatly contained systems”, but do not work in dynamic systems, especially when they include human behavior “because conditions are constantly changing”.

For that reason, she brings to light this idea of “thick data” which she defines as “precious data from humans that cannot be quantified”. Like stories, emotions and interactions that cannot be quantified. This also reminds me of what we had seen in the first Ted talk given by Malcolm Gladwell. Malcolm mentions how there was a shift from universals to variability and that there is not only one universal answer that satisfies everyone. Similarly, I believe that when looking at human behavior especially when it comes to food one has to recognize that there will be what the data shows but one must also consider the different human behavioral factors which “thick data” can narrate for us. “Thick data” is to be used to complement big data models as it is what helps fill the missing pieces and help give context to the human narrative.

However, one should not underestimate the power of big data when it comes to understanding or predicting the behavior of individuals. An example would be when looking at retailers. Retailers collect and keep huge amount of data regarding everyone’s personal life and their purchasing patterns. They can analyze what worked and what did not work as they try new ways of merchandising or promoting their products. In some instances, they are also able to predict if you are about to have pivotal moment in your life such giving birth or moving in together with someone. Some other online platforms such as real estate websites can know if you are planning on moving into a new home.

Sources:

- Malcolm, Gladwell. (January, 2017). “Choice, happiness and spaghetti sauce”. Tedtalk. Retrieved from: https://www.youtube.com/watch?v=iIiAAhUeR6Y

- Charles, Duhigg.(February, 2012). “How Companies Learn Your Secrets”. Retrieved from: https://www.nytimes.com/2012/02/19/magazine/shopping-habits.html

- Tricia, Wang. (August, 2017). “The human insights missing from big data”. Tedtalk. Retrieved from: https://www.youtube.com/watch?v=pk35J2u8KqY