Andreas Weigend, Social Data Revolution | MS&E 237, Stanford University, Spring 2011 | Course Wiki

DF2 - Background Readings

Assigned: March 31, 2011
Due: April 14, 2011

Though most dog foods will be very hands on and interactive, think of this second one as a primer and as a chance for you to really think about the social data revolution before we dive right into it this quarter!

Read the 3 articles from the Economist, found in as well as one Harvard Business blog post,
Having reflected on what you read, please boil it down as one crisp answer to each of the following two questions:

What is one insight you gained from these readings?
Did anything you read change the way you view the wealth of data around you?

Be sure to elaborate in your response, making sure to reference which articles you derived your insights from.


By Thursday, April 14th at 12:00 PM noon, fill out the simple Google Form:

With your answer. Also, post your insight onto and lastly go to Twitter (make an account if you don't have one) and tweet your insight with the #socialdata hashtag. Sorry if this seems like a hassle! We just want to make sure people familiarize themselves with these forms of social sharing, and also get a chance to follow and read other students' responses!

DF2 - Notable Answers

If we enter a world of near-perfect recommendations, where past behavior determines future behavior, what fun is lost from discovery / exploration? How will we be pushed to see completely new perspectives and think in radically different ways from our past selves? How will we grow and be challenged?

~ Asha Gupta

The main insight I got from the readings is about cognitive overload: "The mind can handle seven pieces of information in its short-term memory and can generally deal with only four concepts or relationships at once. If there is more information to process, or it is especially complex, people become confused". Thus there is a risk that a wealth of information creates a poverty of attention. At the end of the day, the algorithms will have to do most of the thinking to simplify things for people, which is a great move forward, but also dangerous, as algorithms can be wrong (just think of the recent financial crisis for instance).
(Handling the Cornucopia, The Economist p. 13, 02/27/2010)

After reading these articles, I realize how much transparency will be key to succeeding as a business in the world. As Andreas Weigend underlines in his HBR blog post, industries which increase transparency using data contributed by customers will be the ones that win in the end (The Social Data Revolution, Weigend A., Harvard Business Review Blogs, 05/20/2009).
Increasing transparency also fosters growth in a lot of businesses. For instance, the opening of information from the government allowed to improve restaurant sanitation, car safety, nutrition, educational performance, etc. The Economist insists that "Public access to government figures is certain to release economic value and encourage entrepreneurship" (The Open Society, The Economist p. 9, 02/27/2010).

Secondly, even though transparency is the future of social networks and businesses, the real need is for a community. So the challenge is to provide incentives to people to supply the data, or even penalties for withholding them (The Open Society, The Economist p. 9, 02/27/2010).
This relates to a lot of thinking made in the January 30 2010 Economist: companies realize they need to engage with people, and not to advertise on them (Profiting from Friendship, The Economist p. 9, 01/30/2010). Incentives are the key of the system: companies have to figure out ways to entice people to actively participate in their business. I think Andreas Weigend puts it in the right words in his article: "the world has shifted to a model of collaboration and explicit data creation" (The Social Data Revolution, Weigend A., Harvard Business Review Blogs, 05/20/2009)

Third, I believe that the next challenge is to process information and have it presented in a way that the human brain can get leverage from it. Indeed, as I emphasized in my first insight above, the human minds can only process so much information at a time and can only deal with a limited number of concepts or relationships. As the amount of data is growing exponentially today, Dr Carl Pablo highlights "an immense risk of cognitive overload" (Handling the Cornucopia, The Economist p. 13, 02/27/2010).
Relating to that, a lot of interesting development is done, notably in ways to visualize the data, to help reduce the amount of info the mind has to process (see Show me, The Economist p. 9, 02/27/2010) "If a picture is worth a thousand words, an infographic is worth an awful lot of data points".

Finally, I think that our society is undergoing a profond behavioral change through the use of social networks. Even though people can argue they are a lost of productivity ("personal use of social networks cost almost $2.3 billion a year in lost productivity" - Yammering Away at the Office, The Economist p. 9, 01/30/2010). The network effects that sites such as Facebook know makes it indispensable - and if it is not Facebook it will be another service! There is still much to be discussed, such as legal implications (New rules for big data, The Economist p. 13, 02/27/2010) but the social graph is definitely a breakthrough for society.

~ Louis Lecat

I have several insights gained from these readings: (1) individuals are no longer isolated atoms in cyberspaces, but rather our entire set of behaviors are linked with our real identities, social networks, peers, and other attributes. On the one hand, computation and algorithms are constantly improved our experience with information; on the other hand, we constantly contribute our information to help algorithms to provide better information for us. (2) personalization would be a trend for the internet development. the Internet will smartly know individuals' intents, motives, and behaviors better. (3) the constant information contribution through conscious expression (like commenting, sharing, self-presentation, or blogging) and unconscious ways (such as page viewing, social connections, geo-location information or mouse movements) make online collaboration universal and improve information transparency. In fact, people are constantly involved in collaboration, not only collaboratively filtering information, but also shape the entire information world. The computation and algorithms of such information will greatly improve transparency so as to have great political and societal impacts, like promoting political autonomy, political accountability, and low the economic cost.

These readings are wonderful and greatly expand my understandings about the social data. Information is the true power. Nowadays, almost all of a individual's behavior can be transferred into data and recorded in the database, such as interpersonal interaction, physical locations and movements, consumptions, and other all kinds of social activities. We should recognize the potential benefits and dark sides of mining such data. As Larry Page said, we have only used 1% of social data. On the technical side, we should develop better algorithms to compute and use the huge amount of data. On the social science side (especially communication field), we should better understand individuals' behaviors and activities through these data and improve people's experience with the data and the life quality.

~ Yushu Zhou

One insight I gained from the readings is how the rapid change in data usage and creation is enabling greater value creation across economic and social areas. From the article "Data, data everywhere" I learned that data is no longer viewed as solely an output of some process, but rather using the same data stream as both input and output creates a real-time feedback loop that drives value creation through insight.

From the Harvard Business Review blog post I learned that explicit data creation has shifted power from corporations and governments to the consumer (data creators). This has changed the way I view the wealth of data around me, especially how the data I choose to share (and the data I choose to keep private) can potentially have a powerful effect in shaping the decisions of corporations, governments and those connected to me.

~ Pat Lai