Andreas Weigend, Social Data Revolution | MS&E 237, Stanford University, Spring 2011 | Course Wiki

DF1: Instrumenting the World

Due: April 7, 2011

Background: Next Tuesday, Rick Smolan, a former Time, Life, and National Geographic photographer best known as the creator of the "Day in the Life" book series, will be joining us in class as we talk about "BIG DATA". Every year, Rick and a team of hundreds of journalists focus their attention on emerging trends and topics using state-of-the-art technology to capture the "human face" of that topic. His projects include The Obama Time Capsule, America at Home, Blue Planet Run, America 24/7, One Digital Day, 24 Hours in Cyberspace, The Power to Heal and From Alice to Ocean.

Your job is to present ONE idea: Imagine that on September 13, 2011, 10,000,000+ people will put an application on their phones that will measure some metric about each person, their interactions with others, their interactions with the world, etc. If you could write that application, what metric would your application measure about each person? Keep in mind the class is on "BIG DATA" and how it is beginning to change the way humanity tackles its biggest problems. Note that the best ideas will be considered for REFLECTIONS IN A DIGITAL MIRROR, so do your best and think of something super interesting and creative to mesaure!

Please submit your responses to the Google Form http://bit.ly/SDR_DF1_2011



Notable Answers

"Parse all of the text of sent text messages. create regional tag-cloud-esque frequency clouds of the words in the text messages and present temporally. not sure what the results will show but i'm sure this would be very interesting. trying to do what twitter wants to do but more effectively because people don't have to opt in."

"For one day, each time a person is annoyed by the status quo, i will ask them to open the app and briefly describe the situation that annoyed them. It could be something like:
  • Stuck in traffic
  • Annoyed at the email deluge, etc etc

Just get a list of anything and everything that ticked the person of. Then, use NLP and Data Mining techniques to cluster the annoyances. At the end, we will have a big prioritized (prioritized by frequency) list of the common problems\inconviniences faced by people in a day-to-day basis."

"To record a 30 sec sound bite once each day. This sound bite can be anything from the 'om' in a yoga class, the punchline of a Ted talk, sounds of birds, traffic in Bombay, sounds of waves of a beach in Bali, or more serious sound snippets: Sounds of Palestine, sounds of conflict in Libya, sounds of an earthquake aftermath. Each day these sounds can either be recorded automatically (so that the user doesn't have to think about it), or the user can choose to record specific sounds.

This project is about three things:
1) Preserving the history of a year through a different medium - sound is a very under-utilized way of capturing a moment. Re-listening to previously captured sounds does take you back to that moment, it can be a very powerful trigger. Sound is also a perspective that hasn't been heavily analyzed. When is a city louder? Was the world slightly louder after India won the world cup?
2) Demonstrating that there is beauty in the everyday. Even mundane sounds listened to carefully in isolation are quite beautiful (close your eyes an notice the sounds or silence that is occurring with you right now).
3) Whole is greater than the sum of the parts -- what 'Big Data' is all about. After a years worth of collection from 10M phones, you can mix the sounds together to produce a total novel concept of a 'symphony.' Individuals can also track their own sounds hearing their entire year in a few mins from a unique perspective. You could also overlay all the sounds and see when / where the world was harmonious and when / where we were cacophonous.

Ideally, you could overlay geo-locations over the sounds to produce a symphony map, you could listen to different sounds in different parts of the world, or hear the entire time-sound series. I think this would be a great addition to the 'pulse of the planet'."

"We make BIG DATA every day through each transaction. Every payment at a coffee store, gasoline station, pharmacy, etc contains much more data than what a typical receipt says (how much was spent). The big question is why do we buy what we buy? What external factors affect us to induce our purchasing habits? If we could mesh the data of a person’s transaction throughout a day with the location of each, their friends location and transactions, the weather, events of that day, very interesting conclusions could be drawn. From why a person buys something at a specific point in time, for example on a rainy Tuesday a person is more probable to buy this type of coffee, to what they buy together at different points of a day to build physical recommendation systems (such as amazon’s service), to invisible group interactions. Currently, with the hype of collective group buying and daily deals, the process of people getting together to buy because it makes it cheaper is becoming common in our lives. But the main problem is that I don’t specifically know what my friends buy or intend to buy at any point in day, or where they are. Maybe analyzing these group patterns of how a person’s 1st degree connection had the same exact behavior but 30 minutes after, could be an indicative of how to connect people together to buy, share, interact in a physical group environment. Maybe noticing that my 2nd degree connection which I don’t know is at the same place as me buying/consuming the same or complementary item could be automatically recognized and a café for example could find a way for both of them to connect and interact having a common ground to talk about. The sensors to use in the app would be to track the locations where the person goes in a day, maybe connect to the bank account to log the purchase transactions and use the person’s contacts (address book) and social network connections to gather data. Intent to buy from persons before a purchase could also be explored by having them scan the barcodes of every item they look at or consider. The question is, do our friends and close group connections go to the same places, purchase similar things, have similar social interactions but during different points in a day? How do social groups combine and dissolve throughout the day in different locations and settings? How can you use this and the purchasing habits of people to create real-time collective buying a possibility in the real world?"

"My idea is for an application that measures the flow of money amongst the 10,000,000+ users. With a userbase of 10 million people, it is highly likely that a subset of users will be engaging in monetary transactions with one another, whether it involves buying or selling. All electronic transactions (credit card payments, etc.) already generate records/receipts with the card companies, but if it is practical, it can also be extended to cover currency/banknotes, which cannot be easily monitored, by "tagging" them, as shown in Rick Smolan's "MIT Trash Track" example. The exchange of money is clearly a key interaction between people, leading to insights about consumer spending habits, shopping trends (hot toys and gadgets) as well as the flow of money. Economists use the term "velocity of circulation" (Wikipedia) to refer to the frequency at which money changes hands, but it is difficult to directly measure in the real world due to the difficulty of tracking down currency. If the application is expanded to include tagging bills, it would be possible to track the geographical movement of currency notes amongst a sizable population sample."