What is Big Data – Part 2 - dfcrosby/Blogs GitHub Wiki
In a prior blog post we talked a little bit about what makes Big Data big. As we saw, the challenges of Big Data come from velocity and variety as well as volume. We are seeing trends from Big Data—trends and other data about the Big Data add to the volume, making Big Data not only bigger but better. So Big also refers to the impact that Big Data can have, due in no small part to Big Analytics. But where does the data come from and how are we going to put it to good use?
No discussion about Big Data would be complete without including the Internet of Things (“IoT”). And I want to focus the discussion on the Internet of Useful Things. Just because we have enough IP addresses to enable every tree, shrub and blade of grass to be connected to the internet, does not mean we should. Smart uses of data and analytics can improve the things we take for granted, like energy efficiency, environmental sustainability and health care, ultimately improving the quality of life for many.
The Internet of Things refers to connecting devices to the internet so they can be managed remotely and, more importantly, receive data that enables them to do a better job of performing their function. One example is your thermostat, the device that hangs on the wall and controls your heat and/or air conditioning. The original thermostat measured the temperature in the room and turned the heat or air conditioner on or off depending on the set temperature. Maybe you have a “smart” thermostat that you can program to turn the heat down when you go to work and back up just before you come home. The IoT thermostat (Nest or Honeywell) can monitor how you set it and learn what temperature you find most comfortable. It can detect when you are home, so it can adjust the temperature for your comfort while you are home and turn itself down to save energy when you are away. Maybe it can communicate with your smart phone, so it knows where you are and turn on the heat or air conditioning to make the house comfortable when you come home. Over time, the thermostat can also access weather data to find the outside temperature so it can determine how long it will take to warm or cool the house for your arrival. When you turn down the thermostat, it can send you a text message or e-mail suggesting that you open a window because it’s cool outside. As you head toward the door, a light over your coat closet will flash to indicate that you should take a coat or the umbrella in the umbrella stand will flash to let you that rain is in the forecast and you should take the umbrella. Or maybe the appointment reminder will include the weather forecast and suggest that you take your coat or umbrella. None of these things requires a large volume of data. Indeed, while the data is rather small, the variety of sources, and analytics of the data from diverse sources is what makes this data big in terms of impact on our lives. Providing this service for many people will require a lot of data. We can also use this data (aggregated over time) to analyze our homes for energy efficiency and determine changes in our behavior or improvements to the system that help to make it more efficient.
Big Data analytics hope to make big improvements in the quality of health care. Wearable sensors can monitor your vital signs and activity levels and help you to achieve physical fitness goals. Some, like the Reebok CHECKLITE, provide new information to help us make better decisions. The promise is that the analytics can identify indicators or trends that might predict an adverse event enabling you to seek medical attention in advance of a potentially life threatening condition.
What we are learning from Big Data is that our prior assumptions based on discrete or isolated data points may not be correct or that there is additional valuable insight to be gained by informing those assumptions with other data from diverse sources. By combining these diverse information sources we can provide new insights into what is happening in complex systems, such as the human body.
In addition, this data can be aggregated over many users to study trends and help modify behavior, helping us to make informed choices. However, we need to be careful about how this data is used and protected in order to preserve our privacy.