Samuel Arbesman writing in WIRED makes a useful suggestion. It’s an obvious one, and that is not intended as a sneer. Re-seeing the obvious is a great gift, as you may have noticed when you ask a friend to look at something you wrote and s/he spots a typo right off, a glaring one too, that changes the whole sense of what you meant. So, Mr. Arbesman usefully directs our attention away from the hoopla about the importance of big sets of data to a consideration of data over time…what he calls “long data”. If your doctor tells you that your blood values are a little odd, and that X is a little low and Y a little high, that might be better understood in a context in which, over a number of years, X and Y have always been that way, and at about the same value too, and you’re not sick and weren’t then. It’s a better picture. You’re a ‘low normal’, with a slight anemia, maybe. So, eat liver. He offers several examples, but the one that impressed me the most was that from the Atlantic cod fishery, which almost collapsed. People were paying attention to the yearly harvest but much less attention to tracking the harvest through time. They would not have been surprised to see declines, due to over-fishing. His point is that we should calm down about the Big and figure out ways to turn it into the Big and the Long.
http://www.wired.com/opinion/2013/01/forget-big-data-think-long-data/