From a trading desk in London, Paul Hawtin monitors the fire hose of more than 340 million Twitter posts flying around the world each day to try to assess the collective mood of the populace.
The computer program he uses generates a global sentiment score from 1 to 50 based on how pessimistic or optimistic people seem to be from their online conversations. Hawtin, chief executive of Derwent Capital Markets, buys and trades millions of dollars of stocks for private investors based on that number: When everyone appears happy, he generally buys. When anxiety runs high, he sells short.
Hawtin has seen a gain of more than 7 percent in the first quarter of this year, and his method shows the advantage individuals, companies and governments are gaining as they take hold of the unprecedented amount of data online. Traders such as Hawtin say analyzing mathematical trends on the Web delivers insights and news faster than traditional investment approaches.
The explosion in the use of Google, Facebook, Twitter and other services has resulted in the generation of some 2.5 quintillion bytes each day, according to IBM.
“Big data,” as it has been dubbed by researchers, has become so valuable that the World Economic Forum, in a report published last year, deemed it a new class of economic asset, like oil.
“Business boundaries are being redrawn,” the report said. Companies with the ability to mine the data are becoming the most powerful, it added.
While the human brain cannot comprehend that much information at once, advances in computer power and analytics have made it possible for machines to tease out patterns in topics of conversation, calling habits, purchasing trends, use of language, popularity of sports, spread of disease and other expressions of daily life.
“This is changing the world in a big way. It enables us to watch changes in society in real time and make decisions in a way we haven’t been able to ever before,” said Gary King, a social science professor at Harvard University and a co-founder of Crimson Hexagon, a data analysis firm based in Boston.
The Obama campaign employs rows of people manning computers that monitor Twitter sentiment about the candidates in key states. Google scientists are working with the Centers for Disease Control and Prevention to track the spread of flu around the world by analyzing what people are typing in to search. And the United Nations is measuring inflation through computers that analyze the price of bread advertised in online supermarkets across Latin America.
Many questions about big data remain unanswered. Concerns are being raised about personal privacy and how consumers can ensure that their information is being used fairly. Some worry that savvy technologists could use Twitter or Google to create false trends and manipulate markets.
Even so, sociologists, software engineers, economists, policy analysts and others in nearly every field are jumping into the fray.
via ‘Big data’ from social media, elsewhere online take trend-watching to new level – The Washington Post.
That’s very impressive, to be sure, but do you think all of this “data” is really equivalent to a natural resource? The stock trader who buys when the Twitter traffic is happy and sells when it’s sad has been making money, but why not buy when people are sad (picking up bargains when people are giving up on the world and dumping their stocks for cheap) and selling when they are happy (taking advantage of their irrational exuberance)? That is to say, is his data mining resulting in an application that is all that scientific? And in what sense is a Twitter tweet necessarily equivalent to hard data? Can one control for irony, sarcasm, and jokes? I’m not denying that there may be some very useful information amidst all of the clutter, but still. . . .
Ad-Free & Subscriber Log-In