Posts Tagged ‘Data Analysis and Visualisation’

The World Of Big Data – The Daily Dish | By Andrew Sullivan

December 20, 2010 Leave a comment

Great post on ‘The World Of Big Data’ by Andrew Sullivan -reproduced in full below.

In passing, a Government truly interested in developing the smart economy would engage in massive data dumps with the presumption that just about every piece of data it holds (excluding the most sensitive pieces of information) from ministerial diaries to fuel consumption records for Garda cars to activity logs for mobile phones to numbers of toilet rolls used in Government Departments would be dumped in realtime on to externally-interrrogable databases. This would be geek-heaven and would generate new technological applications beyond prediction and application. And the activity would be local – could an analyst sitting in Taiwan really make sense of local nuances? The applications would be universal, portable and saleable, however. They would seed a local high-tech industry – maybe even a local Irish Google. Can’t see the Civil Service going for it, though…

Elizabeth Pisani explains (pdf) why large amounts of data collected by organizations like Google and Facebook could change science for the better, and how it already has. Here she recounts the work of John Graunt from the 17th century:

Graunt collected mortality rolls and other parish records and, in effect, threw them at the wall, looking for patterns in births, deaths, weather and commerce. … He scraped parish rolls for insights in the same way as today’s data miners transmute the dross of our Twitter feeds into gold for marketing departments. Graunt made observations on everything from polygamy to traffic congestion in London, concluding: “That the old Streets are unfit for the present frequency of Coaches… That the opinions of Plagues accompanying the Entrance of Kings, is false and seditious; That London, the Metropolis of England, is perhaps a Head too big for the Body, and possibly too strong.”She concludes:

A big advantage of Big Data research is that algorithms, scraping, mining and mashing are usually low cost, once you’ve paid the nerds’ salaries. And the data itself is often droppings produced by an existing activity. “You may as well just let the boffins go at it. They’re not going to hurt anyone, and they may just come up with something useful,” said [Joe] Cain.

We still measure impact and dole out funding on the basis of papers published in peerreviewed journals. It’s a system which works well for thought-bubble experiments but is ill-suited to the Big Data world. We need new ways of sorting the wheat from the chaff, and of rewarding collaborative, speculative science.

[UPDATE] Something I noticed in The Irish Times:

PUBLIC SECTOR: It’s ‘plus ca change’ in the public service sector, as senior civil servants cling to cronyism and outdated attitudes, writes GERALD FLYNN:

…it seems now that it was just more empty promises – repeating similar pledges given in 2008. As we come to the end of yet another year, there is still no new senior public service structure; no chief information officer for e-government has been appointed; no reconstitution of top-level appointments has taken place; and no new public service board has been appointed [emphasis added].

So nothing will happen.

Europe geographically stereotyped: some nifty geographical data visualisation

September 28, 2010 Leave a comment

Europe geographically stereotyped (via Flowing Data):

Graphic designer Yanko Tsvetkov takes such notions of Europe in his series of stereotype maps, which themselves are stereotypes of stereotypes.

Original here (with high res maps).

Sergey Brin’s Search for a Parkinson’s Cure

July 7, 2010 1 comment

Sergey Brin’s Search for a Parkinson’s Cure.

Several evenings a week, after a day’s work at Google headquarters in Mountain View, California, Sergey Brin drives up the road to a local pool. There, he changes into swim trunks, steps out on a 3-meter springboard, looks at the water below, and dives.

Brin is competent at all four types of springboard diving—forward, back, reverse, and inward. Recently, he’s been working on his twists, which have been something of a struggle. But overall, he’s not bad; in 2006 he competed in the master’s division world championships. (He’s quick to point out he placed sixth out of six in his event.)

The diving is the sort of challenge that Brin, who has also dabbled in yoga, gymnastics, and acrobatics, is drawn to: equal parts physical and mental exertion. “The dive itself is brief but intense,” he says. “You push off really hard and then have to twist right away. It does get your heart rate going.”

There’s another benefit as well: With every dive, Brin gains a little bit of leverage—leverage against a risk, looming somewhere out there, that someday he may develop the neurodegenerative disorder Parkinson’s disease. Buried deep within each cell in Brin’s body—in a gene called LRRK2, which sits on the 12th chromosome—is a genetic mutation that has been associated with higher rates of Parkinson’s.

A slightly daft but nifty piece of data visualisation: Every country must be number one at something…

For the art-loving neuroscientist: From Scientific American – Michelangelo’s secret message in the Sistine Chapel: A juxtaposition of God and the human brain

Douglas Fields has a very interesting post at Scientific American – seems Michelangelo has hidden in many of his Sistine Chapel paintings illustrations of the dissected central nervous system:

At the age of 17 he began dissecting corpses from the church graveyard. Between the years 1508 and 1512 he painted the ceiling of the Sistine Chapel in Rome. Michelangelo Buonarroti—known by his first name the world over as the singular artistic genius, sculptor and architect—was also an anatomist, a secret he concealed by destroying almost all of his anatomical sketches and notes. Now, 500 years after he drew them, his hidden anatomical illustrations have been found—painted on the ceiling of the Sistine Chapel, cleverly concealed from the eyes of Pope Julius II and countless religious worshipers, historians, and art lovers for centuries—inside the body of God.This is the conclusion of Ian Suk and Rafael Tamargo, in their paper in the May 2010 issue of the scientific journal Neurosurgery.

The brainstem dissection looks convincing to me, and most especially the pons and medulla, but as Fields notes: ‘The mystery is whether these neuroanatomical features are hidden messages or whether the Sistine Chapel a Rorshach tests upon which anyone can extract an image that is meaningful to themselves. The authors of the paper are, after all, neuroanatomists. The neuroanatomy they see on the ceiling may be nothing more than the man on the moon.’

Concealed Neuroanatomy in Michelangelo’s Separation of Light From Darkness in the Sistine Chapel,” by Ian Suk and Rafael J. Tamargo in Neurosurgery, Vol. 66, No. 5, pp. 851-861.

Der Spiegel: ‘The Climategate Chronicle: How the Science of Global Warming Was Compromised’

May 25, 2010 2 comments

A fantastic story from Der Spiegel:  ‘The Climategate Chronicle: How the Science of Global Warming Was Compromised’ By Axel Bojanowski.

To what extent is climate change actually occuring? Late last year, climate researchers were accused of exaggerating study results. SPIEGEL ONLINE has since analyzed the hacked “Climategate” e-mails and provided insights into one of the most unprecedented spats in recent scientific history.

Is our planet warming up by 1 degree Celsius, 2 degrees, or more? Is climate change entirely man made? And what can be done to counteract it? There are myriad possible answers to these questions, as well as scientific studies, measurements, debates and plans of action. Even most skeptics now concede that mankind — with its factories, heating systems and cars — contributes to the warming up of our atmosphere.

But the consequences of climate change are still hotly contested. It was therefore something of a political bombshell when unknown hackers stole more than 1,000 e-mails written by British climate researchers, and published some of them on the Internet. A scandal of gigantic proportions seemed about to break, and the media dubbed the affair “Climategate” in reference to the Watergate scandal that led to the resignation of US President Richard Nixon. Critics claimed the e-mails would show that climate change predictions were based on unsound calculations.

Although a British parliamentary inquiry soon confirmed that this was definitely not a conspiracy, the leaked correspondence provided in-depth insight into the mechanisms, fronts and battles within the climate-research community. SPIEGEL ONLINE has analyzed the more than 1,000 Climategate e-mails spanning a period of 15 years, e-mails that are freely available over the Internet and which, when printed out, fill five thick files. What emerges is that leading researchers have been subjected to sometimes brutal attacks by outsiders and become bogged down in a bitter and far-reaching trench war that has also sucked in the media, environmental groups and politicians.

SPIEGEL ONLINE reveals how the war between climate researchers and climate skeptics broke out, the tricks the two sides used to outmaneuver each other and how the conflict could be resolved.

A few items from ‘Flowing Data’: Marge Simpson is Europe in disguise; Air traffic rebooted in northern Europe & Discuss: Powerpoint is the enemy?

Marge Simpson is Europe in disguise (seriously).


Air traffic rebooted in northern Europe

[By Nathan Yau – Apr 26, 2010: ‘Air traffic has returned to normal levels in northern Europe, and planes fill up the sky once again. Ito world, who has been doing some great stuff lately, visualizes the reboot of air traffic. We start to see some planes on April 18, and by April 20, everything is back to normal.’]

Ireland starts to re-appear at around the 00.30 sec mark.

Discuss: Powerpoint is the enemy? This graphic has to be seen to be believed. An object lesson in how not to do infographics or visualise relationships.

Check here for some superb guidelines on how to do data visualisation. And The Scientist has a helpful article on scientific data presentation –  ‘Pimp Your Powerpoint’ (which promises to help you design ‘…attention-grabbing presentations that stand out from the typical snoozers’).