Great post on ‘The World Of Big Data’ by Andrew Sullivan -reproduced in full below.
In passing, a Government truly interested in developing the smart economy would engage in massive data dumps with the presumption that just about every piece of data it holds (excluding the most sensitive pieces of information) from ministerial diaries to fuel consumption records for Garda cars to activity logs for mobile phones to numbers of toilet rolls used in Government Departments would be dumped in realtime on to externally-interrrogable databases. This would be geek-heaven and would generate new technological applications beyond prediction and application. And the activity would be local – could an analyst sitting in Taiwan really make sense of local nuances? The applications would be universal, portable and saleable, however. They would seed a local high-tech industry – maybe even a local Irish Google. Can’t see the Civil Service going for it, though…
Elizabeth Pisani explains (pdf) why large amounts of data collected by organizations like Google and Facebook could change science for the better, and how it already has. Here she recounts the work of John Graunt from the 17th century:
Graunt collected mortality rolls and other parish records and, in effect, threw them at the wall, looking for patterns in births, deaths, weather and commerce. … He scraped parish rolls for insights in the same way as today’s data miners transmute the dross of our Twitter feeds into gold for marketing departments. Graunt made observations on everything from polygamy to traffic congestion in London, concluding: “That the old Streets are unfit for the present frequency of Coaches… That the opinions of Plagues accompanying the Entrance of Kings, is false and seditious; That London, the Metropolis of England, is perhaps a Head too big for the Body, and possibly too strong.”She concludes:
A big advantage of Big Data research is that algorithms, scraping, mining and mashing are usually low cost, once you’ve paid the nerds’ salaries. And the data itself is often droppings produced by an existing activity. “You may as well just let the boffins go at it. They’re not going to hurt anyone, and they may just come up with something useful,” said [Joe] Cain.
We still measure impact and dole out funding on the basis of papers published in peerreviewed journals. It’s a system which works well for thought-bubble experiments but is ill-suited to the Big Data world. We need new ways of sorting the wheat from the chaff, and of rewarding collaborative, speculative science.
[UPDATE] Something I noticed in The Irish Times:
PUBLIC SECTOR: It’s ‘plus ca change’ in the public service sector, as senior civil servants cling to cronyism and outdated attitudes, writes GERALD FLYNN:
…it seems now that it was just more empty promises – repeating similar pledges given in 2008. As we come to the end of yet another year, there is still no new senior public service structure; no chief information officer for e-government has been appointed; no reconstitution of top-level appointments has taken place; and no new public service board has been appointed [emphasis added].
So nothing will happen.
Another nice follow-up to the post below from Chris Dillow on the forthcoming cuts to the UK science budget.
[blogpost reproduced in full]
Forget for one moment the fall out of Vince Cable’s speech on science funding.
http://www.guardian.co.uk/science/2010/sep/08/vincent-cable-science-budget-cutsForget the fact the government has ignored the recommendations of last parliaments Science & Technology committee.
Forget the fact that a Liberal Democrat Secretary of State with a remit for science has utterly reneged on Liberal Democrat commitments to science that were made to the electorate.
Forget even the fact that the Secretary of State does not appear to understand the criteria used for assessing the research that his department is responsible for.
Forget all this. For it is not the most weird thing about the speech.
Superstition and irrational prejudice about the natural world are rarely far from the surface and scientists help inoculate society against them – a far from risk-free task as Simon Singh and others have discovered.
Is Cable saying scientists are inherently more rational and less prejudiced than other members of society? This would be contentious. Or does he mean that science can provide rational reasons for events and occurrences once attributed to supernatural forces and that evidence undermines prejudice. This would be true, but you only have too look at the public’s understanding of genetic modification, climate change or immigration to see that in practice throwing facts in somebody’s face is not always the most efficacious way of changing their mind.
And what has Simon Singh got to do with anything? He was sued by a bunch of quacks whose reputation now lies in tatters. This was a terrible abuse of libel law and it needs to be reformed, but this is not the remit of the Secretary of State for Business, Innovation & Skills. Perhaps Cable was worried that libel law can stifle science based criticisms? It can, but I suspect that focussing science on fields that generate short term profit against all the evidence may in fact represent a far greater threat.
Did Cable really think that a poorly conceived nod to skeptical activism and libel reform would sweeten the bitter taste this renunciation of his party’s purported principles has left in the mouths of most scientists?
The Role of PhDs in the Smart Economy – Advisory Science Council – An Comhairle Eolaíochta | www.sciencecouncil.ie
Sometimes, important documents get less publicity than they deserve:
The Role of PhDs in the Smart Economy
Date: 15 December 2009
Source: Advisory Science Council
A flow of knowledge and human capital between enterprise, higher education and the public sector is essential to firmly embed enterprise in the knowledge economy and ensure the recent investment in the research infrastructure is leveraged for economic development in the long term. This report examines the skills that businesses require from 4th level Ireland, the roles in enterprise that are filled by PhD graduates and the barriers that prevent their move into enterprise.
From the press release:
A new report launched today by the Advisory Council for Science, Technology and Innovation (ACSTI) has found that Irish R&D firms employing PhD researchers have rates of patenting 2.5 times greater than similarly active firms which do not employ PhD researchers and have vastly higher collaboration rates with both Higher Education Institutes and other firms. While only 29% of R&D active firms employed PhD researchers in 2007, these companies accounted for 70% of business expenditure on R&D. The report, the Role of PhDs in the Smart Economy, highlights Ireland’s need to maintain a competitive output of PhDs in relevant disciplines in line with other developed countries and sets out a list of recommendations to maximise the development of 4th level education in Ireland and its critical relevance to enterprise and society.
Major Recommendations are below the fold.