Proponents of open government data policies found plenty to praise in the Obama administration's second Open Government National Action Plan (NAP), but at least one advocate says agencies still have a lot of work ahead to make good on those plans.
Vineeta Shetty, Chief Creatrix, Vistara, 11/19/2013 Comment now
In common with many emerging countries, Ghana faces a problem in the way public sector agencies manage existing infrastructure assets. Two years ago, with a view to update management practices and to plan for long-term infrastructure needs, the Ghanaian National Development Planning Commission (NDPC) and Ministry of Finance and Economic Planning (MOFEP) ...
The White House Office of Science and Technology Policy (OSTP) and Networking and Information Technology R&D program (NITRD) on Tuesday introduced a slew of new big data collaboration projects aimed at stimulating private-sector interest in federal data. The initiatives, announced at the White House-sponsored "Data to Knowledge to Action" event, are ...
Would citizen BrianRoden please report to reception -- you left your phone in Room 101.
BrianRoden gave us the late September/early October chuckles with his entry in the George Orwell: 2013 Caption Contest. You can see the winning entry below.
Big data (and sometimes just plain old data) was the subject of a panel discussion at yesterday's CityLab event at the Conrad Hotel in lower Manhattan. Enthusiasm for using city data to create just about any application one can imagine has been high in recent years, but the panelists were frank in admitting that we're still several hurdles away from ...
Robert Plant, Associate Professor, School of Business Administration, University of Miami, 9/25/2013 Comment now
Twitter is not just a mechanism for sending 140-character messages. It is a community, a culture, and a social network where less can be more, and more is often a bad idea. On Sept. 12, the company filed an S-1 document with the SEC as a first step toward going public.
The S-1 is a mechanism for registration of a company's securities with the SEC, ...
Ariella Brown, Technology Blogger, 8/9/2013 Comment now
One of NASA's main areas of research is the earth's climate, and clouds are a significant factor in that study. The problem is that while NASA's satellite images are very accurate about earth readings, the evanescent and localized nature of clouds can throw their numbers off. To get better data about clouds, NASA is enlisting the help of people with a ...
Daniel D. Gutierrez, Data Scientist, 7/25/2013 Comment now
In honor of Big Data Republic's very well-timed Machine Learning Month, I'd like to offer the following mini case studies from my data science consulting practice. Each use case example demonstrates a different orientation of machine learning that I used to answer a client question, solve a business problem, or uncover previously known knowledge lurking ...
Robert Plant, Associate Professor, School of Business Administration, University of Miami, 7/24/2013 Comment now
Data Privacy: The 21st Century's Oxymoron? , was an early topic of discussion on Big Data Republic, centering around the concept of personal data and its attraction to spurious, unauthorized, and boundless analysis by the corporations that collect it. The debate was bifurcated by the Atlantic -- the North American view of data surrounds corporations and ...
The recent events surrounding the NSA and its now publicly known Prism project have put big data in the limelight once again. I believe there are lessons to be learned from this development, both negative and positive.
For example, it emphasizes the critical role that big data has in business and society as a whole -- now and in the future. Our country's information security services have the task of finding the threat signals among the noise, much as businesses must find valuable signals in the noise of their digital footprint.
However, the parallel does not end there. The recent events have shown that it is vital for a big data project to be transparent to its users. You cannot treat it as a black box, because your users cannot put their trust in it. Any big data project, whether it involves your citizens, your staff, or your clients, must have clear and transparent processes in place.
- Ben Pottier, search technology specialist, Funnelback UK
At the Big Data Show earlier this year, we caught up with Duncan Ross, director of Data Science at Teradata International.
He was excited to talk to us about the launch of DataKind in the UK. DataKind is a community of brilliant data scientists who have taken on the challenge of implementing data-led projects "in the service of humanity."
Pig basically simplifies the processes needed to get analytics done through Hadoop on your big data sets.
Like the animal, Pig is not a fussy eater, getting its name from its ability to crunch through data, no matter what form it takes. It acts as a scripting interface to
Hadoop, meaning a lack of
MapReduce programming experience won't hold you back.
Example: Harvey works in a government office, looking to formulate new solutions for his city's parking problems. He knows how to use data, but writing his own mapper and reduce functions is a little beyond him.
Luckily, he's been set up with access to the databases through Pig, meaning he can draw on sources like parking ticket records and population density maps. Taking advantage of Pig's eat-anything attitude, he can also mine topics from a call for email suggestions his department sent to local residents, as well as sensor information about the amount of traffic on the roads. In spite of his limited programming capabilities, Pig allows Harvey to query these data sets and sketch out some draft suggestions he can use to alleviate the local parking problems.