"In a lot of call centres, the agent is just a human telephony switch," said Jeremy Payne, International Group Marketing Director of Enghouse. He told me this as my eyes were still bleary, making a pre-9 a.m. start at the Call Centre Expo in London.
In Big Data Republic's search for areas yet to take full advantage of big data technology, ...
It's time to look at objects and buildings as quasi-sentient things that can communicate and perform tasks based on what people tell them.
In June, Future Cities managing editor Mary Jander listed the Hello Lamp Post project in Bristol, England, as one of several "silly" city projects that "may be fun, but they should not be taken as anything heavier ...
Greg Todd, CTO, Revolution Analytics, 8/26/2013 Comment now
It might just be time to simplify your business's approach to data.
Too much of a good thing?
For a number of years, organizations have focused on the ingestion of data and content into systems focused on enterprise resource planning, enterprise content management (ECM), customer relationship management (CRM), enterprise data warehouse (EDW), and ...
Olivier Janus, Global Data Director, Havas EHS, 8/20/2013 Comment now
In an increasingly digital world, smart brands will have some kind of data strategy in place in order to respond to the changing market landscape. However, these data strategies are so often only a matter of survival and do not give the competitive edge that a brand needs to thrive in an increasingly crowded market place.
Understanding customers with ...
The biggest hurdle to leveraging big data results isn't in physically desiloing existing data, but rather in metaphorically desiloing the way different business units work. So much so that last month Lynn Collier, Director of Enterprise Platform and Software Solutions at Hitachi Data Systems, told me something that nearly knocked me out of my chair.
Business cultures are failing to keep up with the promise of big data technologies, a fact never more evident than when examining the inability to achieve efficient data integration. As aggregation services appear, it lessens the load, but is it at the cost of progress?
Data governance is the set of rules a business creates to ensure data is maintained in a meaningful and responsible way. While the main motivator for this is usually legal compliance, governance also plays an important role in maintaining data quality.
Let's explore this with an example
Sarianne is hard at work in the financial markets. After some exploratory queries of her stored data, she's realised it's a mess. She needs a system to make sure the errors and lazy management stop as soon as possible.
Creating a data governance plan allows her to dictate how certain data will be collected and initiate plans to monitor its quality as it is entered. Crucially, a well drawn governance plan will also allow her to hold departments and team members responsible for their own areas of data. It will put a stop to data issues being brushed under the carpet or dismissed as being a wider company problem.
These rules will help Sarianne going forward, but what about all that rotten data she's already got?
Recently, we caught up with Ben Pottier, Search Technology Specialist at Funnelback UK, to pick his brain on how SMEs can leverage big data.
He acknowledges that it's easy for smaller organizations to feel isolated from real big data benefits by the jargon and hype currently in circulation. However, he also maintains that SMEs are almost in a better position to leverage big data technologies, because of the ability to push big data technologies to grow their business.
He concludes "don't do big data with a big bang approach." Are you an SME leveraging big data, or looking to do so? Do you agree with Ben's advice? Let us know below.
Data Munging is boring but necessary, as it involves getting raw data ready so that the exciting work of analysis can be done.
Let's look at this with an example
Sara's hard at work on data from her local schools, but oh boy, what a mess! Attendance registers are pretty clean, but each school uses different formatting for keeping track of grades and discipline, some keep dates in different formats -- not to mention all the unstructured data sitting there in the form of comments and social media interactions.
Before Sara can get anywhere with this, she needs to do some serious munging, or wrangling. That means getting this information to match up with each other -- so she'll need to extract all of this raw data and run algorithms on it to match up with her preferred columns and rows -- and depositing the finished dataset in her data store so she can start running queries.
Hive allows users to take advantage of Hadoop using a language similar to SQL, something most relational database developers have in their toolkit.
Let's examine how it helps with an example
Michael is a medical researcher who has had experience running relational data-bases, but knows that real insight could be found by accessing more data. After years of lobbying, he's managed to create a project which combines data from a variety of hospitals.
The upside is he has much more data to experiment with, the downside is that to get results quickly, he's having to use Hadoop, something he is unfamiliar with.
However, by leveraging Hive, he can write instructions in Hive Query Language (HQL), which isn't a huge leap from the SQL he knows so well. That means less time studying up on his language, and more time looking for correlations that can help patients recover quicker.
Jamie Turner, CTO at Postcode Anywhere, caught up with us at The Big Data Show this spring to give us his take on big data for SMEs.
He makes some really interesting points. One that sticks with me: "Not using elaborate stuff like multinationals and governments" is probably an underdelivered message in this space. It's easy to salivate at the opportunities exploited by Walmart and Tesco, but having a clear understanding of your own business needs (and not Walmart's or Tesco's) will be the best way to reap rewards.
The recent events surrounding the NSA and its now publicly known Prism project have put big data in the limelight once again. I believe there are lessons to be learned from this development, both negative and positive.
For example, it emphasizes the critical role that big data has in business and society as a whole -- now and in the future. Our country's information security services have the task of finding the threat signals among the noise, much as businesses must find valuable signals in the noise of their digital footprint.
However, the parallel does not end there. The recent events have shown that it is vital for a big data project to be transparent to its users. You cannot treat it as a black box, because your users cannot put their trust in it. Any big data project, whether it involves your citizens, your staff, or your clients, must have clear and transparent processes in place.
- Ben Pottier, search technology specialist, Funnelback UK
Sam Zindel, Data Strategist at iCrossing Digital Marketing, filled us in on how supermarket delivery giant Ocado uses big data to identify and serve specific content to vegetarians once they spot them on their website.
This was part of Sam's talk at The Big Data Show about putting the customer at the heart of digital marketing, as well as making the most of the data you already have.
On the opening day of the Big Data Show, Mike Cornwell, CEO of The IDM, was generous enough to give us some time to discuss his afternoon panel session. He also offered a word of caution on the state of marketing data. We're all getting excited about big data, but it seems most people still can't deal with their small data in the best possible manner.
"Just knowing enough to find some insight from information and using it intelligently for marketing still seems to be beyond a lot of organizations," he said.
Does this resonate with your business? Have you got the small data figured out before you invest time in de-siloing and bringing more information together?
Big data is awash with acronyms at the moment, none more widely used than HDFS. Let's cut to the chase... it stands for Hadoop Distributed File System.
This is the system of distributing files that allows Hadoop to work on huge data sets at speed. It spreads blocks of data across different servers, as well as duplicating those blocks of data, and storing them distinctly.
Let's see why with an example.
Sarianne works in the financial markets, and runs a lot of predictive models to make sure her investments are minimum risk.
Utilising HDFS, her queries through Hadoop can run quickly because the data blocks are stored separately -- meaning all the computation can happen in one go, rather than queuing up behind each other.
As an added benefit, if one server fails (as one is bound to, given the amount of servers and disk drives needed to run big data projects) it won't stop Sarianne's models from pulling the data they need, because HDFS duplicated those blocks -- meaning Hadoop can return Sarianne's results in double quick time.
Pig basically simplifies the processes needed to get analytics done through Hadoop on your big data sets.
Like the animal, Pig is not a fussy eater, getting its name from its ability to crunch through data, no matter what form it takes. It acts as a scripting interface to
Hadoop, meaning a lack of
MapReduce programming experience won't hold you back.
Example: Harvey works in a government office, looking to formulate new solutions for his city's parking problems. He knows how to use data, but writing his own mapper and reduce functions is a little beyond him.
Luckily, he's been set up with access to the databases through Pig, meaning he can draw on sources like parking ticket records and population density maps. Taking advantage of Pig's eat-anything attitude, he can also mine topics from a call for email suggestions his department sent to local residents, as well as sensor information about the amount of traffic on the roads. In spite of his limited programming capabilities, Pig allows Harvey to query these data sets and sketch out some draft suggestions he can use to alleviate the local parking problems.
In the first of a series of interviews with business leaders who leverage big data, we talk to James Robinson, CTO and co-founder of OpenSignal.
OpenSignal combines big data technologies and sensor data from mobile phones to give insight to both mobile consumers and telecommunications giants. Robinson is also a contributing writer on Big Data Republic.