It’s always tempting to make big data projects very complex -- but often that’s beside the point. Our recent project exploring prescription in the UK's National Health Servive is a great example of how simplicity works
Just working with the full volume and detail of a big dataset, presented in a helpful way, can be enough to add plenty of value, and can be much easier for business users to understand and to immediately get value from than some deep analytics, which can be harder for the business to make sense of and to trust.
The surprising value of simplicity is one of those lessons that I relearn every couple of projects -- going into new clients braced to offer the latest and greatest science, and frequently finding that keeping it simple is much more important and useful for them. It was particularly brought home this week when we released a piece of NHS prescriptions analysis, which was approvingly written up in the Financial Times, Economist, and by Francis Maude the Cabinet Office Minister.
The analysis explored variation of prescriptions of different types of statins in the UK: Proprietary versions cost 10x to 20x more than generics, so the NHS has been trying for some time to reduce their use. Working with practicing doctors and technologists at Open Healthcare UK and with Ben Goldacre, as well as the new Open Data Institute, we were able to identify and to map for the whole of England exactly where unexpected variation was happening. By enabling simple comparisons and drilling down into local prescribing patterns, we were able to offer something genuinely useful to NHS staff and leadership, which points to hundreds of millions of pounds of savings -- even though there is definitely no rocket science involved in the analysis. In fact, according to high-level analysis by senior doctors in BMJ, the total issue is worth more than £1 billion a year, so it’s well worth building any tools that will help to generate insights and to affect prescribing behavior.
The analysis also reminded me of two other important life lessons that come up frequently when working in big data:
The effort of getting and using big data doesn’t necessarily match the impact. This whole project was based on open, public data from the NHS Information Centre, which is there as a public resource. We didn’t have to do anything special to get hold of it -- all we needed to do was to be smart about how we used it and understand technically how to handle data that was in the tens of millions of rows. Using open data also had the nice side effect that we didn’t need to worry so much about accidental privacy or personal impacts, which are so fraught in the health field, as Ariella Brown pointed out on this site last week in Big Data Health Hazards.
Domain knowledge is vital. Without working with doctors, we’d never have known that prescribing of proprietary drugs was such a major cost issue. Without them explaining that, and pointing us to the right questions, this analysis would have been meaningless, regardless of what kind of killer tools and techniques we’d had. With them, it has become something hugely useful.
Doing deep analysis with data is a lot of fun -- but it’s always even more important to hold on to the value of simplicity.
deastman,
User Rank: Bit Player 12/31/2012 | 10:56:01 PM
Re: big darta savings Ariela,
I don't think they needed this type of article to come to the realization as you mentioned before. I feel it's just one more of a slick marketing campaign by drug companies . I've been in a Drs office when a pharmacutical rep comes in. They go on great fancy lunches, trips you name it no price is too high. As long at that doctor agrees to push the drug the rep wants.
So to flip the argument of how collecting so much data may be useful. So much of this 'innacuous' information can be collected eventually the plans can become clear that in fact now the unsuspecting person/people will get flodded with ads junk mail and so on.
technetronic,
User Rank: Exabyte Executive 12/31/2012 | 3:34:19 PM
Re: Yes, Keep It Simple Where are the consumers and citizens in that equation? I think that's where all power ultimately derives from but consumers are generally presumed to be reactionary to marketing and lobbied policies, instead of being seen as the actors that they could be.
What role do consumers play in UK healthcare? In the US, I feel that consumers are powerful but equally powerfully influenced by marketing and pharma-inspired news articles.
alvb1227,
User Rank: Petabyte Pathfinder 12/31/2012 | 11:48:19 AM
Re: Yes, Keep It Simple I think the biggest issue with analyzing big data is that many see it as a "big complicated issue." As a result, they often forget what the primary focus should original was - why do this? What was the original goal? Many times, if everyone is constantly reminded of the original goal, it helps to keep the project on track. Yes, simplicity is definitely key.
technetronic,
User Rank: Exabyte Executive 12/29/2012 | 3:35:11 PM
Re: Yes, Keep It Simple Yes, agreed, and glad that you pointed out that this kind of processing can be fun and exciting, once it's boiled down to a simple starting point.
With all new technologies, especially those digital, I hear inklings of the holy grail, when in reality there is never going to be a holy grail of a technology because they're just tools.
Does anyone have good writeups on how advertising networks learn to target ads? Those are (I hope) good examples of machine learning in a big data age.
netcrawl,
User Rank: Exabyte Executive 12/29/2012 | 6:57:41 AM
Re: Yes, Keep It Simple We can't make real-time analytical decisions because we don't have the right tool or interface that provide actionable information, without those tool we can't tame the data deluge and drive profit, big data is amazing, that's true only if we have the tool to handle it.
Keith.Grinsted,
User Rank: Petabyte Pathfinder 12/29/2012 | 6:49:53 AM
Re: Yes, Keep It Simple Unfortunately it is one thing using data to identify a problem - it is another to then rectify that issue.
We have known for years in UK that proprietary drugs cost us £millions and successive governments have tried to do something about it but it is the old pyramid again.
At the top is the single piece of data that says we are wasting £££s.
Then there are the government depts with programmes sponsored by the pharmacy industry.
Then the managers / administrators.
Then the doctors who are being courted by the pharmaceutical companies to prescribe their drugs.
And Doctors' Surgeries now have the final say as they are run as independent trusts, so are they going to listen to the data or the powerful pharmaceutical industry?
Saul Sherry,
User Rank: Blogger 12/28/2012 | 5:34:26 AM
Re: Yes, Keep It Simple Where to find those humans though @netcrawl? Francine's input here is great as it seems her specific skillset was brought in to tackle a specific NHS problem... imagine what could be done for healthcare throughout the world with a similar approach.
Anna Young,
User Rank: Exabyte Executive 12/26/2012 | 8:33:43 AM
Stubborn Privacy Issue There's always a snag in this type of analysis and that's the unknown reaction of the people who are the sources of Big Data. I enjoyed your analysis but found something missing. Yes, "Big Data Can Save Millions - in Surprisingly Simple Ways" as you made the case so carefully. However, what does the public think of the usage of Big Data and will there be an uproar if they weren't consulted? If we don't address privacy concern in every analysis, we may find that the savings from Big Data may turn into huge losses when the public fights back.
netcrawl,
User Rank: Exabyte Executive 12/26/2012 | 3:35:05 AM
Re: Yes, Keep It Simple We need to use some business analytics tools, data comes from everywhere, they need to captured, stored, analyzed and processed, before we can use nad harness its power. But the result of this big data analysis could mean something big for company, of course there's always some human intervention. data can't just stand alone without human.
To save this item to your list of favorite Big Data Republic content so you can find it later in your Profile page, click the "Save It" button next to the item.
If you found this interesting or useful, please use the links to the services below to share it with other readers. You will need a free account with each service to share an item via that service.
Tweet This
