What do you do when the unexpected happens? Go to your event knowledge base of past historical unexpected happenings… of course.
“Hang on!” I hear you say, “We don’t have one.”
I would ask, “Why not?”
Is it because you have not experienced the pain of forecasting company outcomes and getting it horribly wrong? I have fond memories of walking out of performance result meetings with a number of virtual daggers in my back because my forecasters did not see the tumultuous occurrence of an event that generated “What the…”
This is where unformatted big data comes into its own. If you need to establish a knowledge base of what on earth happened to drive unforeseen happenings in the past (as we had to do) then your unstructured big data can help.
Performing forensics on a crash
As a hypothetical example, on the 29th of January the web traffic on a company site increased to a point the website crashed. Your big data database contains the unformatted information that will support the event knowledge base to substantiate why this crash occurred.
The criteria for the event (the date and keywords like "sales campaign" or "special offers") gets passed to the big data database, which then extracts all the relevant knowledge, which is then stored in the event knowledge base for future reference. The next big decision you have to make is how to estimate the likelihood of a similar event occurring in the future, when it will occur, and whether it will have a similar impact.
In this example, the event knowledge base is used to assist with forecasting future web traffic and for identifying similar occurrences in the future. On the 7th of February a similar outcome occurred and was much more sustained. This information is once again passed via the big data information sources to augment the information asset that will assist with determining the relevant drivers for forecasting future website traffic.
This is a very brief explanation on how to make this work for you. Post a comment if you want to discuss things in additional detail.
Re: What the....! That's understandable and I am sure you do a great job on it Terry - but it could be a potential political timebomb in teams. I guess that's one reason to push the 'people skills' as essential in the Data Scientists, so they can managed these situations with kid gloves?
User Rank: Exabyte Executive 2/26/2013 | 10:58:46 AM
Re: What the....! Without historic data for the subject we study, we might get some benchmark data from other places as comparables. Otherwise, as you mentioned, it becomes pure guess work. It is a good suggestion to utilise this approach at the top predictor level or as inputs for determining, modelling and managing the drivers of your factor models.
Re: What the....! Hi and thanks for your comments. Yes you have pretty much got the idea.
The whole aim of building a knowledge event database is to reduce the amount of guess work. You cannot remove it completely however. You will be able to undertake quite a bit of scientific rigour around the analysis as the knowledge base grows.
In the past, as things grew we captured professional judgement adjustments to the forecasts in the knowledge database as well which also helped to develop a scorecard for internal users so they could assess who was making useful changes and who was not.