Executives reading the business press cannot have failed to notice the stories of companies enjoying the great benefits offered by big data technologies. This may lead them to anticipate that the implementation of such systems at their firms will be quite straightforward and will quickly boost their bottom lines.
However, from the CIO’s perspective the reality is very different. The CIO understands that the emergence of a major new technology paradigm is hugely disruptive for the whole technology organization and its practitioners.
The true impact of technology
While a new technology can be acquired like any piece of capital equipment, the true impact of its influence is in the human capital dimension of the organization. Technologists are used to change -- the changes are usually manageable, and they learn to live with them. In the 70s and 80s core skills were FORTRAN and COBOL; in the 90s C and Java took hold, to be followed by Internet-based skills and most recently the need for mobile platform developer skill sets.
How bright is your big data IQ?
Big data is about to change all this once again and, appropriately enough, in a big way. The reason for this is that the new required skill sets are not incremental: They are not reminiscent of prior generations of programming, and the applicability of common development methodologies is therefore also limited.
Your big data IQ
A survey I carried out of 166 companies, running projects self-described as "big data" in nature, revealed a set of 30 software systems and associated tool-based technologies being deployed. In light of that, an interesting big data IQ test may be to ask your firm's human resources IT/MIS team how many they can name. OK, you may not want to be quite as blunt as this, but a discussion of the technologies at the water cooler may be quite revealing.
A count of five should be the minimal passing grade. The trade press and web news reports usually discuss the big names: Hadoop, Pig, Amazon S3, along with Scala and Python. A score of five to ten demonstrates someone is starting to pay more detailed attention. Again the knowledge gained here is usually based around the Apache tool set: Zookeeper, Thrift, Chukwa, Accumulo, and Hive. Beyond ten and your HR folks have been doing their homework and are already on track to furnish the talent your organization is going to need for its big data initiatives. For those firms not yet at this level, there are three steps that can be taken.
Three steps to increase your IQ
First, the CIO needs to create a team with the HR group for assessing the current state of the technical group's own skill set. This is a complex problem. With many Baby Boomers aging out of the organization, legacy skill sets are also disappearing, unfortunately leaving a secondary skills gap around the maintenance of the old code. Problematically, Gen Y (born approx. 1977-1994), a group usually touted as the answer to corporate technology issues, due to the fact they grew up with PCs and tech in school and college, will typically lack the very programming skills they now need, having graduated college just before these technologies arrived in the classroom.
Second, assess what it is that your big data initiative is intended to achieve. As in previous generations of software, the standard is still developing, and so there is typically more than one way to achieve many desired goals. Tool and vendor assessment can then be made with the objective of "connecting the dots" among the plethora of technology options available in relation to the company’s potential project goals.
Third, as in any technology hiring practice the goal is to hire adaptive, innovative talent intrinsically motivated to grow and change as the area changes. Late Gen Y and early Gen Z (born approx. 1995-2012) lucky enough to be in colleges with cutting-edge computer science and IT will be in high demand. Leading HR departments will be very active right now in offering internships and recruiting these high-potential students. This human capital resource will be made all the harder to obtain as entrepreneurial graduate students spin out to start their own Silicon Valley startups.
A final thought is that if the HR person you were chatting to at the water cooler can name more than 25 big data technologies she may be secretly thinking of leaving your organization and starting a big data headhunting firm. Maybe its time to offer her a raise and keep her on board.
Chicken Egg Problem Do you find that companies are challenged to break into big data? I am wondering if some companies just hire big data people because it is a hype without having a strategy. Consequently, losing the people after a short while or burning capital for little return. Conversely, some advanced IT departments may have the ideas ready but not get through to HR to get the right people?
User Rank: Petabyte Pathfinder 2/3/2013 | 12:39:37 PM
Re: Interesting how big data can help HR Many other departments are already using and embracing technology to drive the workforce and move forward in their own respective teams. It was only a matter of time before HR followed suit.
User Rank: Exabyte Executive 2/2/2013 | 7:12:18 AM
Re: Interesting how big data can help HR I saw an article where Lowe’s has been studying the relationship between employee engagement, compensation, managerial effectiveness, and store sales for years. Their model is sophisticated enough now that when a store is underperforming the HR Analytics team can immediately provide insights to the management team to help them implement corrective action. I think that is the value of big data to HR.
User Rank: Exabyte Executive 1/31/2013 | 10:13:31 AM
Re: Interesting how big data can help HR I guess with more development and the maturity of some of the technologies, standardization can help. Not sure why HR department has the need for big data though? How big the data can be?
User Rank: Bit Player 1/31/2013 | 8:05:50 AM
Interesting how big data can help HR Interesting article,the next gen workplace will require HR to embrace technology to drive the workforce , read an interesting whitepaper on points discussed in this article readers might also find interesting @ http://bit.ly/WMC5kl
Re: Bringing HR Up to Speed Yes, its curious that the trainning of HR may have to be organized by HR for themselves, with the IT group. The question of self awareness comes to mind, does the HR group know what it does not know, and then are they willing to admit it, finally are they willing to do something about it.
Re: Big Data IQ great point, if we tech folks cant speak business then this is never going to work as wel will be pointing fingers at each other as to whose responsibility it is to 'understand' the technology and the organizational needs. The IT group needs to get the organization up to speed and as was disussed in another comment we need to break down walls on the BD topic and the needs of the business for technology as a whole.