One great way to stay on top of all of the tasks involved in a complex data project, and stay close to your business stakeholders, is to use a lightweight version of the Agile development methodology.
You may choose to do two-week sprints where the project tasks are divided into manageable chunks, that when all completed, add up to the delivery of the full project. Decompose your project into a series of user stories, which can be understood by both developers and non-technical business users. Then, if needed, break those stories down further into tasks and sub-tasks. Get as granular as you need to with your development team so there is never any doubt about who is responsible for what on a given day.
At the conclusion of your sprint, hold a demo with your key business stakeholder -- this is a great opportunity to highlight the work that has been done, and to show where you are on the path towards completing the project.
Think of this as the sync meeting but with some kind of data deliverable to check out and discuss. This can be a data model, an initial report or mock up, a sample data set, or something that highlights the work accomplished during the sprint.
To enhance the speed of delivery, it’s a good idea to co-locate your team in a “war room,” or other designated space where the team can spend their workdays. This improves communication and collaboration between team members, and breaks the project out of the business-as-usual routines.
Bringing in lunches for your team will give them more time to be productive and less time traveling or standing in line waiting for food. This may seem like a little thing, but it will add up over the course of the project schedule.
Pitfalls and how to avoid them
The success of the project will depend on a few variables, including:
Project resourcing relative to scope of deliverables
Motivation and level and skill set of development team
Trust within the team and trust in the PM leading the project
Ongoing communication with the business stakeholders
The degree that the scope of the project can be kept in check at the outset will help a whole lot as you go forward. Spend a little extra time upfront to prioritize and stack rank the feature set of the project. It may be that with a little more understanding and negotiation, some features that were considered critical are really “nice to have.”
Alternatively, if the scope cannot be reduced, the PM or the manager will need to aggressively fight to acquire the resources that are needed to support the requirements. This may include negotiating to borrow a superstar from another team, onboarding vendors, or merging teams that were previously working on different projects.
Hiring contractors can be a good way to quickly augment staff to support a time-sensitive project. However, it’s important to remember that onboarding vendors requires its own ramp up time, similar to any new employee -- this can include making sure they have an entry badge, to gaining permissions to various systems, to learning the data models and the business terminology.
That said, many of the leading IT consultancies have some really tremendous coding talent on staff -– folks who learn very quickly and are used to being expected to quickly deliver something of value.
You may want to embed these additional resources alongside the FTE team that is working on your project. This will likely work out better than bringing in a team of vendors to execute the project while leaving the FTEs out of it. The FTEs will feel like the work product of the vendor is lesser quality than what they themselves could have developed with their expertise and domain knowledge. This can lead to unhappy team members and a feeling of being left out of the really critical projects, as well as grumbling and negativity, which is really something to be avoided.
It’s best then to augment the staff as needed and build a dream team of the best talent available to you within your group, as well as some star talent from a vendor company.
User Rank: Exabyte Executive 2/5/2013 | 3:25:56 PM
Re: The team that sits together... I like the sprint model, and I've always been a proponent of buying talent as needed. I suppose it's fair to say there is no one size fits all model when it comes to a Big Data project, but how can we determine what is the best model to use? Any metrics out there to guide us?
Re: The team that sits together... Thank you, Saul! The environment at eBay is open floor plan w/o cubicles. The seating is laid out so cross-functional teams are co-located and can communicate as easily as possible. This is a great way to be set up as people feel really comfortable dropping by to chat so this fosters a good sense of team and camaraderie. The downside is, there's some ambient noise from lots of phone calls and conversations -- so a lot of folks wear headphones and listen to music if they need to really focus in and not be distracted. There are also small "focus" rooms that provide a quieter environment for eliminating distractions and noise. I think this type of approach to 'housing' is really well thought out and works great for data delivery teams.
Re: The team that sits together... Keep it lean and learn from your mistakes seems to be the only real solution - The variety in data keeps these sets super hard to define and create a plan to suit all.
User Rank: Exabyte Executive 2/6/2013 | 7:04:17 PM
Re: The team that sits together... Buying talent is crucial to making these projects successful. An organization should also look for contracting IT that have years of experience in big data or the area of the project to save ramp up time. Skills and team alignment are indeed crucial. Sometimes too much focus on budgeting and objectives can bring challenges causing time delays and more dollars being spent.