Wednesday, October 13, 2010

Project and Product Management & Metrics - A Simple Overview

Not everyone has to practice a rigorous analysis technique such as Earned Value Management (EVM) or Six Sigma, where each deliverable is monitored, evaluated, and analyzed for delivery against a pre-determined schedule using milestones. I've found that establishing a schedule in the beginning of the project, and expecting all identified tasks to be listed and monitored, is a losing battle. In video game development, an Agile technique which includes flexible timelines and task lists is most effective.

Agile management techniques, Scrum specifically, focus on a simple set of metrics:

  • velocity - the speed of the team to complete a set amount of task units in a set period of time (usually one iteration)

  • burn-down rate - the rate at which items to be done are completed

  • number of defects logged against the finished iteration

If you are looking at collecting product metrics, then you have many more options:

  • Google Analytics can be easily added to gather data about visitors to a web site

  • KISSMetrics will gather data about specific events on the website or in a web-based game

  • Kontagent offers social analytics for online games

Once you have data about your user base and their usage patterns, you may wish to dig deeper and perform user testing, customer surveys, focus groups, or even one-on-one interviews with customers. Armed with statistical data from an analytics tool (or two) and interview feedback, you may now compare what the customer says they want versus what the customer is actually doing to determine the key junctions where the product can be improved. For example, if a customer complains that a game lacks a certain type of gameplay, you might learn from metrics that what they really mean is that the gameplay is available, but is too difficult - this could be established by looking at session times for an aggregate set of users in that level or mode and seeing where people drop off.

If you combine the data gathered in creative ways, looking for dimensions not previously visible when only one source was used, you can gain a deep understanding of your products' quality, your customers' satisfaction, and the deficiencies which need to be addressed to improve both!

No comments: