Communicating IT Effectiveness, Part 2

This blog recently discussed IT investments for innovation and gave a framework for communicating and discussing those investments (from Poniatowski & Wischer, 2006). Clearly, the next question becomes what metrics exist to evaluate and communicate about IT investments? In an interesting article, Chabrow (2006) outlined twelve techniques for measuring IT effectiveness. We discuss them briefly in the following paragraphs, with the names and the real-life examples taken from Chabrow's (2006) article.

1) Missed Opportunities: American Power Conversion (APC) focuses on continuous customer service. They have designed and implemented an alert system that informs employees of failures in customer service. Furthermore, they measure success by decreases in missed opportunities to provide services, creating an ever-higher bar for excellent customer interactions.

2) Client Impact: Paul Heller, CIO of Vanguard Group, the mutual fund company, has created a metric consisting of a 3x3 grid to measure the intersection of client impact and operational impact. Each is ranked as low, medium or high impact, but the overall company benefit (or lack thereof) can be easily and consistently visualized. IT funding is allocated and tracked for low, medium and high client impact.

3) Self-Help: Pacific Blue Cross in Vancouver, British Columbia, categorizes IT expenditures as either innovation or support projects. Sometimes, those two intersect, as in the domain of self-help for filing claims. Their new e-claims system has been considered successful because dentists who use it are increasing and manual claims are decreasing, reducing operational overhead.

4) Staff Mix: Effectiveness of an IT project can also be assessed in terms of knowledge gained and hours spent, which can be characterized by the proportion of in-house and outsourced staffs. Though outsourced projects may be cheaper, with territory specific projects, knowledge of local needs, challenges, and obstacles are critical to long and short term project success. One company, EMC, strives for a 40% outsourced, 60% in-house staff to maximize knowledge and minimize costs.

5) Tied To Profit: Steve Phillips, CEO of Avnet, benchmarks IT spending against gross profit instead of revenue or expenses. IT spending is evaluated by if it increases revenue or decreases cost.

6) Perception Counts: A group at MIT's Sloan School of Management has developed a methodology for assessing gaps between an organization's IT security measures and employees' perception of security. Organizations then have a checklist for areas for operational, security improvement, particularly with regards to current practices.

7) Development Speed: Incorporating best practices during software development can help with the ever-important development speed of IT projects. University of Pittsburgh Medical Center has adopted Capability Maturity Model Integration and went from performing best practices 30% of time to currently seeing them 90% of the time.

8) Dropouts: This measures success of a new Web feature. Vanguard Group, the mutual fund company, conducts 80% of customer interactions online. Vanguard tracks the number of people who use a feature and when and where they stop using it--or if it is highly popular.

9) Sales Engine: IT centers must understand and engage customers to increase revenue. At Global Crossing, CIO Dan Wagner encourages IT people to attend customer meetings to understand business issues firsthand and hopefully deliver possible solutions to increase revenue. Knowing the customer changes development processes and can inspire new kinds of IT projects.

10) Innovation: IT projects can focus on creating new capabilities. While CIO Wagner at Global Crossing speaks of "strategic software development," CIO Randy Mott at Hewlett-Packard talks about IT "innovation." Innovative IT capability has to be actively pursued. Global Crossing insists that IT employees keep track of the hours spenton new projects, so as to assess time spent on new capabilities per IT project.

11) No Train, No Gain: CIO Wagner at Global Crossing emphasizes employee training. Each employee's training is tracked and the linkage to business requirements or objectives is essential. In other words, training for IT groups is actively pursued to create new business opportunities and growth.

12) The Ultimate Test: Vanguard Group, the mutual fund company, regularly surveys employees with two key questions: Will I recommend my area to a co-worker, and will I recommend my boss to a co-worker? Engagement with the job is thus assessed, and job switching is encouraged across business functionalities. Employee satisfaction takes on new meaning and can direct job placements.

References:
Chabrow, E. (2006) Techniques For Measuring IT's Effectiveness, Information Week, October 23, available at http://www.informationweek.com/story/showArticle.jhtml?articleID=193401033