There are strong motivating factors for more effective performance measurement practices in software development. Astute practices in this domain are lauded to improve efficiency and effectiveness. However previous studies have shown that measurement in software is intricate, complex and fraught with challenges. Consequently it is poorly managed in practice. Our research seeks to better understand performance management in a real world software development setting in order to identify the challenges and generate a roadmap for improvement. This paper presents findings from an inductive analysis of a radical measurement program in a global software organization. Our study investigates the level at which non-compliance with best practice can explain the company's disappointing results. We found that a narrow focus on projects, rather than on organizational goals, has seriously hindered its success. We also found that the rate of change in the organization as a whole was impinging on the effective implementation of its measurement program. An analysis of the results demonstrates just how challenging software measurement is. The findings provide an evaluation of best practice relative to the literature that is informed by real industry experience.