The QA metrics and specific project KPIs have a very important role in the project development lifecycle, as they are used to:
Asses the quality of the delivered services
Determine the effectiveness of the quality assurance strategy and activities
Determine the evolution of the team work outcome – productivity and quality improvements.
Continuous improvement, through analysis and act to fix and improve.
We use the following main categories of QA metrics and KPIs (grouped by functional or domain area), to identify the project areas that needs improvement, and also to guide the improvement activities:
Systems high-availability
- Modules/services availability
- Uptime numbers: average uptime and variance
- Continuity of services
System performance
- System components performance targets compliance (response time to various operations)
Change management
- Successful and failed changes
- Changes backlog timeline and variance
- Mean RFC (Request for Change) turnaround time
Monitoring and auditing
- System monitoring and auditing quality targets compliance
- Incidents & issues management
Incidents & issues management
- Number of incidents reported per module/service area
- Number of escalations
- Number of repeated Incidents
- Mean time between incidents are registered
- Mean time between high-priority incidents are registered
- Incidents severity/priority distribution (timeline)
- RCA (Root Cause Analysis) incidents distribution
- Average period for open issues
- Average time to fix an issue
- Average issues fix cost (man-hours)
- Reopened issues rate
- Ratio of accepted/rejected/postponed issues
- Incidents resolved remotely
Service validation and testing
- Failed Release Acceptance Tests
- Incidents caused by new releases
Service asset and configuration management
- Incidents due to inaccurate configuration management information
- Number of unauthorized changes
- Configuration management coverage
- Average effort for configuration management verifications
Backup and recovery
- Successful completion of systems backup activities
- Successful completion of systems restore activities
Delivery performance
- Operations/tasks completed on time
- Completeness
As a general rule, metrics are customized for each activity and each project. The number of metrics deployed must be kept as small as possible and they must complement each other.