In the software project that I work, we have a requirement to do a Root Cause Analysis for the bugs that are released to the production (Residual bugs) which needs to be done in order to fulfill the ISO quality standards that our company is certified with, as well as to improve our quality assurance process in the project.
When it comes to root cause analysis of the issues, we can consider them in 2 aspects.
1. Bugs found by the development team during their testing cycles
2. Bugs found and reported by the customers after product is released to the production.
Root causes analysis of the bugs in these 2 types would be different.
Root causes in the 1st, would reveal the issues related to requirement, design and coding phases of the development life cycle. Root causes analysis of the latter will reveal the issues regarding the quality assurance process that a team has adopted.
Both types of root cause analysis is very important in order to improve the overall quality of the product being developed as well as improve skills of the human resources involved
There can be 2 ways of doing a root cause analysis
1. Analyse root cause for each bug individually, getting input from each stakeholder (may be a coder and tester) and write it down in the bug itself, and discuss the leanings at each review meeting ex: if you are in agile scrum environment, you can take them to the retrospective meeting and do a 'learnings' session, which we can consider a more active process
2. Do a 'Trend analysis' instead of analyzing each issue individually. You look for a pattern , which areas need attention more , and then take necessary actions for them which we can cosider more of a passive process.
You can decide on either way depending on the requirement that is asked by your project. In my case, I adopted the second option - that is to go for a trend analysis which was more appropriate in our case.
This is something I tried for the first time, which goes like following.
Firstly, I decided that I would start this for the last major release we did few months ago , instead of going back to more previous releases, because the data I will have to go through will be in manageable level and be more accurate.
We use Team Foundation Server (TFS) as our main data management system and I created a search query to get the relevant data for the last release. I filtered out the severity medium and low issues and focused on severity high issues
Secondly I designed a set of root causes, that I thought can be use againt the issues. Following is the list.
Thirdly, I imported the issues list to an excel sheet and try to guess root cause for each issues from the list i created about.
Fourthly, I tried to plot the above information in to a bar chart. it looks like following.
Based on the graph, I could see some area that I can improve on. So I thought this is a good learning experience and worth giving it a try next time as well :)
When it comes to root cause analysis of the issues, we can consider them in 2 aspects.
1. Bugs found by the development team during their testing cycles
2. Bugs found and reported by the customers after product is released to the production.
Root causes analysis of the bugs in these 2 types would be different.
Root causes in the 1st, would reveal the issues related to requirement, design and coding phases of the development life cycle. Root causes analysis of the latter will reveal the issues regarding the quality assurance process that a team has adopted.
Both types of root cause analysis is very important in order to improve the overall quality of the product being developed as well as improve skills of the human resources involved
There can be 2 ways of doing a root cause analysis
1. Analyse root cause for each bug individually, getting input from each stakeholder (may be a coder and tester) and write it down in the bug itself, and discuss the leanings at each review meeting ex: if you are in agile scrum environment, you can take them to the retrospective meeting and do a 'learnings' session, which we can consider a more active process
2. Do a 'Trend analysis' instead of analyzing each issue individually. You look for a pattern , which areas need attention more , and then take necessary actions for them which we can cosider more of a passive process.
You can decide on either way depending on the requirement that is asked by your project. In my case, I adopted the second option - that is to go for a trend analysis which was more appropriate in our case.
This is something I tried for the first time, which goes like following.
Firstly, I decided that I would start this for the last major release we did few months ago , instead of going back to more previous releases, because the data I will have to go through will be in manageable level and be more accurate.
We use Team Foundation Server (TFS) as our main data management system and I created a search query to get the relevant data for the last release. I filtered out the severity medium and low issues and focused on severity high issues
Secondly I designed a set of root causes, that I thought can be use againt the issues. Following is the list.
1. This area is already tested earlier so done a smoke test
2. Scenario not tested - due to lack of time
3. Scenario not tested - too many combinations to test so missed this
4. Scenario not identified - this was not identified as a test case at design phase
5. Dependencies not tested properly - effect to and from other systems - impact analysis not done
6. Lack of valid/invalid input data usage while testing
7. Not tested this particular scenario in particular environment
8. Edge case scenario - very unlikely to test
9. Requirement not understood correctly
10. Specification not studied properly
11. Scenario not covered during testing
12. Customer specific modification- not standard
13. Not existing feature- But customer expects to have
14. Not a bug- Customer mis-configuration
15. Known issue - May not be fixed either due to technical difficulty or not prioritised
16. Difficult to test due to non-existing resources/ skills /environments (ex: language expertise, lotus notes)
17. Bug already added to TFS but not prioritized for fix
Thirdly, I imported the issues list to an excel sheet and try to guess root cause for each issues from the list i created about.
Fourthly, I tried to plot the above information in to a bar chart. it looks like following.
Based on the graph, I could see some area that I can improve on. So I thought this is a good learning experience and worth giving it a try next time as well :)
No comments:
Post a Comment