Bug Severity Criteria

Richard Martin
Experienced Software Developer and Quality Engineering Manager

I tend to use a pretty standard definition.
Severity
---------------
1. Bug causes system crash or data loss.
2. Bug causes major functionality or other severe problems; product crashes in obscure cases.
3. Bug causes minor functionality problems, may affect "fit anf finish".
4. Bug contains typos, unclear wording or error messages in low visibility fields.

Priority
---------------
1. Must fix as soon as possible. Bug is blocking further progress in this area.
2. Should fix soon, before product release.
3. Fix if time; somewhat trivial. May be postponed.
Posted 1 day ago

Andrey Konushin
President at RSTQB
We use the following severity classification for some years already:

0. Customer Bug (Can be found only at customer's side)
- Any defect found by end user/customer in the release of the product.

1. Critical
- Inaccessible system functionality (user cannot perform the job), no workaround to perform the job;
- Application cannot run (buzzing);
- OS automatically closes the application;
- System security violation.
2. Major
- Inaccessible system functionality but workaround accessible;
- Unhandled exception.
3. Average
- System error message (handled exception).
4. Minor
- Non-business logic defect (defect of little significant that not leads to corruption or loss user data);
- Application GUI defect.

Defect priority:
1- Resolve immediately;
2- Give high attention;
3- Normal queue;
4- Low priority.
Posted 1 day ago
Martin Orlando
Senior QA Engineer
We used two fields for priority – when creating a bug, the Original priority field is filled in, and the Assigned priority is copied. After review by QA and/or Dev management – the Assigned priority field can be modified. When designing a bug writing course, I suggested that within the contents of the bug, you should describe why you are assigning it a certain priority.

Here are the priority levels:

Urgent – The complete inability to use the product at all on multiple platforms. The product cannot ship on any platform with any Urgent open issues. Urgent issues would result in a customer call to support. Unable to install or use the product at all fall into Urgent issues.

High – An issue that greatly affects the product, but is limited to specific platforms. System crashes, memory leaks or other major problems that would be found in normal, everyday activity. Cannot ship the product on a given platform with a High open issue. High issues would result in a customer call to support.

Medium – An issue that occurs at an extreme boundary condition and not likely to be found with normal use of the product. Medium bugs should not generate support calls, but may be seen as more annoying.

Low – Cosmetic bug that doesn’t affect end usability. Low bugs will not generate support calls.


Along with priority - we have a menu driven field called Blocking Status. If the issue is considered to be blocking someone from doing and/or completing their job this field is used to raise the fixing priority.

Some of the blocking
QA
Applications
Support
Customer

Along with that, there are some binary fields used:

Regression Status: - Setting to Yes means that this issue was found/fixed or working with a previous version and is now not working.

Repeatable: Some issues can be timing related – so if you ran the test several times, but were not able to repeat the result every time, set this to no…

There is also a field that Development used to assign the fix order priority – so if an engineer has 10 High bugs assigned, they can see which ones that management wanted them to fix first.
Posted 1 day ago

Brian Masinick
Project Manager at Fidelity Investments
The categorizations provided by Richard, Andrey, and Martin are quite common, based on what I have seen on a variety of projects across multiple businesses and industries. Priority, severity, reproducibility, and whether the defect has a work around or if it is a show stopper are among the factors that I have seen present in most defect tracking systems.

In my current job, most of the organizations use a common severity code system, where a Sev. 1 issue matches the Critical category mentioned by Andrey and the Urgent priority mentioned by Martin. Sev. 1 issues are tracked by senior management and they require an open network bridge to keep all interested parties informed of the current status and the steps being taken to resolve the issue.

Once there is at least a workaround, a Sev. 1 defect can be lowered to the next category, Sev. 2, which represents major defects, but where there does exist a work around.

Sev. 3 issues represent a significant defect, but there are work arounds available, potential alternatives, and there is no imminent business impact.

Sev. 4 issues represent minor issues that do not meet the functional requirements completely.

All of these categorizations contain elements that have been discussed in each of the previous set of comments.
Posted 1 day ago
Diane Miller
Sr. QA Analyst Consultant
For the most part I've worked under the same guidelines as Richard describes, and if the priority was anything over a 2, it never got fixed and we got to deal with it with every release.

In the bug review meetings, many times the developer would assign the priority but the QA lead would give the prospective of the QA team. The managers had the last word. And it would greatly depend on who the manager was and what the product was. If the product was likely to go in great numbers to 1 particular customer to where the customer would likely see the issue multiple times over their order, the priority would go up. If the product was likely to go out in one or two unit orders, the priority would go down.

Diane
Posted 1 day ago

Martin Orlando
Senior QA Engineer
It is rather funny - you will find a medium level bug that gets closed as being "as designed" with no actual fix involved - then the product goes to a Beta site and a Beta user complains about the same thing...and suddenly it is priority number 1 and needs to be fixed yesterday, with the question of "Why didn't you find that one?" comes up.
Posted 23 hours ago
Diane Miller
Sr. QA Analyst Consultant
Quote>>It is rather funny - you will find a medium level bug that gets closed as being "as designed" with no actual fix involved - then the product goes to a Beta site and a Beta user complains about the same thing...and suddenly it is priority number 1 and needs to be fixed yesterday, with the question of "Why didn't you find that one?" comes up

That's why even as designed or previously defined bugs get documented every time, to cover yourself from those type of questions.


Diane
Posted 21 hours ago
Fin Nugent
ICT Professional
Yshai - Are you looking for bug priority & severity criteria for a specific phase of testing or a "catch-all" set of definitions for all phases in the SDLC? I ask because what may be relevant for Unit Test may not necessarily suit for Business Acceptance Test, for example. Feel free to contact me to dicscuus your exact requirements.
Posted 20 hours ago
Richard Martin
Experienced Software Developer and Quality Engineering Manager
Diane raises a good point but I didn't think we were covering defect management here; just the classification based on severity & priority. There is a constant need to re-prioritize the bug backlog. At one company, when we scheduled minor releases we would set aside a fixed level of effort, say 20% of the release to address the backlog. If the backlog grew too high, we would adjust the % of effort up. The bugs scheduled in as part of the work-down of the backlog were selected in large part by the customer support and field service staff with input from QA.
Posted 19 hours ago
Shai Nadel
QA TEAM LEADER in Nice Systems
we use set two separate categories one is the Bug severity decided by the QA according to the following standards:
Low
* GUI ‘cosmetics’
* General minor issues
Medium
* Missing\wrong messages
* Not perfect functionality
* GUI friendliness
* GUI - Spelling errors
* Documentation errors
High
* Feature malfunction
* ‘Bad’ errors messages
* Fields validation
* Translation problems
* Missing Install Shied
* Missing documentation
* Wrong version
Show Stopper
* System\Component Crash
* Data loss
* Missing Feature
* Security intrusion
* Performance failures
* Customer determination
QA Stopper
* Stops Testing progress
================================
Next is Bug Priority which is determined according to the urgency of the fix in respect to the related costumer
Posted 9 hours ago
Fin Nugent
ICT Professional
Yshai:

I would generally expect that there would be some bug threshold-levels set in the exit criteria for was test phase.
For example, the Acceptance Test exit criteria would state that the following maximum level of bugs would be acceptable to exit the test phase:
Severity 1 - 0
Severity 2 - 5
Severity 3 - 10
Severity 4 - 25

The Severity definitions already provided by Richard in the first reply above would be those that I would generally use myself. The Priority applied to each bug should be reviewed on a regular basis between the Test Manager & the Development Manager. The PM being called upon to mediate in any disputed bug priorities