Montreal Section

 View Only

ASQ — Problem Solving Tools Resources: Evaluation And Decision-making Tools (5)

By Jean-Pierre Amiel posted 02/23/23 04:36 PM



This last article introduces two evaluation and decision-making tools to use when you want to narrow a group of choices to the best one, or when you want to evaluate how well you’ve done something, including project results. The Decision matrix and Multivoting tools are briefly described below and on ASQ's Quality Resources pages.

This series of articles presents techniques to identify a "problem", identify its most likely cause (root cause) and then resolve the situation. These articles are also extracted from ASQ's Quality Resources library which showcases information related to the quality world's Body of knowledge (BOK) and compiled from more than 75 years’ worth of content for all experience levels. It is only meant to be a starting point.

Decision Matrix

The decision matrix, also called Pugh matrix, decision grid, selection matrix or grid, problem matrix, problem selection matrix, opportunity analysis, solution matrix, criteria rating form and criteria-based matrix is used to evaluate and prioritize a list of options. The team first establishes a list of "weighted" criteria and then evaluates each option against those criteria.

When to Use:

To expand the horizon of a team searching for answers and to develop a better and more detailed understanding of a problem or solution which will be helpful in the root cause analysis process.

  • When a list of options must be narrowed to one choice;
  • When the decision must be made on the basis of several criteria;
  • After a list of options has been reduced to a manageable number by list reduction.
Typical situations are: when one improvement opportunity or problem must be selected to work on, only one solution or problem-solving approach can be implemented, or when only one new product can be developed.

How to Use:

    1. Brainstorm the evaluation criteria appropriate to the situation.
    2. Discuss and refine the list of criteria.
    3. Reduce the list of criteria to those that the team believes are most important.Assign a relative weight to each criterion, based on how important that criterion is to the situation.
    4. Draw a matrix and write the criteria and their weights as labels along one edge and the list of options along the other edge (see the example of a customer service team evaluating the overall problem of "long wait time" to tackle first). Typically, the group with fewer items occupies the vertical edge.
    5. Evaluate each choice against the criteria.
    6. Multiply each option’s rating by the weight. Add the points for each option. The option with the highest score will not necessarily be the one to choose, but the relative scores can generate meaningful discussion and lead toward team consensus.


    A customer service team at a restaurant wanted to decide which aspect of an overall "long wait time" problem should be addressed first. Problems were identified and then the Criteria to address — wording the criterion was important so as to ensure that a high rating on each defines a state that would help best describe the problem. For Criteria, the team chose and used a rating scale for High (3), Medium (2) and Low (1).

    Decision Matrix Example

    In the example, the Problem "Customers wait for food" was rated Medium (2) because the restaurant ambiance is nice, it would not be easy to solve (Low - ease = 1) as it involves both waiters and kitchen staff, the effect on other systems is Medium (2), because waiters have to make several trips to the kitchen and it will take a while to solve (Low speed = 1), as the kitchen is cramped and inflexible.
    Each rating was then multiplied by the weight for that criterion. In the example, "Customer pain (5)", the problem "Customers wait for host" with high (3) received a total score of 15. All the scores are added across the rows to obtain a total for each problem. In summary, the "Customers wait for host" obtained the highest score of 28 and as the next highest score was 18, the Host problem seemed to be the first the team should be addressed.


    When to Use:
    • After brainstorming or some other expansion tool has been used to generate a long list of possibilities;
    • When the list must be narrowed down;
    • When the decision must be made by group judgment.

    How to Use:
    Materials needed: Flipchart or whiteboard, marking pens, 5-10 slips of paper for each individual, and pen or pencil for each individual.

    1. Display the list of options. Combine duplicate items. Affinity diagrams can be useful to organize many ideas, eliminate duplication and overlap.
    2. Number (or letter) all items.
    3. Decide how many items must be on the final list and how many choices each member will vote for — usually five. The longer the list, the more votes will be allowed, up to 10.
    4. Each member selects his five choices, ranks them in order of priority (first choice ranks highest). Each choice is written on a separate paper, with the ranking underlined in the lower right corner.
    5. Tally the votes and record on a flipchart or whiteboard. For each item, rankings are totalled next to the individual rankings.
    6. If a decision is clear, stop. Otherwise, continue with a brief discussion of the vote — look at dramatic differences and possible errors from misunderstandings about an item. The discussion is not to pressure vote changes.
    7. Repeat the voting process in steps 4 and 5. If greater decision-making accuracy is required, voting may be done by weighting the relative importance of each choice on a scale of 1 to 10 (most important).
    8. Draw a box at the top of a piece of flip chart paper and write the problem or solution to be explored.
    9. Below the statement box draw five lines in descending order.
    10. It may take less or more than five times to reach the root cause or solution.


    A team had to develop a list of key customers to interview. A first brainstorm identified a list of possible names for representation in three different departments, the list was divided into three groups. Within each group, multivoting was used to identify four first-choice interviewees. The votes and tally for one department are shown below.

    Multivoting Example

    Fifteen of those names were in that department. Each member had five votes — five points for the top choice, four to the second and so on down to one point for the fifth. Although several choices emerge as agreed favorites, significant differences are indicated by the number of choices that have both high and low rankings. The team will then discuss the options to ensure that everyone has the same information, and then vote again.

    Article adapted from: The Quality Toolbox, Second Edition, ASQ Quality Press.

    § This News post was adapted by J.P. Amiel, ASQ Senior, CQA ret., Web committee Chair, from content at ASQ's Quality Resources pages, which is excerpted and adapted from various publications in the ASQ Quality Press.
    ASQ References:

    Quality tools  ► Quality tools A to Z  ► Download quality templates and Excel tools  ► Quality tools resources  ► Quality glossary