Quality Review - department-of-veterans-affairs/caseflow GitHub Wiki

Background

AMA appeals will need to be routed to quality review before dispatching appeals in order to monitor decision-writing quality and consistency.

  • They have a quality review team queue, with individuals on the QR team having their own queues
  • QR admin will manually assign cases selected for Quality Review to individuals on the team
  • Quality Review has the ability to send a case:
    • Back to their judge (if there are required corrections to the decision)
    • Straight to BVA dispatch (if there are no issues with the decision and it can go straight out)
  • Cases are selected for QR using the below logic

QR selection logic

  • Github issue with history
  • QR_SELECTION_PROBABILITY = 0.04
    • Probability that a given case should be selected for Quality Review (p)
    • We want the number of cases selected in a given month (X) to equal or exceed the target (x), as any excess selected cases will be ignored. The target during the pilot is 34 cases.
    • We want the probability that X is less than x (P(X<x)) to be less than 0.000833, an event that would happen once per century.
    • In December 2017, the Board wrote 5,549 decisions. Taking 1/4 of that figure, reflecting that Caseflow is piloted with 1/4 of the Board's judges, we have a number of trials (n) equal to 1,387.
    • When n = 1,387, x = 34, and p = 0.04, we find that P(X<x) = 0.000464, which is less than our maximum probability 0.000833.
    • This probability should be 0.032 when Caseflow expands to include all judges.
    • When n = 5,549, x = 136, and p = 0.032, we find that P(X<x) = 0.00043, which is less than our maximum probability 0.000833.
    • An easy way to avoid doing the math is to use this calculator: https://stattrek.com/online-calculator/binomial.aspx