Quality Review - TISTATechnologies/caseflow GitHub Wiki

Background

AMA appeals will need to be routed to quality review before dispatching appeals in order to monitor decision-writing quality and consistency.

• They have a quality review team queue, with individuals on the QR team having their own queues
• QR admin will manually assign cases selected for Quality Review to individuals on the team
• Quality Review has the ability to send a case:
  • Back to their judge (if there are required corrections to the decision)
  • Straight to BVA dispatch (if there are no issues with the decision and it can go straight out)
• Cases are selected for QR using the below logic

QR selection logic

• Github issue with history
• QR_SELECTION_PROBABILITY = 0.04
  • Probability that a given case should be selected for Quality Review (p)
  • We want the number of cases selected in a given month (X) to equal or exceed the target (x), as any excess selected cases will be ignored. The target during the pilot is 34 cases.
  • We want the probability that X is less than x (P(X<x)) to be less than 0.000833, an event that would happen once per century.
  • In December 2017, the Board wrote 5,549 decisions. Taking 1/4 of that figure, reflecting that Caseflow is piloted with 1/4 of the Board's judges, we have a number of trials (n) equal to 1,387.
  • When n = 1,387, x = 34, and p = 0.04, we find that P(X<x) = 0.000464, which is less than our maximum probability 0.000833.
  • This probability should be 0.032 when Caseflow expands to include all judges.
  • When n = 5,549, x = 136, and p = 0.032, we find that P(X<x) = 0.00043, which is less than our maximum probability 0.000833.
  • An easy way to avoid doing the math is to use this calculator: https://stattrek.com/online-calculator/binomial.aspx