Scoring API - quandis/qbo3-Documentation GitHub Wiki

The Scoring module is designed to handle 'number crunching'. It is (of course) interface-based, which means plug-ins can be designed to handle number crunching, like Excel, Google Spreadsheets, SPSS, LexisNexis, or custom third party web services (such as bank NPV calculation engines).

Examples of scores include:

• Performing an NPV calculation on a loan
• Analyzing a broker's performance on a portfolio of BPOs
• Analyzing a user's performance across all smart worklists
• Determining a borrower's propensity to succeed in a loan modification given their credit report trade line detail

At it's core, calling Score/Calculate performs the following:

• Queries the QBO database for data (by default, Parent/Summary)
• Passes the data to a scoring engine (plug-in) via the IScore interface
• Accepts resulting data points from the scoring engine, storing each one in a ScoreItem row

The most popular plugin for the Scoring module is the Excel Engine. It allows organizations to harness the power of their intellectual property already in Excel, without the need for humans to touch Excel on a daily basis.

Auditing is a subset of scoring, typically performed manually by a human. The most common use case is to score the quality of work performed by a user after completing a Custom Form. For example:

• 10% of Foreclosure Referral forms may be audited for regulatory compliance
• 100% of BPO forms may be audited for quality control
• 25% of the work performed by a new employee may be audited by a supervisor

The steps required to set up an audit include:

• Create a custom form to serve as a task for users to complete (e.g. Foreclosure Referral)
  • This will automatically create a Smart Worklist Template matching the name of the form (e.g. Foreclosure Referral)
• From Design > Smart Worklists, search for and click on the worklist to be audited
• From Smart Worklist Template Summary page:
  • click on the Audit panel
  • leave the Matrix dropdown on the 'Create a Matrix' option, and
  • Click the Save button
  • this will create a new Matrix, with some pre-populated dimensions (see below)
• From Design > Scores, create a Score Template containing the data points you wish to collect during your audit
• From Worklist > Manage Worklists, sort by Template, select the templates to be audited, and choose Options > Audit
  • this will scan every completed worklist item, and determine if the items matches a row in the Audit Matrix
  • for each row of the Audit Matrix, a subset of matching items will be added to an auditing worklist

Based on the setup above, we have:

• A Task (ImportFormTemplate) called 'Foreclosure Referral'
• Two SmartWorklistTemplate rows: 'Foreclosure Referral', and 'Audit: Foreclosure Referral'
• A Matrix called 'Audit: Foreclosure Referral'
• A ScoreTemplate called 'Audit: Foreclosure Referral'

The Audit Matrix is very powerful: it can group worklist items by an number of data points (MatrixDimensions), and audit a subset of that group. The most import output (Measure) of the Audit Matrix is a Template: this is the name of the 'auditing' SmartWorklistTemplate to use. For example, if you are auditing a 'Foreclosure Referral' worklist with 100 items in it, the system will create a new SmartWorklistTemplate called 'Audit: Foreclosure Referral Worklist'.

When choosing how many worklist items to audit, you may define a percentage, minimum and maximum. For example, assume you have 100 items in a worklist:

• Setting Percent to 20 will cause 20% * 100 = 20 items to be audited
• Setting Minimum to 5 would make no difference in this case
• Setting Minimum to 25 would cause 25 items to be audited (since 25 > 20% * 100 = 20)
• Setting Maximum to 50 would make no difference in this case
• Setting Maximum to 15 would cause 15 items to be audited (since 15 < 20% * 100 = 20)
• Setting Minimum to 20 and Maximum to 10 would cause 10 items (but this is a silly setting, so don't confuse yourself)

The audit worklist will route the auditor to the same URL that a user used to edit the data being audited, but will include an &Audit={Audit ScoreID} parameter. The target URL is responsible for implementing the Audit behavior, as follows:

<div class="btn-group" data-behavior="Audit" data-audit-options="{'data': {'Object': '<some object>', 'ObjectID': '<some object id>'} }">
   ...
</div>

This behavior will, upon detection of an Audit parameter in the URL, render the Audit UI.

The audit score determines the questions an auditor will see when navigating to a page to be audited. Each ScoreTemplateItem defined in the score template will be rendered as a question to be answered by an auditor.

A Scoring dashboard displays all scores performed by template, including roll-up statistics on the values entered by auditors for each score.

An audit is launched by calling SmartWorklist/Audit?ID={some SmartWorklistIDs}. This calls the SmartWorklist/Audit method, which does several things:

• executes the SmartWorklist/AuditCheck statement, which returns all SmartWorklistMembers that have been completed, but have not already been marked for an audit
• for each SmartWorklistMember, the SmartWorklistMember/Audit methods is called, which:
  • calls Matrix/Lookup for the SmartWorklistMember to determine if the member is eligible to be audited.
  • ff the matrix returns a row, the SmartWorklistMember.AuditMeasureID will be used to store which matrix 'row' was returned.
  • if the Matrix lookup returns no rows, SmartWorklistMember.AuditMeasureID is set to 0, indicating that the member is not eligible for auditing.

At this point, all completed SmartWorklistMembers in the SmartWorklist we are auditing will have SmartWorklistMember.AuditMeasureID set to 0 (not eligible) or some MatrixMeasureID (indicating the 'type' of audit to be performed). Keep in mind that audits are usually performed on a subset of work completed; we next must calculate that subset.

The SmartWorklist/Audit method will next call the SmartWorklist/Audit statement, which calculates which of the eligible SmartWorklistMembers to actually audit. The Audit statement is pretty slick, doing the following:

• for each distinct MatrixMeasure, count how many SmartWorklistMembers are associated with the 'row'
• ensure a SmartWorklistTemplate exists matching the Matrix row's Template` column
  • the MemberUrl for this template should be the same as the SmartWorklistTemplate being audited, with &Audit={Template}` added to the end
• ensure a SmartWorklist exists matching the Matrix row's Template column
• inserts enough SmartWorklistMembers into the audit SmartWorklist to ensure we meet the Percentage, Minimum and Maximum values specified by the Audit Matrix

If the auditing is not working as expected, key points to check include:

• Does the SmartWorklistTemplate being audited have an Audit Matrix defined? If not, create one in accordance with the instructions above
• Does the Audit Matrix have a Template output (measure)? This template value needs to match the name of both the SmartWorklistTemplate that tracks the audits, as well as the ScoreTemplate used to enter the auditing data.
  • if there is no SmartWorklistTemplate matching the Audit Matrix's Template column, that's okay; one will be created on the fly by the SmartWorklist/Audit statement
  • if there is no ScoreTemplate matching the Audit Matrix's Template column name, that's not okay; we cannot predict what data points are to be audit; you need to create one!
  • if you want to change the name of the ScoreTemplate being used, you can: you must:
    • Ensure a SmartWorklistTemplate exists matching the Audit Matrix Template name, and
    • Set that SmartWorklistTemplate.MemberUrl to include &Audit={your custom Score Template name here}