Achievements discussion - AtlasOfLivingAustralia/volunteer-portal Wiki

##Awarding achievements

Each time a Task or Field is updated, the task is dumped into a request parameter. Once the request completes a grails filter is run, which collects all the task ids that were updated in the request and indexes them (synchronously). Once the index is completed, the current user and all transcribers and validators from the task(s) are collected and all achievements that have not yet been awarded are evaluated for each user.

TODO gather metrics to determine performance impact of sync indexing / evaluation.

##Awarded achievements

The next time the user loads a page after being awarded an achievement they will be greeted with a modal dialogue to inform them:

image

##The Dashboard

All awarded achievements can be viewed on the dashboard. Non awarded achievements are also present but are greyed out (the following is a terrible example because the badge itself is silver/grey):

image

##Creating an achievement:

###ElasticSearch queries, aggregations

Are input using ElasticSearch JSON formats, eg:

{
    "term" : { "fullyTranscribedBy" : "8737" }
}

String substitution is performed on the queries, insert a substitution using ${varName}.

The following variables are available for string subsitution:

eg, to genericise the above, replace the 8737 with ${userId} like so:

{
    "term" : { "fullyTranscribedBy" : "${userId}" }
}

Aggregations are similarly added:

{
    "projectids" : {
      "terms" : { "field" : "projectid", "size": 7 }
    }
}

This aggregation query will:

###Evaluating ES queries:

A non-aggregation query is evaluated against the count field on the achievement.

If the number of documents returned by the query is greater than or equal to the number given in the count field then the achievement will be granted to the user being evaluated.

EG, simple "You have transcribed 100 tasks" type achievements can be done this way. Additional filters can be added such as a "Holiday Hero" badge for someone who transcribed 50 tasks in December, for example.

###Evaluating ES Aggregation queries:

Aggregation queries, the equivalent of an SQL GROUP BY clause, don't have a corresponding SQL HAVING clause. So ensuring that the required number of buckets or number of documents in a bucket is above a threshold can't be done by a query alone.

At the moment, the means of evaluating an aggregation is via a groovy script.

The script is granted access to a org.elasticsearch.action.search.SearchResponse typed variable with the name searchResponse, the id of the task that triggered this evaluation under taskId (may be null and probably to be removed) and the user being evaluated under user. The script should return a boolean, with true indicating the aggregation matches the conditions for the achievement, false otherwise.

#####Aggregation evaluation examples

Assuming we have a search query of:

{
    "term" : { "fullyTranscribedBy" : "${userId}" }
}

And an aggregation of:

{
    "projectids" : {
      "terms" : { "field" : "projectid", "size": 7 }
    }
}

to ensure that a user has transcribed tasks in at least 7 projects, you can use:

Or to ensure that the user has transcribed 10 tasks in 7 different projects you could use:

def projectIds = searchResponse.aggregations.get('projectids')
projectIds.buckets.size() >= 7 && projectIds.buckets*.docCount.min() >= 10

NOTE The count field is ignored in the Aggregation at the moment, probably needs to be hidden on the edit UI.

TODO Can the groovy be removed and replaced with just bucket size, min docCount fields?

###Groovy type queries:

Use a groovy script, which is granted access to:

Typically using this sort of query would be a last resort.

###ElasticSearch index fields:

Task Documents in the ElasticSearch index have the following mapping:

{
  "mappings": {
    "task": {
      "dynamic_templates": [
      ],
      "_all": {
        "enabled": true,
        "store": "yes"
      },
      "properties": {
        "id" : {"type" : "long"},
        "projectId" : {"type" : "long"},
        "externalIdentifier" : {"type" : "string", "index": "not_analyzed" },
        "externalUrl" : {"type" : "string", "index": "not_analyzed"},
        "fullyTranscribedBy" : {"type" : "string", "index": "not_analyzed"},
        "dateFullyTranscribed" : {"type" : "date"},
        "fullyValidatedBy" : {"type" : "string", "index": "not_analyzed"},
        "dateFullyValidated" : {"type" : "date"},
        "isValid" : {"type" : "boolean"},
        "created" : {"type" : "date"},
        "lastViewed" : {"type" : "date"},
        "lastViewedBy" : {"type" : "string", "index": "not_analyzed"},
        "fields" : {
          "type" : "nested",
          "include_in_parent": true,
          "properties": {
            "fieldid" : {"type": "long" },
            "name"  : { "type": "string", "index": "not_analyzed" },
            "recordIdx" : {"type": "integer" },
            "value"  : {
              "type": "string",
              "index": "not_analyzed",
              "fields": {
                "analyzed": {
                  "type": "string",
                  "index": "analyzed"
                }
              }
            },
            "transcribedByUserId": {"type": "string", "index": "not_analyzed" },
            "validatedByUserId": {"type": "string", "index": "not_analyzed" },
            "updated" : {"type" : "date"},
            "created" : {"type" : "date"}
          }
        },
        "project" : {
          "type" : "object",
          "properties" : {
            "name" : {
              "type": "string",
              "index": "not_analyzed",
              "fields": {
                "analyzed": {
                  "type": "string",
                  "index": "analyzed"
                }
              }
            },
            "projectType" : { "type" : "string", "index": "not_analyzed" },
            "institution" : {
              "type": "string",
              "index": "not_analyzed",
              "fields": {
                "analyzed": {
                  "type": "string",
                  "index": "analyzed"
                }
              }
            },
            "institutionCollectoryId": { "type" : "string", "index": "not_analyzed" },
            "harvestableByAla": { "type" : "boolean" },
            "mapRef": { "type": "geo_point", "lat_lon": true },
            "templateName" : { "type" : "string", "index": "not_analyzed"},
            "templateViewName" : { "type" : "string", "index": "not_analyzed"},
          }
        }
      }
    }
  }
}

###Testing achievments

The achievement admin page has tabs for testing a user's eligibility and for awarding achievements to a user regardless of eligibility. These tools can be used to test an achievement behaves correctly prior to flicking the "on" switch.

####Edit page, test tab

image

The testing tab defaults to the current user. You may enter an alternative user in the type ahead field. The user will then be evaluated and a simple true/false will be displayed to determine if the user would be awarded the achievement.

####Edit page, award tab

image

The awards tab lists all users currently awarded the achievement. It will also present whether they are currently eligible for the achievement. The red "X" button can remove an individual award. The large buttons at the top can be used to evaluate and award the achievement to all users if they're eligible or remove all awards of the achievement.

NOTE: The Award All Eligible button can put the server under considerable load.

Removing an achievement award is noted in the log, including the user, the achievement and the date awarded in case it needs to be re-instated manually.

TODO Add confirmation dialogues for unaward actions.

An award can be granted manually even if the user is not eligible for it using the "Grant achievement" box at the bottom of the page.

####Admin Tools page

image

Finally, the admin tools page allows the submission of arbitrary queries and aggregations to be run against the current index. It returns a JSON document that represents the entire SearchResults object.

####ElasticSearch queries should be changed to filters?

Currently ES Queries are being used. Perhaps it should be filters only:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html

"Filters are very handy since they perform an order of magnitude better than plain queries since no scoring is performed and they are automatically cached."

Investigate whether filters can be used directly in the setQuery call or whether they require their own call to the ES client.

May be some issues as filter syntax looks weirder and may present issues with extracting results:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html

vs

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-query.html