Manual Testing of Distributions - department-of-veterans-affairs/caseflow GitHub Wiki

Environments

Local/Demo

This is naturally the easiest and most methodical environment to test distributions because we have complete control over the data in these environments. We can initially seed the required data, add data via a rake task or seed file that is manually triggered (via UI or Rails console), or add data directly via the Rails console using FactoryBot.

UAT

This is the most difficult environment to test distribution changes in, and is generally avoided as part of internal testing and user testing. We do not have access to FactoryBot in a UAT instance rails console, so generating mass amounts of data would require us to create Rake tasks which directly use the methods on the many models tied to an appeal. While this is possible, it becomes tedious very quickly.

ProdTest

We always test distribution changes in ProdTest prior to release. The data in ProdTest is a direct copy of production data and is reset daily, making it the perfect way to test our changes in production without actually being there. The extent of testing in ProdTest depends on the effort, but the minimum expectation is that we will at least request a distribution while impersonating a judge via the UI. There are some issues with testing distributions in ProdTest:

ProdTest Database Issues

In ProdTest, both the Caseflow and VACOLS databases are re-created daily. This means that low-level database self-optimizations like caching are not persisted from day-to-day, nor are they copied from the production databases. In practice, this causes queries to fail until those self-optimizations are completed, independent of any code changes. To get around this, we can either:

Request cases, verify that the error is a OCI8 or PG Timeout, repeat until it works
Use the rails console to "warm up" the databases and queries

Using the console, we need to call a method which will query each of the affected tables to ensure that the self-optimizations are thorough enough to complete the distribution. The below steps will cover this:

judge = User.find_by_css_id("{JUDGE_CSS_ID"}) # Or go by JudgeTeam: JudgeTeam.first.judge
dockets = DocketCoordinator.new.dockets.values
dockets.map { |d| d.age_of_n_oldest_priority_appeals_available_to_judge(judge, 12) } # 12 is an arbitrary value here

Once the final command above returns an array of dates as it would during a distribution, the databases should be ready to go for testing distributions.

Other Tips

Clearing all ready-to-distribute appeals

When using the default seed files or when in between tests (for example, during user testing), you may want to or need to get a clean slate of cases ready to distribute.

For legacy appeals, the distribution code specifically looks at appeals in location 81 or 83, so setting those as not ready to distribute is a simple map over the cases:

VACOLS::Case.where(bfcurloc: ['81', '83']).map { |c| c.update!(bfcurloc: 'testing') }

The location "testing" is arbitrary, but makes it easy to undo this change. NOTE: Do not use the word "case" in the map block, as that is a reserved keyword and will raise an error

For AMA appeals, the distribution code is looking at appeals which have an assigned DistributionTask. We can't set the status to an arbitrary value because it is a declared enum on the Task model, so we instead just put them on hold.

DistributionTask.assigned.map(&:on_hold!)

Running distribution-related test files

Copy and paste the command below into a terminal in the Caseflow root directory to run most (if not all) of the distribution-related test files locally. This is helpful for testing something without waiting on Github Actions.

bundle exec rspec \
"spec/jobs/push_priority_appeals_to_judges_job_spec.rb" \
"spec/jobs/update_appeal_affinity_dates_job_spec.rb" \
"spec/jobs/start_distribution_job_spec.rb" \
"spec/controllers/distributions_controller_spec.rb" \
"spec/controllers/case_distribution_levers_controller_spec.rb" \
"spec/models/concerns/by_docket_date_distribution_spec.rb" \
"spec/models/concerns/distribution_concern_spec.rb" \
"spec/models/dockets/direct_review_docket_spec.rb" \
"spec/models/dockets/hearing_request_docket_spec.rb" \
"spec/models/dockets/legacy_docket_spec.rb" \
"spec/models/tasks/distribution_task_spec.rb" \
"spec/models/vacols/case_docket_spec.rb" \
"spec/models/appeal_affinity_spec.rb" \
"spec/models/case_distribution_lever_spec.rb" \
"spec/models/distribution_spec.rb" \
"spec/models/docket_coordinator_spec.rb" \
"spec/models/docket_spec.rb"