This is the master copy of the history updates and testing checklist for Racket releases. Edit this version to add testing responsibilities or to include the commands used to run listed tests.

A copy of this checklist for the current release in progress (or most recently completed) is at Release-Checklist-8.13. That page is edited when testing items are completed.

The template follows this next header:

For one-time items such as blockers, it probably makes sense to check them off using github's markdown (that is, use the edit button to put an "x" between the square brackets. For all standard "testing" and "history" items that are copied from the template, though, please just go ahead and delete them entirely when they're done; since this list is long, it's otherwise difficult to locate the un-checked items.

• {Matthew Flatt [email protected]}

  • Updates:
    • Racket Updates: update HISTORY (updates should show v$RKTNVER as the most current version)

• {Robby Findler [email protected]}

  • Updates:
    • DrRacket Updates: update HISTORY
    • Redex Updates: update HISTORY (updates should show v$RKTNVER as the most current version)

• {John Clements [email protected]}

  • Updates:
    • Stepper Updates: update HISTORY (updates should show v$RKTNVER as the most current version; include the text "merge" in the commit message, or tell me if there are no such changes.)

• {Sam Tobin-Hochstadt [email protected]}

  • Typed Racket Updates: update HISTORY (updates should show v$RKTNVER as the most current version; include the text "merge" in the commit message, or tell me if there are no such changes.)

• {Matthias Felleisen [email protected]}

  • Updates:
    • Teachpack Updates: update HISTORY (updates should show v$RKTNVER as the most current version; include the text "merge" in the commit message, or tell me if there are no such changes.)

• {Blockers}

• {Matthew Flatt [email protected]}

  • Racket Tests

  racket -l tests/racket/test
  

  • Languages Tests

  racket -l tests/htdp-lang/test-htdp
  

  • GRacket Tests (Also check that gracket -z and gracket-text still work in Windows and Mac OS X)

  racket -l tests/gracket/test
  gracket -z
  gracket-text
  

  • mzc --exe tests

  racket -l tests/compiler/embed/test
  

  • .plt-packing Tests

  racket -l tests/racket/test-pack
  

  • Games Tests

  plt-games
  

  • R6RS Tests

  racket -l tests/r6rs/run.sps   # 1 failure expected; 3 failures expected for BC
  

  • PCPS test suite (in "pcps-test" repo)

  racket <file in pcps-test repo>
  

  • Create an executable from a BSL program

  # make big-bang program with literal image in DrRacket, 
  # then create executable
  

  • Try creating a BSL sandbox in DrRacket

  (require racket/sandbox)
  (define e (make-evaluator 'lang/htdp-beginner))
  

  • Run COM tests

  racket -l tests/racket/com
  

  • Embed-in-c test

  racket -l tests/racket/embed-in-c
  

  • Try compiling with -funsigned-char

  configure CPPFLAGS=-funsigned-char ; make
  

  • Try compiling with TEST_ALTERNATE_TARGET_REGISTER

  configure CPPFLAGS=-DTEST_ALTERNATE_TARGET_REGISTER=1 ; make
  

  • Run the unix installer tests (in "distro-build-test" package)

  racket -l distro-build/tests/unix-installer <version>
  

• {Robby Findler [email protected]}

  • DrRacket Tests:

  # from the top-level directory of the release bundle, run
  cd "$(dirname "$(./bin/racket -e '(display (collection-file-path "io.rkt" "tests" "drracket"))')")"
  chmod +x ./run.sh
  # adjust the RACKET environment variable to point to the right binary, eg:
  env RACKET=~/Desktop/Racket\ v6.3.90.900/bin/racket ./run.sh
  

  • Framework Tests:

  # from the top-level directory of the release bundle, run
  cd "$(dirname "$(./bin/racket -e '(display (collection-file-path "main.rkt" "framework" "tests"))')")"
  raco test .
  

  • Contracts Tests:

  env PLT_RUN_UNRELIABLE_TESTS=timing racket -l tests/racket/contract/all
  racket -l tests/racket/contract-stress-argmin
  racket -l tests/racket/contract-stress-take-right
  

  • Games Tests: play a bunch of games (not automated)
  • Teachpacks Tests: image tests

  racket -l 2htdp/tests/bitmap-as-image-in-universe
  racket -l 2htdp/tests/image-equality-performance-htdp
  racket -l 2htdp/tests/image-equality-performance
  racket -l 2htdp/tests/image-too-large
  racket -l 2htdp/tests/test-image
  

  • PLaneT Tests:

  # (the output of these tests is hard to read)
  raco test -l tests/planet/run-all
  

  • Redex Tests:

  racket -l redex/tests/run-tests
  # the results of color-tests require interpretation (and clicking)
  racket -l redex/tests/color-test
  

  • Pict Tests. Uncomment the tests disabled here then

    raco test -p pict pict-lib pict-doc pict-test
    

  • Release tests for the Windows release:

    • Test that the racket-minimal source release compiles fine; follow the instructions in src/README.
    • Test that the binary installers for both work.

  • Mac OS release tests

    • Test that the binary installer for 32-bit macOS works (install, start DrRacket)

• {Sam Tobin-Hochstadt [email protected]}

  • Match Tests:

  raco pkg install --skip-installed --auto racket-test
  raco test -l tests/match/main
  

  • Typed Racket Tests:

  raco pkg install --skip-installed --auto typed-racket-test
  racket -l typed-racket-test -- --all
  

  • Unit Contract Tests

    raco test -l tests/units/test-unit-contracts
    

  • Contract Region Tests

    racket -l tests/racket/contract/define-contract
    racket -l tests/racket/contract/with-contract
    

  • Class Contract Tests

    racket -l tests/racket/contract/class
    

  • EoPL Tests

    raco test -c eopl/tests
    

• {Matthias Felleisen [email protected]}

  • Teachpacks Tests: check that new teachpacks are addable

    1. create foo.rkt: #lang racket (define x 0) (provide x)
    2. create bar.rkt: #lang htdp/bsl (require "foo.rkt") x; RUN and check for 0 to pop out
    3. delete require line, use teach pack menu to add foo.rkt; RUN and check for 0 to pop out
    4. delete foo.rkt and bar.rkt
    

  • Teachpack Docs: check teachpack docs in the bundles

    type "big-bang" into Definitions area, highlight, use F1; check for manuals 
    

  • Try teaching-languages testing framework (check-expect)

    run tests in plt: extra-pkgs/htdp/htdp-test/tests/test-engine/
    

  • Test #lang htdp/* languages

• {Ryan Culpepper [email protected]}

  • syntax-parse Tests

    raco test -c tests/stxparse
    

  • Data Tests

    raco test -c tests/data
    

  • Macro Debugger Tests

    racket -l tests/macro-debugger/all-tests -- --gui
    

  • RackUnit GUI Tests

    # not automated; use the Macro Stepper tests to exercise the RackUnit GUI
    

  • DB Tests

    # basic tests using sqlite3
    raco test -c tests/db
    # server-dependent tests: pg, my, etc
    ./share/pkgs/db-test/tests/db/bin/docker-util.sh pg     # start server
    racket -l tests/db/all-tests -- -t pg                   # run tests
    ./share/pkgs/db-test/tests/db/bin/docker-util.sh stop   # stop server
    

  • Rackunit Tests

    # note: some tests are intended to fail
    racket -l tests/rackunit/run-tests
    

  • SRFI Tests

    racket -l tests/srfi/run-tests
    

• {John Clements [email protected]}

  • Stepper Tests

  tests/stepper/run-manual-tests.rkt ;; run in DrRacket, ensure that expected failures occur.
  tests/stepper/manual-tests.txt ;; follow the instructions in this file
  

  • Release tests for (a chosen) linux release:
    • Test that a racket source release compiles on linux Hint: ../configure --enable-bc --enable-cs
    • Test that a racket-minimal source release compiles on linux
    • Test that every binary installer for linux works in normal mode (install then start racket) [FIXME: is this enough?]
    • Test that a binary installer for linux works in unix-style mode (install then start racket) [FIXME: is this enough?]
  • Release tests for the Mac OS release:

    • On MacOS: Build from sources using the minimal distribution. First thing is to delete existing libraries you have for the test version, they will otherwise collide.

      rm -r ~/Library/Racket/$RKTNVER/
      cd src
      mkdir build
      cd build
      ../configure
      make
      make install
      cd ../..
      

      then

      ./bin/raco pkg install --auto -i racket-lib
      ./bin/raco pkg install --auto -i main-distribution
      

      (Why not just test the full source release? "The problem is that when packages are already included in a source bundle, then it doesn't include the right versions of packages that have to be platform-specific, such as the one that supplies "libintl.9.dylib".)

    • Test that every binary installer other than the 32-bit one for MacOS works (install, start DrRacket)

  • FrTime Tests (in frtime/demos) set language level to frtime before running
    • Test that expressions with time-varying values (e.g., seconds, (build-list (modulo seconds 10) identity)) render and update as expected in the FrTime language level. Check that they continue updating even after a garbage collection.
    • Test that a sampling of graphical demos (e.g., analog-clock.rkt, tetris.rkt) work as expected, responding to relevant key and mouse events and interactions with control widgets.
  • Inspect icons

    raco docs icons # then look at them (look for icons, ensure no red error text on the page)
    

• {Jay McCarthy [email protected]}

  • Web Server Tests

    raco test -c tests/web-server
    

  • XML Tests

    raco test -c tests/xml
    

  • HTML Tests

    raco test -c tests/html
    

  • PLAI Tests

    raco test -c plai
    

  • Racklog tests

    raco test -c racklog/tests
    

  • Datalog tests

    raco test -c datalog/tests
    

• {Stephen Chang [email protected]}

  • Lazy Racket Tests

    raco test -l lazy/tests/main.rkt
    

  • Lazy stepper tests

    raco pkg install --skip-installed --auto htdp-test
    raco test -l tests/stepper/automatic-tests.rkt
    

• {Mike Sperber [email protected]}

  • DMdA Tests Test properties, World teachpack
  • Stepper Tests Check format of lists, records
  • Signature Tests Test signatures