Block and booklet design - tmatta/lsasim GitHub Wiki
The function block_design facilitates the determination of the number of blocks and the assignment of items to those blocks with two required arguments and one optional argument.
lsasim::block_design(n_blocks = NULL, item_parameters,
item_block_matrix = NULL)The n_blocks argument specifies the number of blocks while the item_parameters argument takes a data frame of item parameters. The item_block_matrix argument takes an (I \times K) indicator matrix where (K) is the number of blocks and (I) is the number of items. For the following examples, we will be generating (I = 15) items from a model and (K = 4) blocks.
I <- 15
K <- 4
item_pool <- lsasim::item_gen(n_2pl = I, thresholds = 2, b_bounds = c(-2, 2),
a_bounds = c(.75, 1.25))When the item-block indicator matrix is omitted, the item_gen function allocates items to blocks using a default spiraling design. For (1, 2, ..., K) blocks, the first item is assigned to block 1, the second item is assigned to the block 2, and the (K)th item is assigned to block (K). The process is continued such that item (K+1) is assigned to block 1, item (K+2) is assigned to block 2 and item (K+K) is assigned to block (K) until all items are assigned to a block.
block_ex1 <- lsasim::block_design(n_blocks = K, item_parameters = item_pool,
item_block_matrix = NULL)
print(block_ex1)## $block_assignment
## b1 b2 b3 b4
## i1 1 2 3 4
## i2 5 6 7 8
## i3 9 10 11 12
## i4 13 14 15 0
##
## $block_descriptives
## block length average difficulty
## b1 4 0.152
## b2 4 -0.098
## b3 4 0.170
## b4 3 -0.300
block_ex1 is a list that contains two elements. The first element, block_assignment, is a matrix that that identifies which items from the item pool correspond to which block. The column names of block_assignment begin with b to indicate block while the rows begin with i to indicate item. For block b1, the first item, i1, is item 1 from item_pool, the second item, i2, is item 5 from item_pool, the third item, i3, is item from 9 from item_pool, and the fourth item, i4, is the item 13 from item_pool. Because the 15 items do not evenly distribute across 4 blocks, the fourth block only contains 3 items. To avoid dealing with ragged matrices, all shorter blocks are filled with 0s.
print(block_ex1$block_assignment)## b1 b2 b3 b4
## i1 1 2 3 4
## i2 5 6 7 8
## i3 9 10 11 12
## i4 13 14 15 0
The second element in block_ex1 is a table of descriptive statistics for each block. This table indicates the number of items in each block and the average difficulty for each block. Again, notice blocks 1 through 3 each have 4 items while block 4 only has 3 items. Furthermore, the easiest block is b4 with an average difficulty of -0.300 while the most difficult block is b3 with an average difficulty of 0.170.
print(block_ex1$block_descriptives)## block length average difficulty
## b1 4 0.152
## b2 4 -0.098
## b3 4 0.170
## b4 3 -0.300
As mentioned above, users have the capability of specifying which items belong to which blocks using the item_block_matrix argument. Keeping with our example of (I = 15) items and (K = 4) blocks, we create such a matrix where a 1 indicates which item (row) is associated with which book (column).
i_b_design <- matrix(c(0, 1, 0, 0,
0, 1, 0, 0,
0, 0, 1, 0,
0, 0, 1, 0,
0, 0, 0, 1,
0, 0, 0, 1,
1, 0, 0, 0,
1, 0, 0, 0,
0, 1, 0, 0,
0, 1, 0, 0,
0, 0, 1, 0,
0, 0, 1, 0,
0, 0, 0, 1,
0, 0, 0, 1,
1, 0, 0, 0), nrow = I, ncol = K, byrow = T)
block_ex2 <- lsasim::block_design(n_blocks = K, item_parameters = item_pool,
item_block_matrix = i_b_design)Now, block 1 includes items 7, 8, and 15, the row numbers with a 1 for column 1 in i_b_design.
print(block_ex2$block_assignment)## b1 b2 b3 b4
## i1 7 1 3 5
## i2 8 2 4 6
## i3 15 9 11 13
## i4 0 10 12 14
After the items have been assigned to blocks, the blocks are then assigned to test booklets. The booklet_design function facilitates this process with one required argument and one optional argument. The item_block_assignment argument takes the block_assignment matrix from block_design. The book_design argument takes a (B \times K) indicator matrix where (B) is the number of booklets and (K) is the number of blocks.
lsasim::booklet_design(item_block_assignment, book_design = NULL)Like block_design, booklet_design provides a default spiraling booklet assembly method, as illustrated by book_ex1 below. Using the blocks from block_ex1, the first book, B1 include the items from block 1 (1, 5, 9, 13) and the items from block 2 (2, 6, 10, 14). Book B2 includes the items from block 2 and block 3, book B3 includes the items from block 3 and block 4, and book B4 includes the items from block 1 and block 4. Notice that the first two test booklets contain 8 items while the last two books contain 7 items. This is because block 4 only has 3 items whereas blocks 1, 2, and 3 have 4 items. Like block_design,booklet_design avoids ragged matrices by filling shorter booklets with 0s.
book_ex1 <- lsasim::booklet_design(item_block_assignment = block_ex1$block_assignment)
print(book_ex1)## B1 B2 B3 B4
## i1 1 2 3 1
## i2 5 6 7 5
## i3 9 10 11 9
## i4 13 14 15 13
## i5 2 3 4 4
## i6 6 7 8 8
## i7 10 11 12 12
## i8 14 15 0 0
Users can also explicitly specify the booklet assembly design using the optional argument book_design. By specifying a booklet design matrix, users can control which item blocks go to which booklets. The book_design argument takes an indicator matrix where the columns indicate the blocks and the rows indicate the booklets.
In the code below, we create a matrix, block_book_design, that will create six test booklets. Booklet 1 (row 1) will include items from blocks 1, 3 and 4 while booklet 6 (row 6) will include items from block 1 only.
block_book_design <- matrix(c(1, 0, 1, 1,
0, 1, 0, 0,
0, 0, 1, 1,
0, 0, 0, 1,
0, 1, 1, 1,
1, 0, 0, 0), ncol = K, byrow = TRUE)
book_ex2 <- lsasim::booklet_design(item_block_assignment = block_ex1$block_assignment,
book_design = block_book_design)
print(book_ex2)## B1 B2 B3 B4 B5 B6
## i1 1 2 3 4 2 1
## i2 5 6 7 8 6 5
## i3 9 10 11 12 10 9
## i4 13 14 15 0 14 13
## i5 3 0 4 0 3 0
## i6 7 0 8 0 7 0
## i7 11 0 12 0 11 0
## i8 15 0 0 0 15 0
## i9 4 0 0 0 4 0
## i10 8 0 0 0 8 0
## i11 12 0 0 0 12 0
The booklet_design was design to create multiple test booklets from multiple item blocks, and thus, default booklet assembly requires more than 2 blocks. To work with less than three blocks requires a user-specified booklet design matrix.
block_ex3 <- lsasim::block_design(n_blocks = 1, item_parameters = item_pool)
lsasim::booklet_design(item_block_assignment = block_ex3$block_assignment)## Error: Default booklet assembly requires more than 2 blocks
a single booklet from a single item block, we need to make the booklet design matrix equal to 1. Note that <the argument in book_design> must be class "matrix".
book_ex3 <- lsasim::booklet_design(item_block_assignment = block_ex3$block_assignment,
book_design = matrix(1))
print(book_ex3)## B1
## i1 1
## i2 2
## i3 3
## i4 4
## i5 5
## i6 6
## i7 7
## i8 8
## i9 9
## i10 10
## i11 11
## i12 12
## i13 13
## i14 14
## i15 15
Creating two booklets, each containing one of two item blocks requires a size-2 identity matrix to be specified for the book_design argument.
block_ex4 <- lsasim::block_design(n_blocks = 2, item_parameters = item_pool)
book_ex4 <- booklet_design(item_block_assignment = block_ex4$block_assignment,
book_design = diag(1, 2))
print(book_ex4)## B1 B2
## i1 1 2
## i2 3 4
## i3 5 6
## i4 7 8
## i5 9 10
## i6 11 12
## i7 13 14
## i8 15 0