Reporting checklist - TMSMultiLab/TMSMultiLab GitHub Wiki
This page details past work on TMS reporting quality assessments. All the data here are from meta-analyses which have used the Chipchase et al. (2012) reporting checklist to assess the studies included in their meta-analysis. Some of these meta-analyses have added questions and operational definitions to the Checklist.
The data and figures come from .csv files and matlab code in the the Reporting folder.
The latest work on developing and testing a reporting checklist is in progress.
The proportion of Checklist criteria Reported (in red), or Controlled (in blue), by 681 TMS studies, according to Chipchase's (2012) checklist. Year is on the x-axis of all panels, and the proportion of criteria that each study meets is on all y-axes. The top two panels show all individual studies (dots) with a best-fit linear regression (solid line), 95% confidence interval (broken lines), and R2. The lower two panels show the means (squares) and 95% confidence intervals (vertical lines) for data across articles published in each year.
The proportion of 681 TMS studies which Report (in red), or Control (in blue) each of the 30 items of Chipchase's (2012) checklist separately. Item number (1-30) is on the x-axis. Data shows the mean (circle) and the bootstrapped 95% confidence intervals (vertical lines) across all studies. Items 1-8 are for Participant factors; 9-25 are Method factors; 26-28 apply to Paired-pulse TMS studies only; 29-30 are Analysis factors.
The proportion of criteria reported, for 126 TMS studies across the four most-commonly assessed journals (plus 'Other' articles not (yet) published in journals. There are no clear differences between journals, even for the journal that published the Chipchase et al. (2012) checklist (Clinical Neurophysiology).
The mean proportions of criteria reported, for the 25 most-commonly assessed journals (plus 'Other' articles not (yet) published in journals. Data are mean (filled circles) and 95% confidence intervals (lines).
Which checklist items are used in different articles is indicated with these codes:
- C: from Chipchase et al. (2012) (Delphi process)
- R: from Rohel et al. 2021
- G: from Gefferie et al. 2023
Operational definitions were first taken from Rohel et al. (2021) and are abbreviated in these tables. Inter-rater reliabilities are also taken from Rohel et al. (2021), who reported Gwet's AC1 across 41 studies.
N/A = not applicable
NSD = not significantly different
- = not assessed or no data
The percentages of studies Reporting and Controlling come from a meta-review of 32 meta-analyses reporting 681 studies. See the Reporting folder for the data and matlab analysis code. Numbers are means and {bootstrapped 95% confidence intervals}.
| Criterion & sources | Operational definition Reported |
Operational definition Controlled |
% Reported Mean {95%CI} |
% Controlled Mean {95%CI} |
Inter-rater agreement Gwet's AC1 {95%CI} |
|---|---|---|---|---|---|
|
Age C1 R1 G1 |
Participant age is reported | If 2+ groups, age has been statistically compared between groups & is not NSD N/A if only 1 group |
89.6 {86.5, 92.6} |
28.7 {20, 38.9} |
0.97 {0.92, 1.00} |
|
Gender C2 R2 G2 |
Participant gender is reported | If 2+ groups, proportion male & female statistically compared between groups (eg, Chi-square test) & is NSD N/A if 1 group |
90 {87, 92.8} |
12.9 {4.84, 21} |
0.92 {0.82, 1.00} |
|
Handedness C3 R3 G3 |
Participant handedness is reported | If 2+ groups, proportion right- & left-handed statistically compared between groups (eg, Chi-square test) & is NSD N/A if 1 group |
51.4 {45.8, 57} |
43.1 {31.9, 54.2} |
0.79 {0.62, 0.95} |
|
Prescribed medication C4 G4 |
28.6 {23, 34.3} |
7.59 {3.8, 12} |
- | ||
|
CNS active drugs C5 R4b |
N/A | Statement that participants with neuroactive drugs are excluded |
40.4 {34.5, 46.3} |
19 {8.62, 29.3} |
- |
|
Neurological or psychiatric conditions C6 R4a G5 |
N/A | Statement that participants with neurological conditions are excluded |
54 {46.6, 60.8} |
41.9 {32.4, 51.4} |
0.59 {0.33, 0.86} |
|
Medical conditions C7 |
55.5 {49.4, 61.6} |
37.5 {27.3, 47.7} |
- | ||
|
History of specific motor activity C8 R5 |
Proportion of participants with a significant history of repetitive motor activity is reported (eg, musician, athlete, video games, repetitive work) | If 2+ groups, proportion performing repetitive motor activity statistically compared between groups (eg, Chi-square test) & is NSD N/A if 1 group |
18.6 {14.2, 23.4} |
2.78 {0, 6.94} |
0.97 {0.92, 1.00} |
| Criterion & sources | Operational definition Reported |
Operational definition Controlled |
Reported % |
Controlled % |
Inter-rater agreement Gwet's AC1 {95%CI} |
|---|---|---|---|---|---|
|
Position & contact of EMG electrode C9 R6 |
Preparation & procedure of EMG electrode placement is described with sufficient detail to allow replication. For intrinsic hand muscle, statement that electrodes placed in belly-tendon or belly-belly configuration is sufficient. For other muscles, exact configuration required. Citation of recognised standards is sufficient | Citations provided to justify EMG preparation methods (SENIAM, ISEK, etc) |
80.9 {76.2, 85} |
39.9 {30.9, 49.1} |
0.83 {0.68, 0.98} |
|
Contraction level of target muscles C10 R7 |
For experiments performed at rest, method reported for ensuring the participant is relaxed (eg, visual EMG inspection, online EMG feedback). For experiments performed during voluntary contraction, target EMG level (eg, %MVC) is described and feedback given to allow the participant to achieve the target contraction level | Level of EMG activity immediately before TMS (eg, RMS value) statistically compared between conditions and is not significantly different |
73.4 {68.4, 78.2} |
32.1 {23.9, 41.3} |
0.55 {0.33, 0.78} |
|
Prior motor activity of target muscles C11 R8 |
N/A | Level of motor activities performed between trials (eg, during breaks in the experiments) is controlled. Includes at least instructions about the level of activity wanted (most of the time "stay relaxed") during the experiment. For experiments including the evaluation of both active and resting MEPs, the interval between and/or the order of the conditions should be reported and justified to avoid contamination of resting MEPs by prior active contraction |
45 {38.8, 51.2} |
34.9 {25.7, 44} |
0.45 {0.14, 0.76} |
|
Contraction level of non-target muscles C12 R9 |
N/A | The process of excluding trials based on the level of relaxation / contraction of other muscles is reported and sound. For instance, visual inspection of pre-stimulus EMG in muscles other than those being tested |
32.7 {26.5, 39} |
16.7 {10.2, 24.1} |
0.87 {0.74, 1.00} |
|
Coil type (size & geometry) C13a R10 G15 |
Coil geometry reported in sufficient detail to allow replication | N/A unless study performed in multiple centers or with multiple coils, then: same coil type used for all participants |
86.3 {82.7, 89.5} |
60.4 {45.8, 75} |
0.84 {0.71, 0.98} |
|
Coil orientation C14 R11a G17 |
Coil orientation reported with sufficient detail to allow replication | Use of neuronavigation system to control coil orientation |
64 {59.1, 68.9} |
27.1 {19.1, 35.5} |
0.73 {0.55, 0.91} |
|
Direction of induced current C15 R11b G19 |
Direction of induced current in brain reported with sufficient detail to allow replication | Use of neuronavigation system to control coil orientation |
41.7 {36.4, 46.9} |
27.5 {17.4, 37.7} |
- |
|
Coil location C16a R12a G18 |
A sound method for ensuring coil location has been used with or without the use of a neuronavigation system. For instance, marking the stimulation location on subjects scalp is sufficient | Use of neuronavigation system to control coil location |
65.7 {61.1, 70.3} |
28 {20, 37.3} |
0.89 {0.77, 1.00} |
|
Coil stability C16b R12b |
Method for ensuring coil stability used with or without neuronavigation system. Marking the stimulation location on scalp is sufficient | Use of neuronavigation system to control coil stability |
65.7 {61.1, 70.3} |
28 {20, 37.3} |
- |
|
Stimulator C17 R13a G14 |
Type of stimulator is reported | N/A unless study performed in multiple centers, then: same stimulator type used for all participants |
92.8 {90, 95.4} |
94 {88.1, 98.5} |
0.91 {0.81, 1.00} |
|
Stimulation intensity C18 R14 G21 |
Stimulation intensity reported with sufficient detail to allow replication | Stimulation intensity is determined individually for each participant (eg, based on motor threshold or standardized targeted MEP size) |
84.1 {80.3, 87.7} |
80.4 {72.9, 87.9} |
0.70 {0.52, 0.89} |
|
Pulse shape C19 R13b G24 |
Pulse shape used is reported | N/A unless study performed in multiple centers, then: same pulse shape used for all participants |
29.2 {24.4, 34.1} |
26.9 {16.4, 38.8} |
- |
|
Determination of optimal hotspot C20 R15 |
Method described with sufficient detail to allow replication, or referenced | Appropriate reference or justification provided to justify method. If 2+ muscles evaluated, hotspot determined individually for each muscle |
65.4 {60.6, 70.1} |
34.4 {25.5, 43.6} |
0.43 {0.20, 0.66} |
|
Time between MEP trials C21 R16 G26 |
Time between MEP trials is reported with sufficient detail to allow replication | Time between MEP trials should vary slightly from one trial to the other in order to prevent anticipation of TMS |
37.6 {32.7, 42.5} |
37.6 {28.4, 46.8} |
0.51 {0.28, 0.74} |
|
Time between days of testing C22 R16 G13 |
Time between days of testing reported with sufficient detail to allow replication | If more than one group of participants, time between days of testing should be the same between groups N/A if only one group of participants N/A if only one day of testing reported and controlled criteria are N/A |
91.8 {87.1, 95.9} |
70.2 {57.4, 83} |
0.83 {0.69, 0.98} |
|
Subject attention (arousal) during testing C23 R17 |
N/A | Sound method to control subjects' attention during, reported with sufficient detail to allow replication. Task controlling attention should be used when testing subjects at rest (eg, counting stimuli, detecting visual stimuli, pain rating after each block). When active TMS is evaluated, the presence of a target contraction to sustain with feedback is sufficient to control subjects' attention |
18.2 {13.1, 23.4} |
34.4 {26.4, 43.6} |
0.29 {-0.03, 0.60} |
|
Method to determine motor threshold C24 R19 |
Statement explaining how motor threshold was determined with sufficient detail to allow replication, or referenced | Appropriate references provided to justify method. If 2+ muscles evaluated, threshold determined individually for each tested muscle |
73.6 {69, 77.8} |
46.3 {37.3, 55.5} |
0.71 {0.51, 0.90} |
|
Number of MEP measures made C25 R20, G27 |
Number of MEP measures in each condition is reported | Number of MEPs included in the analyses and how excluded stimulations were managed is described (eg if X stimulation were not valid, were they retested or excluded from analyses) |
62.8 {57.9, 67.6} |
42.7 {33.6, 51.8} |
0.60 {0.39, 0.82} |
| Criterion & sources | Operational definition Reported |
Operational definition Controlled |
Reported % |
Controlled % |
Inter-rater agreement Gwet's AC1 {95%CI} |
|---|---|---|---|---|---|
|
Paired-pulse: Intensity of test pulse C26 |
36.1 {29, 43.2} |
12.7 {7.46, 18.7} |
- | ||
|
Paired-pulse: Intensity of conditioning pulse C27 |
36.3 {29.1, 43.4} |
13.4 {8.21, 19.4} |
- | ||
|
Paired-pulse: Inter-stimulus interval C28 G25 |
36.4 {29.4, 43.3} |
4.88 {1.63, 8.94} |
- |
| Criterion & sources | Operational definition Reported |
Operational definition Controlled |
Reported % |
Controlled % |
Inter-rater agreement Gwet's AC1 {95%CI} |
|---|---|---|---|---|---|
|
Method for determining MEP size during analysis C29 R21 |
Method described with sufficient detail to allow replication | If method is reported it is automatically controlled |
74.3 {69.8, 78.8} |
73.9 {65.5, 81.8} |
0.84 {0.68, 0.99} |
|
Size of unconditioned MEP C30 |
55.7 {50.2, 61.3} |
40 {26.7, 53.3} |
- |