Trustworthy Medical Vision Language Models

Technical literature review

1. Executive assessment

The promising starting point is a narrowly defined chest radiograph task in which the system generates image-grounded findings, makes uncertainty visible, and supports clinician review before communicating results to patients. In the current literature, MedGemma offers a practical medical image and language foundation. The other references provide complementary ideas for inspecting internal representations, intervening on potentially harmful features, and constraining diagnostic reasoning via valid clinical concepts. Overall, the proposed combination is a research opportunity, while the existing reviewed studies do not establish that the combined system is clinically safe.

The proposed first-study question can be defined as follows: Can an explicit finding record and checked record-to-text generation reduce clinically meaningful unsupported chest-radiograph assertions while preserving finding sensitivity and uncertainty, at a measured local inference cost? The current summary distinguishes three kinds of statements across the board, namely: (a) reported findings which are supported by a specified paper or official model document, (b) critical analysis which evaluates what those findings do and do not establish, and (c) proposed experiments, which are recommendations for the proposed research and have not been performed.

At this stage, the main conclusions are as follows.

1. A named concept or a plausible explanation is considered as inadequate evidence of correctness. Hence, it is important to examine whether the concept is visible in the image, predicts the intended finding, and changes downstream output selectively when corrected.

2. Bias mitigation must preserve clinically justified information. Maintaining equal output rates across demographic groups are not an appropriate universal medical objective. Measure clinically relevant errors and calibration within appropriately defined groups.

3. XpertCausal is particularly useful as a structured reasoning reference as it explicitly separates the disease-to-finding generative direction from inference of disease given findings. Its results concern a restricted classification task, and not unrestricted medical report generation.

4. MedGemma 1.5 adds useful capabilities, but its performance depends on dataset, label policy, prompt, and task. A newer version does not win every benchmark. Broad benchmark improvements do not naturally demonstrate safe patient communication.

5. The first proposed research study should establish a reproducible baseline and an independent clinical evaluation set before adding multiple interpretability modules. Otherwise, a complex pipeline may improve an attractive metric, but resulting in clinically important omissions.

2. References and evidence scope

Reference	Verified identity	Main role in this review
[1] CB-SAE	Kulkarni et al., Interpretable and Steerable Concept Bottleneck Sparse Autoencoders, CVPR 2026	Concept interpretation and representation steering
[2] DeBiasLens	An et al., Interpretable Debiasing of Vision-Language Models for Social Fairness, CVPR 2026	Sparse feature interventions for social bias auditing
[3] XpertCausal	Rafferty, Ramaesh and Rajan, arXiv 2605.07785, version 3, 21 August 2026	Expert-constrained disease and finding inference
[4] MedGemma	Sellergren et al., MedGemma Technical Report, 2025	Original medical foundation model family
[5–7] MedGemma 1.5 and documentation	Versioned 2026 technical report and official Google model cards	Medical VLM baseline, preprocessing, evaluation and use limitations
[8–23] Proposal sources	Dataset, metric, platform, guidance and venue documentation, and the 8 October proposal	Feasibility, measurement and research-conduct constraints for Section 8

The associated CVPR papers and exact XpertCausal version anchor the current review. MedGemma was named without a version, hence both the original family and the 1.5 4B model are considered. Numerical comparisons retain their source-specific task and metric definitions. MedGemma results primarily use the versioned 1.5 technical report, with material differences in the living model card identified explicitly. XpertCausal can be treated as a preprint only at this stage.

3 Concepts required to interpret the references

3.1 Five objects requiring varying evidence

In sequence:

• A vision encoder converts an image into numerical features
• A sparse autoencoder decomposes features into a limited set of active latent components
• A concept bottleneck predicts a human-readable intermediate representation
• A probabilistic clinical graph constrains relationships between observations and diagnostic hypotheses
• A language generator then communicates information, potentially adding errors of its own

For the proposed medical application, the above stages require different tests.

Object being evaluated	Useful question
Latent feature	Does activation consistently track the intended visual property?
Clinical concept	Is the finding present, absent, or uncertain in this image?
Reasoning pathway	Does the prediction depend on appropriate evidence?
Generated statement	Is every assertion supported and appropriately qualified?
Clinical workflow	Does assistance improve decisions without unacceptable harm?

3.2 Sparse reconstruction and concept bottlenecks

A useful generic representation is an image feature vector h, a sparse encoding z, and a reconstruction h_hat. With a linear decoder, a common form can be defined as follows:

z = E(h)
h_hat = D(z) = Wz + b

The encoder E is trained such that only a small number of components are active while the decoder approximately reconstructs h. Sparsity does not guarantee that each component has one meaning. At the same time, correlated properties may share features, and a meaningful property may be spread across several features. For example, DeBiasLens provides a concrete example of fitting and probing such representations [2, Section 3].

A concept bottleneck instead makes a proposed semantic interface explicit as such:

image x → concept probabilities q(c | x) → task prediction

For medical use, a concept record should distinguish observation, location, laterality, severity, uncertainty, and provenance. A probability of “opacity” should not silently become a diagnosis of pneumonia. The distinction between observations and disease hypotheses is central to XpertCausal [3, Sections 2–3].

3.3 Intervention is not the same as causal identification

Modifying a latent and observing a changed answer establishes a computational effect within the model. However, it does not establish that the latent represents a unique real-world cause. Similarly, updating a disease probability after a clinician corrects a concept is an evidence update. Overall, it is not automatically a causal intervention on a patient or proof of a valid counterfactual medical claim. Therefore, this proposed research must therefore use precise language in the contexts of (a) feature association, (b) model intervention, (c) expert-constrained inference, and (d) clinically validated outcome, which represent different levels of evidence. The distinction hence prevents interpretability results from being promoted into unsupported safety claims.

4 Review of concept bottleneck sparse autoencoders

4.1 Source summary

Concept Bottleneck Sparse Autoencoders (CB-SAE) ranks and prunes sparse autoencoders (SAE) neurons using interpretability and steering scores, freezes the retained branch, and adds a concept encoder and decoder. The added branch targets user-defined concepts which are absent from the retained SAE. Hence, reconstruction, CLIP-based concept supervision, and cyclic concept reconstruction train the new branch, while Top-k activation enforces sparsity. The existing evaluations in the literature leverage ImageNet, LLaVA, and UnCLIP, and not official medical images. [1, Sections 3–6]

Model	Metric	SAE	CB-SAE
LLaVA 1.5 7B	CLIP-Dissect interpretability	0.154	0.244
LLaVA 1.5 7B	White-image steering	0.203	0.250
LLaVA MORE	Unit-vector steering	0.179	0.192

Source: [1, Table 1, proceedings p. 2924]. For the above listed LLaVA results, steering activates a concept using a blank-image representation or an isolated latent and compares the generated output with that concept. Note that the above listed scores measure representation control, and not clinical correctness.

4.2 Critical reading for medical adoption

Original analysis. The most useful question is whether semantic supervision provides researchers a better interface for detecting and correcting failures. In a medical system, due to the level of safety and trustworthiness involved, this interface has to survive several tests that a generic concept label alone cannot answer.

To begin with, a candidate's “pleural effusion” feature must be evaluated against independent clinical annotations, including absent and uncertain cases. A feature name generated by a VLM should always be first treated as a hypothesis. If the same model supplies semantic targets and judges their interpretation, the agreement can reflect a shared error. Independent radiologist review and held-out image evidence are therefore required to break this dependency.

Next, semantically related concepts require explicit separation. Opacity, consolidation, infection, and pneumonia can be associated without being interchangeable. A representation that groups them into one controllable direction might be considered convenient for steering text, but yet inadequate for complete diagnostic reasoning. Concept-specific confusion analysis should thus demonstrate what is gained and what distinctions are lost.

In addition, global image features can conceal localization failure. A concept may correlate with a finding without identifying its anatomical site. In radiology, an otherwise correct finding attached to the wrong lung or the wrong study is a consequential error, Therefore, it is important to pair feature evaluation with localization and laterality checks.

Lastly, steering can create an answer that agrees with the instruction while becoming less faithful to the image. A successful experiment should not thus reward the model merely for mentioning or highlighting a requested concept. It should require that the output changes when the evidence warrants a correction and remain stable when a deliberate irrelevant or mistaken edit is introduced into the instruction.

4.3 Proposed medical adaptation

The proposition is to begin with a small radiologist-reviewed vocabulary for one imaging task. For each concept, document its intended meaning, image support requirements, common confusions, and an explicit uncertain state. Adopt independent image annotations for evaluation, even if weak supervision or a VLM helps to generate training labels.

Compare an ordinary sparse representation, a semantically aligned representation, and a supervised concept head using the same frozen medical encoder and the same data splits. By doing so, this isolates the value of semantic alignment from improvements due to a better visual backbone. In addition, holding reconstruction quality and intervention magnitude visible in the overall comparison.

For each selected latent, collect (a) positive examples, (b) counterexamples, and (c) examples from other sites or acquisition settings. Checek whether the latent tracks pathology, equipment, image quality, or a co-occurring treatment device, followed by evaluate the different feature stability across random seeds and training subsets. Note that a useful clinical concept should not depend on a single convenient run.

A controlled editing study should include correct concept corrections, incorrect corrections, and random edits with comparable magnitude such that we can measure intended output changes and unintended changes separately. The desired outcome is selective, evidence-consistent correction, and not maximum obedience to an intervention.

For the first proposed research, we can adopt the above direction as a hypothesis-generating audit of medical representations. We can add it to the prediction or report-generation path only after its concepts demonstrate independent clinical meaning and that controlled interventions preserve important findings. A small, carefully assessed concept inventory is expected to be more informative initially than a very large dictionary with attractive but weakly verified labels.

5 Review of interpretable VLM debiasing

5.1 Problem and contribution

DeBiasLens aims to reduce social bias by locating and suppressing selected sparse features in frozen VLM representations. It evaluates both CLIP-style retrieval and larger image-to-text models. The contribution is an inspectable intervention mechanism that can be applied without updating the original backbone weights. The SAE itself still requires training, while the medical images and clinical diagnostic fairness are not evaluated [2, Sections 3–4]. An important qualification is that SAE training does not use demographic labels, but subsequent identification of group-specific features does. “Label-free training” should not be interpreted as a completely annotation-free fairness pipeline [2, Section 3.2 and Table 1].

5.2 Training and feature identification

The general methodology is to attach an SAE to encoder features. For example, a Matryoshka reconstruction objective evaluates several levels of latent participation, combines reconstruction loss with sparsity, and adds an auxiliary term for inactive features. In simplified notation, the main objective is modelled as such:

L_main(h) = Σ_m ||h − D(z_1:m)||² + λ ||z||₁
L_total = L_main + β ||reconstruction residual − auxiliary reconstruction||²

where z_1:m denotes the selected latent subset at a reconstruction level. Note that the notation explains the components rather than reproducing implementation-specific tensor orientations [2, Equation 1 and Section 3.1].

For a demographic group g containing S_g samples, the method selects features active in at least floor(τS_g) samples. It excludes features that also meet the effectiveness criterion for other groups, followed by selecting the remaining latent with the highest mean activation for each group. Group specificity here is defined by the thresholded selection rule, and it does not mean that the feature never activates outside that group [2, Section 3.2]. Selected features are suppressed, decoded, and mixed with the original representation as follows:

h_new = α D(z_edited) + (1 − α) h

The mixture weight α controls the intervention strength. The reported main configuration uses expansion factor 8, τ = 0.9, and α = 0.6. However, the earlier Section 4.3 describes an optimal threshold of 0.1. Overall, this needs clarification, or trial and error, before reproduction and it should not be silently resolved by choosing a more convenient value [2, Sections 4.1–4.3].

5.3 Quantitative evidence and trade offs

The below table reports FairFace gender-retrieval Max Skew at 1,000 results, scaled by 100. Lower values mean less deviation from the specified demographic target distribution. These numbers are not considered as diagnostic accuracy or percentages of patients harmed.

Prompt category	Reproduced CLIP ViT B16	DeBiasLens on text features	Absolute reduction
Adjectives	21.9	7.1	14.8
Occupations	33.5	16.2	17.3
Activities	19.8	14.2	5.6
Stereotypes	32.5	8.1	24.4

Source: [2, Table 1]. Reductions are calculated from the reproduced baseline. Other methods outperform this variant on some prompt categories. Overall, text-only intervention is not universally best since changing model size or applying image and text interventions together changes the ranking. Combining modalities does not necessarily generate a reliably additive benefit.

The supplementary results expose the cost of intervention more clearly than a broad claim of capability preservation. The following is the InternVL2-8B FairFace configuration with top-neuron intervention.

InternVL2 8B metric	α = 0	α = 0.6	Interpretation
MME perception	1646.79	1616.65	Some perception capability decreases
MME reasoning	536.78	478.21	Larger reasoning-score reduction
MMMU	0.43	0.39	Four percentage points lower
SEED Bench 2	0.75	0.74	One percentage point lower

Source: [2, Supplement, Table S6, p. 4; arXiv Table 10]. Different metrics use different scales and should not be averaged without a defined normalization. Supplement Table S8 also reports gender predictability of 95.9% in original features and 95.2% in mixed features, where the improved measured output fairness does not imply removal of demographic information.

5.4 A reconstruction confound that should be controlled

Original derivation. Let D(z) = Wz + b and δz = z_edited − z. The intervention can be written as follows:

h_new = h + α[D(z) − h] + αWδz

The above formulation separates reconstruction distortion from the targeted edit. A change in output cannot automatically be credited entirely to suppressing the chosen feature. A medical experiment should thus compare the following conditions under matched data and decoding settings.

Condition	Representation	What it isolates
Original	h	Baseline
Reconstruction only	h + α[D(z) − h]	Effect of SAE reconstruction
Targeted mixture	h + α[D(z) − h] + αWδz	Reconstruction and edit together
Residual preserving edit	h + αWδz	Proposed edit without reconstruction replacement
Matched random edit	Comparable edit to random latents	Specificity of feature selection

The residual-preserving condition is a proposed comparator, and not a demonstrated improvement. It could also create unrealistic internal states. Random edits should be matched for count and magnitude, and preferably for latent activation and decoder-column norm, to avoid an artificially weak control.

5.5 Limits of the interpretation claim

The paper's supplementary discusses why varying active sets may obstruct a single global attribute direction [2, Supplement p. 5, Equations S1–S6]. This is not a general guarantee that the attribute becomes linearly unrecoverable. An original counterexample is the piecewise-linear map F(x₁,x₂) = (x₁, ReLU(x₂)), where its active set changes, yet sign(x₁) remains recoverable by the fixed classifier w = (1,0). Representation complexity and successful information removal are distinct claims.

A medical audit should therefore always use held-out linear and nonlinear probes, multiple seeds, and task-level tests. Even when a failed probe demonstrates only the limits of that probe and dataset, it does not prove/confirm that sensitive information has disappeared.

5.6 Transfer to medical fairness

Proposed interpretation. The useful transfer is considered as targeted and auditable intervention. The objective should not be indiscriminate deletion of age, sex-related anatomy, or other information that may be relevant to a defined clinical task. It is important to first distinguish justified clinical evidence from nuisance correlations, such as acquisition marks or site-specific artifacts. For example, a scanner marker may correlate with disease prevalence in the training cohort. An intervention is useful only if it reduces reliance on that shortcut, while retaining actual pathology evidence.

Adopt authorized, appropriately defined metadata for subgroup evaluation, and never to infer race or gender from radiographs to fill missing labels. Define the role of each variable with clinical collaborators. Report all relevant metrics such as per-condition sensitivity, specificity, false-negative rates, calibration, and abstention coverage, with group sample sizes and uncertainty intervals.

The strongest research extension would be clinically constrained sparse interventions, where the aim is to minimize an identified nuisance dependence while imposing empirical preservation criteria for important findings. These criteria must be assessed on untouched data and cannot be described as universal safety guarantees.

6 Review of radiologist-guided causal concept bottlenecks

6.1 Central contribution and scope

XpertCausal models how diseases generate radiographic findings, followed by using Bayesian inference to estimate disease probabilities from image-predicted concepts. A radiologist specifies allowable disease-to-finding relationships, while learning refines their strengths. Overall, this process makes the reasoning layer compact and inspectable, where it is essentially an InceptionV3-based classification system with concept explanations, rather than a generative VLM [3, Sections 2–3].

The central distinction is between two directions. The assumed generative relationship is disease → finding, where the diagnostic inference runs in the opposite direction, namely, finding → posterior disease probability. Reversing the direction of inference does not reverse the underlying disease model.

6.2 Dataset and supervision

The paper uses 40,083 MIMIC-CXR image-report pairs after selecting posterior-anterior images, keeping one image per study, and undersampling the majority "No Relevant Finding" category with One-Sided Selection. It reports an 80/10/10 training, validation, and test split. Both concepts and target labels are extracted from "Findings and Impression" text using radiologist-curated phrase lists and negation handling [3, Section 2, PDF pp. 2–3].

The six targets are mainly: (a) No Relevant Finding, (b) Suspicious Malignancy, (c) Pneumonia, (d) Pleural Effusion, (e) Cardiac Failure, and (f) Pneumothorax. No Relevant Finding means absence of the other five targets, and it is not a guarantee of a completely normal radiograph. Suspicious Malignancy is a suspicion category, and not a confirmed cancer diagnosis.

The 11 concepts are mainly: (a) Unremarkable, (b) Mass, (c) Nodular Disease, (d) Irregular Hilum/Mediastinum, (e) Pneumonitis, (f) Consolidation, (g) Opacities, (h) Infection, (i) Pleural Disease, (j) Enlarged Heart, and (k) Absent Lung Markings. Note that these are the authors' chosen ontology, and not a universal or exhaustive description of chest radiology.

The expert association matrix assigns strong, weak, or absent links. Considering the following examples:

• Consolidation has a strong link with Pneumonia
• Enlarged Heart has a strong link with Cardiac Failure
• Opacities has weak links with several targets

Categories 2 and 1 encode association strength classes, and not clinical probabilities [3, Table 1].

6.3 Image to concept prediction

All three concept-based comparators share an ImageNet-pretrained InceptionV3 concept predictor. It receives normalized 299 × 299 images and learns multi-label concepts with class-weighted binary focal loss. Reported settings include SGD, learning rate 0.0005, batch size 16, and up to 50 epochs with validation-based early stopping. The study reports use of a GTX 1060 with 6GB memory [3, Section 3.1]. Sharing the predictor makes the downstream comparison easier to interpret. However, it is worth noting that the reported improvements are not simply due to giving one reasoning method a stronger image encoder. The downstream models receive predicted probabilities, rather than ground-truth concepts.

6.4 Noisy OR and Bayesian inference

For pathology configuration y, the probability of concept k is modelled as follows:

p_k(y) = P(c_k = 1 | y)
       = 1 − (1 − λ_k) × ∏ over active diseases j (1 − η_jk)

where η_jk is an association strength and λ_k is a leak probability for findings which cannot be explained by the modeled diseases. An expert-excluded link is constrained to zero. The formulation takes the probability that every active cause fails to produce the finding, then subtracts it from one [3, Equation 1, PDF p. 4].

The inference stage combines the concept likelihood with a disease prior as follows:

P(y | c) = P(c | y) P(y) / Σ over y′ [P(c | y′) P(y′)]

The prior is estimated from training data, while the small target space makes the exact configuration enumeration feasible. Explanations rank supporting concepts by log-likelihood-ratio contributions [3, Equation 2 and Section 3.3]. However, several implementation details need further confirmation. For example, the exact likelihood for soft concept probabilities, smoothing of configuration priors, the full contribution-score formula, and enforcement of No Relevant Finding exclusivity. These details are not fully specified in the main paper. A future implementation should therefore document them rather than assuming a particular probabilistic construction.

6.5 Comparators and reported results

XpertXAI uses a decision tree over concept predictions. Causal Learned uses the same noisy-OR and Bayesian structure as that of XpertCausal, but permits all disease-concept links to be learned. A direct InceptionV3 classifier supplies an image-to-label baseline as follows:

Model	Macro AUROC	Macro F1	ECE
XpertCausal	0.7968	0.7109	0.0079
Causal Learned	0.7631	0.6846	0.0102
XpertXAI	0.7361	0.6802	0.0249
Direct InceptionV3	0.6833	0.6120	0.0377

Source: [3, Table 2, PDF p. 5]. Higher AUROC and F1 are better, while lower expected calibration error is better. Comparing against XpertXAI, the absolute improvements are 0.0607 AUROC and 0.0307 F1, with ECE lower by 0.0170. When compared against that of Causal Learned, AUROC improves by 0.0337. These are calculations extracted from the actual table in the paper, and not significance claims. Confidence intervals and repeated-run variation are also not supplied with these results.

The shared concept predictor has macro AUROC 0.8012 and accuracy 0.8139. Its weaker concepts include Consolidation, with AUROC 0.7246, and Irregular Hilum/Mediastinum, with AUROC 0.7300. The latter nevertheless has accuracy 0.9012, hence underlining why accuracy alone can be misleading for imbalanced findings [3, Table 3, PDF p. 6].

The explanation evaluation measures the mean fraction of report-derived ground-truth concepts which are captured by the top-K explanation concepts. XpertCausal performs best at the lower K values, while the unconstrained causal method slightly exceeds it at K = 5. Exact plotted values are not tabulated, thus they are not estimated here [3, Figure 3].

The graph comparison reports the following mean weights.

Model	Strong links	Weak links	Expert-excluded links
XpertCausal	0.7268	0.0953	0.0000
Causal Learned	0.5783	0.3082	0.0920

Source: [3, Table 4, PDF p. 6]. The unconstrained model also learns links from pathology targets to Unremarkable. This suggests a practical benefit from structural restrictions, while the zero excluded-link weight in XpertCausal is enforced by construction.

6.6 Strengths and limits of the evidence

The strongest design feature is the matched comparison between expert-constrained and unconstrained probabilistic reasoning over the same predicted concepts. Evaluation also goes beyond label accuracy to include calibration and explanatory agreement. The resulting model is adeqeuately small to make a first reproduction approachable.

The authors acknowledge a limited ontology, reliance on one radiologist for concepts and graph design, a need for external validation, and a need for formal clinician evaluation [3, Section 5]. The following critical issues are also important for consideration.

Patient separation. One image per study does not guarantee patient-disjoint splits because one patient may have multiple studies. Verify patient identifiers in the actual split before interpreting generalization.

Coupled labels. Concepts and diagnoses come from the same report-extraction process. Their agreement may partly reflect shared annotation rules or errors. Independently adjudicated image-level labels are needed to assess the full visual reasoning claim.

Causal scope. A plausible disease-to-finding graph is a modeling assumption. The reported comparisons do not establish interventionally identified causal effects or valid patient counterfactuals. Conditional independence assumptions and a limited disease vocabulary may fail for complex presentations.

Explanation baseline. XpertXAI explanations are ranked by concept confidence, while the causal models use contribution scores. This is not a fully matched attribution comparison. A stronger experiment would evaluate decision-tree-specific explanations alongside the probabilistic contributions.

Graph validation. Agreement with the matrix used to constrain training is an architectural integrity check. It is not independent proof that the graph is correct. Obtain expert review independent of graph construction and test rare or atypical presentations.

Calibration transport. Undersampling changes disease prevalence and priors. A small aggregate ECE on this cohort does not imply calibrated disease probabilities in a new hospital. Test class-wise calibration on a natural-prevalence cohort and under site shift.

6.7 Proposed role in a medical VLM

A defensible adaptation is to use a medical visual model to estimate concepts, pass them through an explicit clinical reasoning component, and allow the language model to verbalize only supported observations and qualified hypotheses. This would introduce a new interface between visual evidence and language generation that has not been demonstrated by XpertCausal before. Additionally, the following components can be of interest:

• Compare oracle concepts, predicted concepts, learned graphs, expert graphs, and no-graph baselines separately
• Simulate both correct and mistaken clinician edits
• Measure whether a correction propagates appropriately without introducing new unsupported claims
• Preserve an “unexplained or out of scope” route such that a closed concept vocabulary does not force every image into one of the modeled diagnoses

7 Review of MedGemma

7.1 Family and version distinctions

MedGemma is a family of medical adaptations of Gemma 3. The original report mainly studies a 4B multimodal model and a 27B text-only model, with preliminary multimodal 27B work in its appendix. MedGemma 1.5 updates the 4B multimodal model, while it is not the name of a new 27B model. The existing official documentation also distinguishes MedGemma 1 variants from 1.5 [4, Sections 2–3 and Appendix F; 6–7].

MedSigLIP is a medical image-text encoder rather than a text-generating assistant. It is a useful comparator when the task is classification, retrieval, or prediction of clinical concepts. A generative VLM is most justified when the research question includes language output [4, Section 6; 6].

Component	Function	Proposed initial use
MedGemma 1.5 4B	Medical image and text input with text output	Structured findings and draft explanations
MedGemma 1 4B	Earlier multimodal baseline	Version and regression comparison
MedGemma 1 27B multimodal	Larger medical VLM	Optional capacity comparison
MedSigLIP	Medical visual and text embeddings	Simpler concept and classification baseline

The above table's initial uses are simply recommendations, and not performance guarantees.

7.2 Architecture and original training

MedGemma follows Gemma 3 with a medically adapted SigLIP-derived vision encoder and a language decoder. The original report describes a roughly 400M-parameter visual encoder, 896 × 896 RGB inputs, and interleaved image/text processing. The model card specifies 256 visual tokens per image, a 128K context capacity, and up to 8,192 output tokens. The separately released MedSigLIP uses 448 × 448 inputs, but that resolution should not be substituted silently into a MedGemma preprocessing pipeline [4, Section 2.2; 6].

The original training sequence has three main stages. First, the vision encoder is adapted with more than 33 million medical image-text pairs, heavily dominated by pathology patches. Medical examples have a 2% mixture weight alongside the original SigLIP data, while the raw dataset size and training mixture weight are different quantities. Second, multimodal continued pretraining uses medical image data at 10% mixture weight and replays general data. Third, post-training adds medical-text distillation and medical multimodal reinforcement learning [4, Section 2].

Original CT and MRI training adopted selected two-dimensional slices, rather than full volumetric studies. Some slices were selected for abnormalities which are referenced in the reports. Hence, the performance on such inputs should not be assumed to represent the search on an unselected three-dimensional clinical examination [4, Section 2 and Table 1].

7.3 Original benchmark evidence

Evaluation	Gemma 3 4B	MedGemma 1 4B	Metric
MedQA four options	50.7	64.4	Accuracy
MIMIC adjudicated subset	81.2	88.9	Five-condition macro F1
MIMIC MAIRA split	26.7	40.5	Five-condition macro F1
CheXpert	32.6	48.1	Five-condition macro F1
CXR14	32.0	50.1	Three-condition macro F1
SLAKE	40.2	72.3	Token F1
VQA-RAD	33.6	49.9	Token F1

Source: [4, Tables 3, 7 and 9]. Scores are on a 0–100 scale. The revised VQA-RAD split addresses image overlap between dataset training and test sets, and it does not establish absence of foundation-model pretraining contamination. Each row compares models on the same reported protocol and th rows should not be treated as one common scale of clinical competence.

The two MIMIC scores are particularly instructive. The adjudicated test uses radiologist-reviewed labels and excludes missing or uncertain labels. The MAIRA-style evaluation uses different cases and treats missing or uncertain labels as negative. The gap therefore cannot be described simply as random model instability or an unexplained site effect [4, Section 3.3; 5, Section 3.1].

Task adaptation can generate large gains. The original report gives pneumothorax F1 of 59.7 before adaptation and 71.5 after it, and histopathology patch weighted F1 of 32.8 and 94.5 respectively. These are task-specific fine-tuned results, and do not represent performance of an unchanged foundation model [4, Table 13].

For chest report generation, the report separately gives RadGraph F1 29.5 for the pretrained 4B checkpoint and 30.3 for task-specific adaptation of the instruction-tuned 4B checkpoint. These are not one uninterrupted before-and-after sequence. A human review used 306 cases and one cardiothoracic radiologist who was not blinded to report origin. Note that its favorable retrospective ratings do not establish prospective patient benefit or a blinded multiclinician outcome [4, Sections 3.5 and 5.1 and Tables 10 and 13].

7.4 What changes in version 1.5

The 1.5 report keeps the MedSigLIP encoder frozen and further trains the language decoder using additional data, domain-specific teachers, distillation, and reinforcement learning. The added/augmented data include 605,732 chest radiograph examples, 282,963 CT studies, 167,674 MRI studies, and 335,825 whole-slide image/report pairs [5, Table 1 and Section 2].

Its volumetric support leverages sequences of two-dimensional slices, and not a new native three-dimensional visual encoder. The preprocessing limits a query to 85 axial slices, sampling equidistantly if necessary. CT intensities are mapped into three specified windows, while MRI uses per-volume min-max normalization. Consequently, this creates potential coverage and preprocessing failure modes which require testing for small or subtle findings [5, Section 2.3.1].

Whole-slide processing uses tissue detection, a selected magnification, and up to 126 patches. The selected order is retained, but sampling still limits what the model can see. As such, the whole-slide output quality therefore depends on whether diagnostically useful tissue is included [5, Section 2.3.2].

The expanded evaluations include paired current/prior radiographs for progression, anatomical bounding boxes, and medical document extraction. Anatomical localization should not be relabeled as lesion segmentation or proof that the generated diagnosis uses the localized region [5, Section 3.2].

7.5 Results and regressions

Evaluation	MedGemma 1 4B	MedGemma 1.5 4B	Metric
Internal CT	58.2	61.1	Macro accuracy
Internal MRI	51.3	64.7	Macro accuracy
CT-RATE	23.5	26.9	Macro F1
Whole-slide reports	2.2	49.4	ROUGE-L
Chest ImaGenome anatomy	3.1	38.0	Mean IoU
MS-CXR-T progression	61.1	65.7	Macro accuracy
MIMIC report generation	21.9	27.2	RadGraph F1
SLAKE	72.3	59.8	Token F1

Source: [5, Tables 3–5]. Scores use a 0–100 scale. Internal CT and MRI evaluations use balanced samples per condition and include patient-history text; they are not image-only estimates of real-world positive predictive value. CT-RATE is an external-distribution evaluation and its macro F1 remains modest despite improvement.

The MS-CXR-T task further inquires whether the five cardiopulmonary conditions improve, remain stable, or worsen between paired studies. It supports a specific temporal comparison capability, not unrestricted longitudinal clinical reasoning. Likewise, an anatomical IoU of 38.0 should be interpreted as a particular box-overlap score rather than 38% diagnostic correctness.

SLAKE performance regresses, as do some other legacy tasks. Therefore, the important lesson is to retain version-specific baselines and select a checkpoint for the actual intended use, rather than assuming the most recent model is uniformly better.

7.6 Source discrepancies that affect interpretation

Issue	Verified evidence	Treatment in this report
MRI improvement	Abstract says 11 points & Table 4 gives 51.3 to 64.7	Use the table-derived 13.4 percentage points
Whole-slide metric	Abstract refers to macro F1 & methods and Table 4 specify ROUGE-L	Use ROUGE-L and a 47.2-point increase
Combined MedXpertQA	Versioned paper lists 26.4 & model card lists 20.9	Do not merge; reason for the discrepancy is unresolved
Multiple-image limitation	Card retains a broad unevaluated claim & report evaluates slice sequences and paired CXRs	Use task-specific evidence; do not infer general multi-image reliability
Report-generation baseline	Original pretrained score is 29.5 & 1.5 comparison uses a 21.9 baseline	Preserve checkpoint and protocol distinctions

Sources: [4–6]. Small additional differences exist, including CT-RATE 26.9 versus 27.0 and SLAKE 59.8 versus 59.7. A version-pinned experiment should therefore record the exact checkpoint, processor, prompt, decoding configuration, and evaluation script rather than treating all reported numbers as interchangeable.

7.7 Safety and research implications

The official model card positions MedGemma as a development starting point that requires careful adaptation and validation. It describes outputs as preliminary and excludes direct use of unvalidated outputs for clinical diagnosis, management, or treatment decisions [6, Use and limitations]. This is directly relevant to the intended doctor and patient use case, under the current AI context.

In addition, content-policy safety evaluations and clinical factual safety answer different questions. A response can avoid prohibited content while still missing a pneumothorax, reversing laterality, or converting uncertainty into reassurance. The proposed research should thus measure those clinical error categories directly.

For an initial study, MedGemma 1.5 4B is a reasonable candidate since the proposed research project requires both medical images and language outputs. The first experiment in the below Section 8 deliberately uses this model alone, followed by retaining the original 4B and a simpler MedSigLIP concept/classification baseline for the later extensions in Sections 9 and 10. A larger model can be added if the research question concerns with capacity, rather than as an automatic substitute for careful evaluation.

8 Updated proposal for domain constrained chest radiograph reporting

8.1 Current scope and research question

For the 1st research phase, the objective narrows the project to a frozen MedGemma 1.5 4B model on the author's personal Lenovo ThinkStation PGX that has built-in Blackwell GPU architecture coupled with 128GB unified memory. The proposed contribution is a controlled evaluation of explicit finding records and checked language generation, including their latency and energy costs. It does not require training an SAE, a clinical graph, or an adapter in the first experiment. Those remain subsequent research options motivated by the literature. The MedGemma 1 4B and MedSigLIP comparisons recommended in Section 7.7 are likewise deferred.

Current question: Can a structured finding record and a checked record-to-text stage reduce clinically meaningful unsupported assertions while preserving finding sensitivity and uncertainty, at a measured local inference cost? The initial output is a restricted findings draft for expert review. Patient-facing explanations, treatment recommendations and complete autonomous radiology reporting are outside this experiment.

The vocabulary V5 comprises pleural effusion, pneumothorax, cardiomegaly, consolidation and pulmonary edema. A record contains finding, present/absent/uncertain status, and laterality where clinically meaningful. Missing entries remain missing. They must never become negative findings. Uncertainty in the record is an assertion state, not a calibrated probability. Invalid, conflicting or truncated records trigger an abstention with a logged reason.

Status: The proposal describes planned experiments. No local inference, cohort extraction, reader study, quality gain, latency or energy result has been established by this review. The design below incorporates the methodological corrections identified during rechecking; it supersedes the uploaded proposal's conflicting sampling and attribution claims.

8.2 A five arm design that isolates the checking stage

Use one pinned checkpoint, the same eligible image per study, and fixed decoding within each runtime/precision condition. Before evaluation, freeze two prompt paraphrases, their pairing across arms, output limits and the order of information. A and A* receive images. The structured pathway first produces and saves R from the image. B then generates an unchecked draft from R alone. C starts by checking that exact saved B draft, rather than creating a different initial draft.

Arm	Input and output	Meaning of the comparison
A	Image and generic instruction produce free findings	Baseline
A*	Same image and an instruction naming V5 produce free findings	A versus A* measures the instruction package
B	Image produces R; a separate text-only call maps R to saved unchecked draft T_B	A* versus B measures the structured pipeline, including its extra call
C	Check exactly R and T_B; release T_B if it passes, otherwise one fresh retry released only if it passes, otherwise the validated template; an invalid R causes abstention	B versus C measures the checking, repair and fallback policy
Template	Render the same R using deterministic, validated sentences	Baseline for record fidelity and wording preference

This is a deliberate revision to the uploaded design. Its B jointly generated R and a draft from the image, whereas C generated a fresh text-only draft before checking. That comparison changed modality access, prompting and generation as well as checking. Freezing weights alone did not remove the confound. In the revised design, B versus C identifies the entire checking policy, including repair and fallback, rather than the checker rule engine in isolation. A* versus B still tests a pipeline change, not a unique internal mechanism or causal effect of interpretability.

Charge both B and C for image-to-record creation and initial record-to-text generation. Charge C additionally for checking, retry and fallback. Charge Template for record creation and rendering. Offline reuse can reduce experimental runtime, but must not remove those shared costs from the reported deployment cost of each arm. Report invalid-record coverage separately for B, C and Template and preserve all cases in the failure accounting.

Two properties of this design affect the endpoints. First, A* names V5 but does not prevent the model from describing findings outside it, whereas B, C and Template work from a record that contains only V5 findings. Section 8.6 therefore scores the primary endpoint within V5 scope and reports out-of-scope claims separately. Second, C releases T_B unchanged whenever no violation is detected, so B and C differ only on flagged studies. The B versus C contrast is therefore driven entirely by those studies, and readers need to assess a B draft separately only where it differs from the C output.

8.3 The checker contract

The checker compares atomic claims with R for additions, omissions, polarity, uncertainty and laterality changes. One sentence may contain multiple claims; sentence counts and claim counts must not be interchanged. Define approved synonyms, negation scope, uncertainty scope and finding-specific laterality rules on development data. Cardiomegaly, for example, should not inherit a lung-side requirement simply because the record schema has a laterality field.

A V5-only extractor cannot detect arbitrary assertions outside V5. The initial implementation should therefore require complete clause coverage by a restricted output grammar, rejecting unexplained text as an unknown clause. This is a proposed implementation requirement, not an existing guarantee. If free language remains unrestricted, add an independently validated open-vocabulary check and report its residual misses. The system must not claim to detect every possible unsupported statement.

Global normality statements need explicit treatment. Phrases such as “no acute cardiopulmonary process”, “otherwise normal” or “the lungs are otherwise clear” assert the absence of findings outside V5, which R cannot support. They are also the form of false reassurance that matters most clinically. The grammar should classify them as out-of-scope claims rather than approved filler; approved sentences may state only V5 findings and their recorded status. This has a cost. Natural drafts often contain such phrases, so a strict grammar may flag most of them and C may converge toward Template. Report the flag, retry and template-fallback rates so that this behavior is visible.

On any detected violation, make one fresh single-turn record-to-text call with the record and flags. Release the retry only if it passes. Otherwise use the validated template; an invalid record causes abstention. Merely attempting the retry is never a release condition. A faithful template can preserve an incorrect record, so it establishes no image-grounding guarantee.

Validate flag precision and recall on approximately 100 manually annotated development drafts, with per-flag denominators and deliberately constructed difficult cases. Report natural-draft performance separately from synthetic stress cases. Use independent evaluation annotations and readers for final quality claims, not the checker itself. Log the first draft, flags, retry, selected output and fallback status within the protected environment.

Illustrative calculation. Natural violations of a given type may be few among about 100 drafts. If all 10 natural violations of one flag type are detected, the exact 95% lower confidence bound for recall is still 69%; for 20 of 20, it is 83%. Per-flag recall claims therefore rest mainly on the constructed difficult cases, which should be reported with their own denominators.

8.4 Cohort feasibility and external evaluation

MIMIC-CXR-JPG v2.1.0 supplies JPG images, labels and split metadata; MIMIC-CXR v2.1.0 supplies the reports and DICOM source images [8, 10]. These are distinct resources and require separate citations. Begin with about 200 official training-split studies for development and a planned 1,000 eligible official test studies for the automated comparison. These are planning targets, conditional on access, available positives, view filtering and precision analysis. The published JPG split contains 3,269 test studies from 293 patients, so 1,000 studies are not 1,000 independent patients [23, Table 3]. The official test split is also not a natural-prevalence sample. It contains all studies of patients with at least one manually reviewed report, selected by stratified sampling to cover pathologies; 89.1% of its studies have a finding, compared with 76.5% in the training split [23, Section 4 and Table 3]. The split averages about 11 studies per patient, so a 1,000-study sample drawn from it would still average more than three studies per patient. Verify counts against the exact downloaded release and final inclusion rules.

MIMIC-CXR-JPG v2.1.0 also includes single-radiologist annotations of test-set reports, with an explicit uncertain value [8]. These remain report-derived labels rather than image adjudication, but they are a stronger reference for the sensitivity safeguard and for uncertainty preservation than automatic labeler output. Confirm how many test studies the file covers before relying on it.

The proposed VinDr sample cannot meet 150 positives for every finding. Its publication reports the following counts for the entire 3,000-image test set [11, Table 2]. A 1,000-image subset cannot contain more positive images than this parent set.

Finding	Positives in full VinDr test set	Consequence
Pleural effusion	111	Below the proposed 150-positive target
Pneumothorax	18	Very limited sensitivity precision
Cardiomegaly	309	Feasible positive enrichment, subject to the joint sampling design
Consolidation	96	Below the proposed target
Edema	0	Sensitivity cannot be estimated

All five concepts occur in the full dataset's local label vocabulary; vocabulary availability does not imply enough positive test examples. Use the full official VinDr test set if feasible, or prespecify an exploratory subset with all eligible rare positives and known inclusion probabilities. Report actual denominators and intervals for each finding. Mark edema sensitivity as not estimable. Its false-positive rate can still be measured on negative cases. VinDr offers finding labels and boxes, not free-text reports or reference uncertainty states, so report-generation metrics and uncertainty-preservation claims require other evidence.

VinDr images are distributed as DICOM and comprise adult PA radiographs, whereas MIMIC-CXR-JPG images were produced by a documented conversion with normalization, inversion where required and histogram equalization [8, 11]. Convert VinDr with a documented pipeline that matches this procedure, or report preprocessing as a separate factor; otherwise site shift and preprocessing differences are confounded. VinDr is distributed under the PhysioNet Credentialed Health Data License, so the local-processing protocol in Section 8.8 applies to it as well.

Keep development patients separate from evaluation patients and respect official splits. Where external patient identifiers are unavailable, describe the independence limitation instead of claiming verified patient clustering. Record the AP/PA mix and number of images per study. Determine image-grounded omissions from evidence visible in the supplied frontal image; statements dependent on unavailable lateral views or priors require separate out-of-scope annotations. Always count unsupported generated additions.

Enrichment changes the target population, and this applies to the official MIMIC test split as well as to any enriched VinDr subset. Report results on the evaluated cohort and, only with known inclusion probabilities and a defined target population, sampling-weighted estimates. A single prevalence multiplier does not correct a multilabel enriched sample. Do not pool MIMIC report agreement, VinDr finding accuracy and radiologist judgments into one quality score.

8.5 Training overlap and evaluation provenance

MedGemma documentation lists MIMIC-CXR and Chest ImaGenome among development sources [12], while the MedGemma 1.5 report explicitly states that its evaluations used data sources or splits held out from training [5, Section 3]. Therefore, corpus-level overlap does not establish that the proposed test studies were used for training, and it does not prove that absolute scores are optimistic. Audit the proposed cohort against available training and evaluation provenance and state what remains unknown. The 1.5 report's chest report-generation evaluation, for example, used a previously defined 912-image MIMIC-CXR test set [5, Section 3.1]; this identifies one evaluated subset, not the full training partition.

Using one frozen model supports paired comparisons but does not guarantee that any overlap bias cancels between prompts or pipeline arms. Likewise, VinDr's absence from a card's dataset list does not prove that it was unseen during all model development. Describe it as evaluation on another data source, with pretraining provenance unresolved where it cannot be verified. An independently authorized institutional cohort would strengthen later generalization evidence. The model card itself notes the contamination risk and advises validation on data that are not publicly available to non-institutional researchers [12].

8.6 Endpoints and statistical interpretation

The clinical primary endpoint should be the reader-adjudicated count of clinically meaningful image-unsupported atomic claims within V5 scope per study, with C versus A* as the prespecified primary contrast. Restricting the count to V5 scope prevents C from appearing safer merely because it cannot describe findings outside V5. Report out-of-scope claims for every arm as a separate secondary measure, with global normality statements as their own category, together with claims per study and the unsupported fraction of claims. Two radiologists independently review the supplied images and model outputs, blinded to arm with randomized presentation, followed by adjudication. Use a fixed rubric for visual support, clinical consequence and important omissions. The proposed 150-study reader subset is a starting design, not a demonstrated sample size sufficient for the intended claims.

If the reader subset is drawn from the MIMIC test split, select at most one study per patient using a prespecified rule. The split's 293 patients can supply the proposed 150 studies, which removes patient clustering from the primary clinical endpoint. Each unique output text needs rating only once: where C released T_B unchanged, the B rating is inherited from C, so B requires additional reading only for flagged studies.

Endpoint	Measurement	Interpretation
Clinical primary	Adjudicated meaningful unsupported V5-scope claims on the reader subset	Clinical conclusion applies to the sampled reader cohort; report selection and denominators
Out-of-scope assertions	Claims outside V5 for every arm, with global normality statements counted separately	Shows what the restricted pathway removes and whether free generation adds unsupported reassurance
Sensitivity safeguard	Paired per-finding sensitivity difference of C versus A*, with confidence intervals and every discordant positive listed	Descriptive in the first study; reference labels and uncertain-label policy stated separately for each dataset
Reference agreement	CheXbert labels, RadGraph overlap and/or GREEN on MIMIC	Automated proxies; not independent proof of image truth or clinical severity
Record fidelity	Independent claim annotations against R	Separates wording errors from errors already present in R
Omissions and coverage	Visible important findings missed; invalid outputs, abstentions and fallbacks	Prevents shorter or more selective outputs appearing safer without cost
Wording utility	Blinded preference between C and Template on matched records	Requires a defined reader question and reporting of ties

CheXbert maps report text to finding labels [14]. RadGraph extracts entities and relations, and its F1 reflects precision and recall together [13]. GREEN evaluates report errors against a reference [15]. None alone determines whether a claim is supported by the actual image. RadGraph F1 and GREEN must not be labeled “full-report recall.” Report precision and recall separately where available, output length, and clinician-rated omissions. A V5-restricted draft cannot be advertised as a complete report; full-reference metrics should be contextual, with in-scope and out-of-scope findings separated. RadFact is another report evaluation reference, not a substitute for image review [16].

Use paired, patient-cluster-aware confidence intervals for MIMIC. Account for enrichment and repeated paraphrases; do not count two prompts as two independent patients. Predefine one main prompt/contrast or an aggregation rule, with sensitivity results for the other prompt. Treat C versus B and B versus A* as secondary or use a prespecified multiplicity procedure. Plan reader effort and effective sample size from development estimates of paired disagreement, clustering, positive counts and the selected margin, before viewing final test outcomes.

For sensitivity, a noninferiority conclusion requires the lower confidence bound of sensitivity_C minus sensitivity_A* to exceed −Δ for the required findings under the prespecified rule. A nonsignificant difference or overlapping confidence intervals does not establish equivalence. Quantized-variant equivalence needs its own clinically justified margin and analysis.

Illustrative calculation. Assume no true difference, 5–10% of positive studies discordant between C and A*, and no patient clustering. The paired 95% confidence interval for the sensitivity difference then has a half-width of about 4–6 percentage points at VinDr's 96–111 consolidation and effusion positives, and about 10–15 points at its 18 pneumothorax positives. Demonstrating a 2-percentage-point margin with 80% power would require roughly 980–1,960 positives per finding, and about 390 even at 2% discordance. Clustering would increase these figures. The 2-point margin suggested in the proposal is therefore unlikely to be reachable on the planned cohorts, except possibly for the most frequent MIMIC findings at very low discordance.

For the first study, prespecify the sensitivity safeguard as descriptive: paired differences with confidence intervals and a listing of every discordant positive case. Reserve a formal noninferiority claim for findings with enough positives and a wider margin that clinicians agree is acceptable. Where neither condition holds, state in advance that the comparison will be reported as inconclusive.

Count abstentions and invalid outputs in the all-study analysis and report conditional performance among answered cases separately. An empty or rejected output has zero emitted unsupported claims but can miss every important finding, so unsupported-claim reduction alone cannot establish success. Sensitivity and coverage are mandatory safeguards. The same caution applies to an incomplete R that all downstream arms preserve faithfully.

8.7 PGX quality and compute experiment

The advertised PGX configuration includes NVIDIA GB10, a 20-core Arm CPU, Blackwell GPU, 128 GB LPDDR5x unified memory and 273 GB/s bandwidth. NVIDIA lists GB10 compute capability 12.1 [17–18]. The pilot must record the actual OS, driver, toolkit, framework runtime, attention backend and container digest; advertised software is not a measurement of the installed machine.

Pin the model and processor revisions, image conversion, normalization, crop settings and generation configuration. The documented base MedGemma image path uses 896 × 896 inputs and 256 visual tokens per processed image [6]. Extra crops can change token counts. Set greedy decoding explicitly; the official repository records its default change on 23 January 2026 [12]. Fresh single-turn prompts are the conservative initial protocol because broad multi-turn capability has not been established by the model card.

Before running the arms, replicate one published MedGemma 1.5 result on the PGX using the report's protocol: temperature 0 and per-condition yes/no prompts for chest radiograph classification [5, Section 3 and Appendix B]. Agreement within the expected variation checks preprocessing, runtime and decoding before any method comparison. Where the published evaluation subset cannot be reconstructed, describe the run as a protocol check on the available split rather than a reproduction. Per-condition prompting is also an alternative way to construct R: five short binary calls instead of one structured call. It offers a natural latency and quality comparison for the compute study, provided the uncertain state is handled explicitly rather than forced into yes or no.

Use BF16 as the reference. FP8 and NVFP4 are optional quantized variants derived from the same checkpoint, subject to demonstrated MedGemma, ARM64, runtime and GB10-kernel compatibility. NVIDIA's general Spark quantization workflow does not validate this exact VLM [19]. Record quantized modules, calibration data from development only, numerical formats and excluded layers. Quantization changes numerical weights; “same frozen weights” is inaccurate. Match serving stacks where possible, otherwise separate runtime and precision effects.

Benchmark batch sizes 1, 2, 4 and 8 where feasible, with at least three independently started runs per configuration. Separate model loading, warm-up and steady-state operation. Use GPU synchronization at isolated-latency interval boundaries; avoid serializing concurrent throughput requests with per-request synchronization. Report end-to-end median/p95 latency, completed studies per minute, and repeated-run variability. Include all retries, failures and fallback work in total costs, and report success coverage alongside cost per completed study.

Measure wall-plug energy with the meter boundary, idle baseline, warm-up treatment and sampling resolution stated. Use the supplied 240 W USB-C power adapter, which NVIDIA advises for full performance, and place the meter between the wall outlet and that adapter [17, 18]. Report framework allocation/reservation, process memory and total system memory separately, including swap activity. Approximate weight-file sizes do not include caches, activations, temporary buffers and operating-system overhead. Concurrent residency of a generator, local judge and checker is a feasibility test. On this platform, nvidia-smi's overall framebuffer-memory field is unsupported, while per-process information may be available [18]. NVIDIA also notes that cudaMemGetInfo can under-report allocatable memory on this unified-memory platform because it does not count memory reclaimable through swap, and suggests estimates based on /proc/meminfo [18]. Record both and state which one supports any feasibility claim.

Prefix caching is conditional on runtime support and an exactly matching encoded prefix. Benchmark it within each arm, with fixed text before variable case content where the model interface permits, and verify cache hits. Shared vocabulary alone does not ensure shared cache entries; image placement and prompt order matter. Measure prefill and total latency separately [20]. Isolate the cache between arms. If arms run back-to-back on the same image, a later arm can reuse an image prefix cached by an earlier arm and appear faster. Reset the cache or use a separate server instance per arm, randomize arm order, and report cold and warm measurements separately.

For a positive measured reduction in clinically defined errors, the incremental energy ratio is (mean J_C − mean J_B) / (mean errors_B − mean errors_C), with an analogous time ratio. This is an original proposed calculation, not a reported result. Use the same cohort, workload and error definition. When the denominator is zero, negative or too uncertain, show the separate cost and error differences rather than a misleading “joules per avoided error” headline.

8.8 Research conduct and GTC submission gate

The proposed protocol keeps credentialed images, reports and case-level derivatives on the workstation, with authorized readers and local evaluation. PhysioNet's published guidance states that the credentialed data use agreement prohibits sharing data access with third parties, including through APIs or online platforms. It requires zero data retention by any third-party service, strongly recommends locally deployed LLMs, and advises against any service whose data handling cannot be fully verified [21]. This protocol therefore uses local deployment only, with no external model service for data-bearing inputs. VinDr is distributed under the same credentialed license, so the same handling applies [11]. Release code, configurations, synthetic examples and non-sensitive aggregate findings. Keep data-bearing logs and case-level outputs under the applicable access controls.

NVIDIA's GTC 2027 poster deadline is 10 November 2026 at 5 p.m. Pacific Time, equivalent to 11 November 2026 at 9 a.m. Singapore time; US daylight saving time ends on 1 November, so Pacific Standard Time applies. The call evaluates proven technical outcomes, a clear benchmarking methodology and the application of NVIDIA technology, and does not consider conceptual submissions [22]. The PGX quality and compute measurements in Section 8.7 are therefore central to the submission rather than a secondary experiment. The present proposal is a research plan targeting that call. Replace placeholders only with completed, clearly scoped measurements. A development pilot may support an explicitly preliminary result, but cannot be described as a preregistered held-out evaluation. Submission-field limits still require checking in the live portal. If the poster is accepted, one author must attend the poster reception in San Jose between 14 and 18 March 2027.

The first milestones are therefore access and cohort feasibility, the local protocol replication check, the corrected B/C implementation, independent checker validation, and a BF16 pilot. Freeze the clinical endpoints and analysis before final testing.

As of 10 October 2026, about 32 days remain before the deadline and dataset access has not yet been obtained. Plan on the assumption that the reader study will not finish before submission. A defensible scoped poster would report BF16 pipeline cost for each arm, automated reference agreement on the evaluated cohort, and checker validation on development data, with the reader study described as in progress. Expand the scope only as completed measurements allow, and state the remaining work. No planned improvement should appear as an observed result.

9 Longer term research extensions

9.1 A division of responsibilities

The following wider architecture is retained as a later research roadmap. The current frozen-model reporting experiment is specified in Section 8. The literature motivates evaluating visual evidence, structured reasoning and communication independently; adding these modules is not a requirement for the initial poster study.

Research component	Proposed contribution	Principal failure to test
Medical visual foundation	Extract useful image representations	Missed or mislocalized findings
Clinically reviewed concepts	Make intermediate evidence inspectable	Incorrect or incomplete concept vocabulary
Sparse feature audit	Investigate shortcuts and controllability	Editing unrelated clinical information
Constrained reasoning	Restrict unsupported inference paths	Incorrect hard constraints or wrong priors
Controlled generation	Express supported findings clearly	Added claims, omitted findings, lost uncertainty
Clinician review	Resolve ambiguity and approve communication	Automation bias and misleading explanations

9.2 Proposed architecture

The following architecture is a research design, not a system reported by the papers. Begin with the main evidence path; add sparse interventions only after the baseline is understood.

flowchart TD
    A["Medical image and approved context"] --> B["Medical visual model"]
    B --> C["Concepts with location and uncertainty"]
    B -.-> H["Sparse feature audit"]
    H -. "validated edit" .-> C
    C --> D["Constrained clinical inference"]
    C --> E["Evidence linked draft"]
    D --> E
    E --> F["Clinician review"]
    F --> G["Approved patient explanation"]

The concept record is the central interface. Each item should contain a finding, assertion state, anatomical location, laterality where relevant, supporting image region if available, confidence from a specified calibrated component, and a provenance identifier. A language model's self-reported confidence should not be accepted as a calibrated probability by default.

The reasoning component should distinguish observations from hypotheses. The generation component should preserve that distinction and attach each medical assertion to evidence. A deterministic consistency checker can identify prohibited additions or contradictions, but its effectiveness must itself be evaluated against clinician annotations.

Patient explanations should be derived from clinician-reviewed content in the initial research workflow. They should preserve uncertainty and avoid adding diagnoses, prognoses, or treatment recommendations that are absent from the approved source. This provides a separate, measurable communication task rather than exposing raw image interpretation directly to patients.

9.3 Research questions and falsifiable hypotheses

RQ1 Grounded concepts. Does an explicit concept record reduce unsupported assertions compared with direct MedGemma generation at comparable finding sensitivity? Section 8 now addresses this question for V5 with a frozen model; later work would extend it to learned concepts and wider vocabularies. The hypothesis fails if factual precision improves only by omitting important findings.

RQ2 Selective intervention. Can an intervention correct a specified finding while preserving unrelated findings? Evaluate beneficial target corrections and collateral errors together, using reconstruction-only and random-edit controls.

RQ3 Clinical constraints. Do expert restrictions improve out-of-site report factuality and calibration beyond a learned graph or a simple concept-to-label head? The hypothesis fails if gains are limited to agreement with the same rules used to create labels.

RQ4 Subgroup robustness. Can shortcut reduction improve the worst evaluated group's clinical error without a clinically unacceptable loss elsewhere? Report the full trade-off rather than a single fairness statistic.

RQ5 Communication fidelity. Does simplification for patients retain findings, uncertainty, and the limits of the examination? Test unsupported reassurance and invented advice explicitly.

These are candidate contributions. A separate targeted prior-art search is needed before asserting novelty in a proposal or publication.

10 Evaluation framework for later extensions

10.1 Start with a bounded task

The current first task is the V5 extraction and restricted draft-generation study in Section 8. The following framework applies to later studies that add learned concepts, graphs, interventions or patient explanations. Confirm the precise cohort, view types, and target findings with Dr. Liu and a radiology collaborator. Do not assume that a result on PA images covers portable AP studies, pediatric images, lateral views, CT, or MRI.

MIMIC-CXR-JPG provides a documented radiograph research resource with structured labels, while Chest ImaGenome provides associated anatomy and scene-graph annotations. Both require attention to their access conditions and label provenance [8–9]. Their use here is a proposed data route, not a claim that access has been obtained. Public-data experiments should be complemented by an independent institutional or temporal test cohort if it becomes available and authorized.

Partition by patient before constructing training examples, crops, report sentences, or concept pairs. Keep feature fitting, feature naming/probing, hyperparameter selection, and final testing separate. Record overlap with foundation-model development datasets where known. A held-out split from a familiar public dataset and an independent hospital dataset answer different generalization questions.

10.2 Metrics and interpretation

Dimension	Proposed measures	Necessary qualification
Finding detection	Per-condition sensitivity, specificity, AUROC, AUPRC and F1	Report prevalence, thresholds and denominators
Report factuality	Unsupported claims, important omissions, negation and laterality errors	Use adjudicated clinical categories alongside automated metrics
Spatial grounding	Finding localization and regional agreement	Anatomical boxes alone do not verify disease evidence
Calibration	Brier score, reliability curves, class-wise ECE	Specify bins and evaluate prevalence shift
Abstention	Risk versus coverage and clinician workload	Measure errors among both answered and deferred cases
Intervention	Target correction and collateral error	Include reconstruction and random-edit controls
Fairness	Group-specific errors, calibration and coverage	Show sample sizes and uncertainty intervals
Communication	Preservation of facts and uncertainty, readability	Fluency is not a substitute for fidelity

For a binary finding, the Brier score is the mean squared difference between predicted probability and outcome. It requires actual probabilities from a specified component, not arbitrary numbers generated in prose.

Brier = (1/N) Σ_i (p_i − y_i)²
Coverage(t) = number of accepted cases at threshold t / N
Risk(t) = mean clinically defined loss among accepted cases

An abstention rule can reduce risk on accepted cases simply by rejecting difficult examples. Report its coverage, group distribution, missed-important-finding rate, and review burden. The operationally useful quantity is the joint risk-coverage behavior, not abstention frequency alone.

10.3 Essential ablations

Use a staged comparison so that each experiment has an interpretable purpose. Experiments are numbered E1–E7 to distinguish them from the Section 8 arms; E1 is partly addressed by arms A, A* and B.

Experiment	Main comparison	Question answered
E1	Direct MedGemma versus structured prompt	Is output structure alone sufficient?
E2	MedSigLIP concept head versus VLM-derived concepts	Does generation improve concept extraction?
E3	Predicted versus independently supplied concepts	How much error originates in perception?
E4	No graph, learned graph, expert graph	What is the value of explicit constraints?
E5	SAE reconstruction versus targeted intervention	Is the effect attributable to the selected feature?
E6	Targeted versus matched random intervention	Is feature selection meaningful?
E7	Original versus patient-facing wording	Does simplification alter medical content?

Where methods are trained, match data access and report trainable parameter counts and compute. Use more than one seed for conclusions about learned features. Avoid granting one method stronger labels or extra clinical context without making that difference explicit.

10.4 Clinician review and statistical discipline

Create a blinded, independently reviewed subset covering both common and consequential rare findings. Where feasible, use at least two readers and adjudicate disagreements. Separate whether an explanation is understandable from whether it is correct, complete, and useful for a decision. A convincing wrong explanation is an important failure category.

Use paired comparisons on the same studies and confidence intervals that respect patient-level clustering. Predefine the main endpoint and a clinically agreed noninferiority margin for important finding detection before final testing. Failure to find a statistically significant decline is not proof of preservation. Small subgroup estimates should be reported as uncertain rather than promoted into fairness guarantees.

Retain difficult cases: poor image quality, unusual devices, postoperative changes, ambiguous findings, conflicting context, missing prior images, and examples outside the ontology. Include prompt paraphrases and deliberately irrelevant context to test whether the model changes its clinical claims without new evidence.

11 Illustrative plan for later research extensions

The following illustrative eight-week sequence concerns later concept, graph and sparse-feature extensions. It is not the schedule for the GTC 2027 reporting pilot and is not a commitment on behalf of Dr. Liu or A*STAR. Begin these extensions only after reviewing the initial experiment and confirming time, data and clinician availability.

Stage	Work	Reviewable output
Weeks 1 and 2	Agree intended use, ontology, data splits, baseline checkpoints and error definitions	Reproducible baseline and initial error analysis
Weeks 3 and 4	Compare concept extraction and constrained reasoning baselines	Concept-quality results and graph ablations
Weeks 5 and 6	Fit sparse features and test selective edits with controls	Intervention and collateral-error results
Weeks 7 and 8	Evaluate external or temporal transfer, subgroups and clinician-facing explanations	Evidence report with supported conclusions and remaining failures

Patient-language evaluation should initially operate on reviewed reports. If clinician access or adequate data is delayed, prioritize a reproducible audit benchmark and concept-intervention study over an end-to-end clinical demonstration.

The first meeting should resolve five concrete choices: the imaging modality and cohort, the clinical task, the source of independent labels, the intended doctor/patient workflow, and the principal harm that the research aims to reduce. The useful deliverable is a falsifiable experiment with a clear baseline, not only a multi-module architecture.

12 Overall research assessment

The references support a coherent research direction: expose image-grounded concepts, inspect potentially harmful representation dependencies, constrain reasoning where clinical knowledge is reliable, and verify the language that communicates the result. The critical unanswered question is whether these mechanisms improve clinical factuality and appropriate reliance under realistic distribution shift.

For the internship, prioritize the controlled record-to-text reporting experiment in Section 8. It creates a baseline for later evidence-preserving concept correction and representation interventions. The first success criteria concern clinician-adjudicated unsupported V5-scope claims, finding sensitivity, uncertainty preservation and measured compute cost. Calibration and subgroup robustness become additional explicit objectives when the relevant probabilities, data and sample sizes are available. The reviewed papers supply methods and baselines; the proposed experiments must establish any medical benefit.

13 References and source notes

The original paper summaries and numerical tables were rechecked on 9 October 2026; the additional proposal sources were checked on that date and rechecked on 10 October 2026. The supplied proposal is dated 8 October 2026. Paper summaries are paraphrased. Numerical tables in this report identify their original locations; calculated differences are explicitly described as calculations. The report does not reproduce source figures.

[1] Akshay Kulkarni, Tsui-Wei Weng, Vivek Narayanaswamy, Shusen Liu, Wesam A. Sakla and Kowshik Thopalli. Interpretable and Steerable Concept Bottleneck Sparse Autoencoders. CVPR 2026, pp. 2918–2927. Requested paper. Supplement. Versioned HTML. Detailed reading locators: main Sections 3–6 and Table 1.

[2] Na Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma, Isabelle Augenstein and Hyunjung Shim. Interpretable Debiasing of Vision-Language Models for Social Fairness. CVPR 2026, pp. 39326–39337. Requested paper. Supplement. Versioned HTML. Official implementation. Accepted main Table 1 is on proceedings p. 39330. Accepted supplement Tables S6 and S8 correspond to arXiv Tables 10 and 12. The arXiv article is available under CC BY 4.0.

[3] Amy Rafferty, Rishi Ramaesh and Ajitha Rajan. Radiologist-Guided Causal Concept Bottleneck Models for Chest X-Ray Interpretation. arXiv:2605.07785v3, 21 August 2026. Requested version 3 PDF. Version 3 HTML. Locators use the nine-page PDF: data pp. 2–3, methods pp. 3–5, results pp. 5–6, limitations p. 7. Available under CC0 1.0.

[4] Andrew Sellergren et al. MedGemma Technical Report. arXiv:2507.05201v1, 7 July 2025. Technical report. Main locators: architecture and training Section 2; classification Tables 3 and 7; VQA Table 9; report generation Table 10; adaptation Table 13; MedSigLIP Section 6. Available under CC BY 4.0.

[5] Andrew Sellergren et al. MedGemma 1.5 Technical Report. arXiv:2604.05081v1, 6 April 2026. Technical report. Main locators: training Table 1 and Section 2; preprocessing Section 2.3; results Tables 3–5; evaluation protocols Section 3 and Appendix B. Available under CC BY 4.0.

[6] Google. MedGemma 1.5 model card. Official documentation. Last-updated date shown as 21 April 2026 at review. Used for model specifications, intended use, and comparison with the versioned report. Documentation is licensed under CC BY 4.0 except as otherwise noted; model use is governed separately by HAI-DEF terms.

[7] Google. MedGemma overview and model family documentation. Official overview. MedGemma 1 model card. Consult the linked model cards for exact checkpoint distinctions and current model-use conditions.

[8] Alistair Johnson et al. MIMIC-CXR-JPG chest radiographs with structured labels, version 2.1.0. PhysioNet, 2024. Dataset documentation. Version 2.1.0 was published on 12 March 2024 and adds single-radiologist annotations of test-set reports. Used for the proposed radiograph data route, label/split documentation and the JPG conversion procedure; access has not been obtained for this review.

[9] Chest ImaGenome Dataset, version 1.0.0. PhysioNet. Dataset documentation. Used for the proposed anatomical grounding evaluation. Inspect annotation provenance and permitted use before constructing a study.

[10] Johnson et al. MIMIC-CXR Database, version 2.1.0. PhysioNet, 23 July 2024. Reports and DICOM resource. Distinct from JPG resource [8].

[11] Ha Q. Nguyen et al. VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations. Scientific Data 9, 429 (2022). Primary paper and Table 2. PhysioNet version 1.0.0. Table 2 supports the published test-positive counts; verify the downloaded annotations during setup. Images are distributed as DICOM under the PhysioNet Credentialed Health Data License 1.5.0. Paper available under CC BY 4.0.

[12] Google. MedGemma 1.5 4B instruction-tuned model repository. Model card and release notes. Used for the 23 January 2026 greedy-decoding default change, the training-source list and the data-contamination advice in the card; pin the actual revision and configuration.

[13] Jain et al. RadGraph: Extracting Clinical Entities and Relations from Radiology Reports. Versioned paper. For RadGraph-based precision, recall and F1, see Delbrouck et al., Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards, Findings of EMNLP 2022, Section 2.2. Metric paper. Select and record the exact metric variant.

[14] Smit et al. CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling. Versioned paper. A text-labeling model rather than an image-grounding evaluator.

[15] Ostmeier et al. GREEN: Generative Radiology Report Evaluation and Error Notation. Versioned paper. Sections 3.2 and 7 describe reference-based error evaluation and limitations.

[16] Bannur et al. MAIRA-2: Grounded Radiology Report Generation. Versioned paper. RadFact describes reference-based logical and spatial evaluation; a different local judge requires separate validation.

[17] Lenovo. ThinkStation PGX Product Guide. Official platform specifications. Also lists NVIDIA DGX OS as preinstalled and a 240 W USB-C power supply. Advertised hardware specifications do not establish the installed software versions or measured throughput.

[18] NVIDIA. CUDA GPU compute capability table. DGX Spark known issues. Used for GB10 capability 12.1, the distinction between unsupported overall framebuffer-memory reporting and available per-process information, unified-memory reporting guidance, and the supplied-adapter recommendation.

[19] NVIDIA. Spark NVFP4 quantization playbook. A general workflow; it does not establish support or speedup for this exact MedGemma configuration.

[20] vLLM documentation. Automatic prefix caching. Caching requires shared encoded prefixes and primarily reduces prefill work; measure compatibility and benefit locally.

[21] PhysioNet. Use of MIMIC Data with Large Language Models and Online Services, 24 September 2025. Published guidance. Credentialed Health Data License. Rechecked 10 October 2026. Consult the applicable signed agreement for data access and release decisions.

[22] NVIDIA. GTC 2027 poster call. Official deadline and criteria. Rechecked 10 October 2026: deadline 10 November 2026, 5 p.m. Pacific; proven technical outcomes, benchmarking methodology and application of NVIDIA technology evaluated; conceptual submissions not considered; one author must attend the poster reception. Submission-portal field limits remain to be checked.

[23] Johnson et al. MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. Version 5 and Table 3. Historical split counts are used for clustering context, subject to verification against current release files. Section 4 describes how the test split was selected and why its findings are enriched.

Terminology

Term	Meaning in this report
VLM	Vision-language model
SAE	Sparse autoencoder
CBM	Concept bottleneck model
CXR	Chest radiograph
V5	Five-finding vocabulary of the Section 8 study
OOD	Outside the evaluated model-development distribution
AUROC	Area under the receiver operating characteristic curve
AUPRC	Area under the precision-recall curve
ECE	Expected calibration error
IoU	Intersection over union for spatial overlap
Percentage points	Arithmetic difference between percentages