Study Cards specification - fli-iam/shanoir-ng GitHub Wiki
Shanoir Study Cards
Goals
The main goal of Shanoir study cards (== rules engine) is to enrich the imported datasets MR metadata, so it can be accessed by Shanoir users and indexed to improve the Shanoir search engine features and performance. Study Cards apply today ONLY on the MR image modality.
Serial number check
In the first version of Shanoir, as any import had to go trough a study card, they were also used to check the equipment serial number in the datasets DICOM metadata. This feature is now a systematic behaviour, independant of the study cards.
Therefore, as soon as a serial number mismatch is detected, an importing user should not be allow to proceed further into the import process. If so, a clear information will be displayed and the user will have the possibility of contacting the administrator(s) of the study with a prefilled mail that will contain the two mismatching serial numbers.
Moreover, the serial number check must discar zeros at the beginning of ther serial.
Rights an roles
An EXPERT should be able to create and edit study cards as long as he has the CAN_IMPORT right.
CT implementation
The CT modality must be implemented.
Updated import page
- Study
- Study Card
- optional
- filtered on the study
- only study cards compatible with the equipment
- Acquisition equipment
- if study card set -> not editable, forced to the equipment linked to the study card
- if not -> filtered on the study
- Subject
- Examination
- Nifti converter
Reversible study cards && Apply study cards on pre-migration data
One flaw of the former study cards system is that the enrichment cannot be reversed. The advantage of reversible study cards is the possibility of editing the study card on any time of the study lifetime.
- In the UI, the re-apply functionalities could be accessible through the study card page.
- In the backend,
- the dataset entity would have two metadata dependencies : originalMetadata and updatedMetadata. A downloaded or displayed dataset should then shows the updated metadata.
- a StudyCard dependency must be added to DatasetAcquisition so we know where to re-apply a study card.
As the former model does not store which study card was applied to a given acquisition, the re-apply functionality will not be available for datasets imported before the migration to the new version. However, a future feature will allow an administrator to select a list of acquisitions and apply a study card to all of them.
[UPDATE] : see reaplication specification
Copy a rule
The rule copy functionality is clumsy : all the study card rules of the database are listed in a single dropdown list. We should limit it to the study's rules.
Mandatory MrDatasetNature
For datasets to be compatible with BIDS, all study cards must implement a rule for setting MrDatasetNature. That rule could be mandatory to implement in the shanoir front or there could be a warning saying the study card is not BIDS compatible until you set that kind of rule.
Enrichable dataset metadata
| Field | OLD -> NG | NG filename | Property path | Elise's comment |
|---|---|---|---|---|
| Dataset modality type | :question: | DatasetMetadata | dataset.updatedMetadata.datasetModalityType | Actuellement seuls les équipements MR et PET sont définis. Voir à raccorder tout ça : laisser CT pour les CT déclarés en tant que MR? Quid de EEG, MEG et SPECT? A ma connaissance les seuls EEG importés viennent de chez nous et le sont en tant qu'extra data. |
| Cardinality of related subjects | :skull: | Pas de connaissance d'utilisation de multiple subject. Un import DICOM = un sujet. Les éventuels templates sont en nifti. | ||
| Protocol name | :star: | MrProtocolSCMetadata | dataset.(mr)datasetAcquisition.mrProtocol.updatedMetadata.comment | A garder |
| Protocol comment | :question: | MrProtocolSCMetadata | dataset.(mr)datasetAcquisition.mrProtocol.updatedMetadata.comment | |
| Transmitting coil | :star: | MrProtocolMetadata | A garder (mais permettre de déclarer les coils dans Shanoir old) | |
| Receiving coil | :star: | MrProtocolMetadata | A garder (mais permettre de déclarer les coils dans Shanoir old) | |
| Explored entity | :star: | DatasetMetadata | Peu utilisé. Redondant avec MR sequence application. | |
| Acquisition contrast | :star: | MrProtocolMetadata | A garder | |
| MR sequence application | :star: | MrProtocolSCMetadata | A garder (plus précis que Explored Entity). | |
| MR dataset nature | :skull: | Pas sur que ce soit très utile - Information sur l'agent de contraste en plus de Acquisition contrast et MR sequence application | ||
| MR sequence physics | :star: | MrProtocolSCMetadata | Ca dit vraiment quelle séquence a été faite et on peut récupérer l'information dans les champs DICOMs (0018,0020 pour tous les constructeurs, 0018, 0024 en langage Siemens) mais ca parle seulement aux MR Physics et quelque radiologues/chercheurs je pense | |
| MR Sequence k-space fill | :skull: | Pas utile aujourd'hui | ||
| Processed dataset type | :question: | DatasetMetadata | Revoir définition Ontologie - Tout est reconstruit aujourd'hui pour moi | |
| New name for the dataset | :star: | DatasetMetadata | A utiliser davantage pour harmoniser le nommage d'un site à l'autre (e.g. 3D T1, 3D T1 MPRAGE, 3D T1 NEURONAV, sont tous des 3D T1) | |
| Dataset comment | :star: | DatasetMetadata | Pas trop utilisé | |
| MR sequence name | :question: | MrProtocolMetadata | Revoir définition Ontologie | |
| Contrast agent (contrastAgentUsed) | :question: | MrProtocolMetadata | Liste des agents de contraste à mettre à jour. Ou bien se contenter d'un oui/non. | |
| Magnetization transfer | :skull: | Pas utile aujourd'hui - je ne vois pas pourquoi on ferait une requete là dessus à travers plusieurs études - information non contenue dans les champs DICOMs par contre | ||
| Parallel acquisition | :skull: | Pas utile aujourd'hui - je ne vois pas pourquoi on ferait une requete là dessus à travers plusieurs études - information contenue dans les champs DICOMs 0051,1011 pour Siemens (p2) | ||
| Slice orientation at acquisition | :skull: | Pas utile aujourd'hui - je ne vois pas pourquoi on ferait une requete là dessus à travers plusieurs études via Shanoir - contenu dans series description le plus souvent | ||
| Axis orientation at acquisition | :skull: | Pas utile aujourd'hui - je ne vois pas pourquoi on ferait une requete là dessus à travers plusieurs études - risque d'erreur d'indexation si dans la study card car non vérifiable | ||
| Slice order | :skull: | Pas utile aujourd'hui |
Model imperfection : Study / Center / AcquisitionEquipment
Today we have :
Study () -------- () Center (1) -------- (*) AcqEq
But in reality a study does not use every acquisition equipment of a center, only a limited and identified part of it. So the model should look like that :
Study () -------- () AcqEq (*) -------- (1) Center
Having an accurate model can only lead to simplifications, for instance no longer would have useless options when selecting an acquisition equipment for an import. But this evolution would also allow us to design a finer control on the data a Shanoir user has access to.
Rights related to an equipment
Today, an "importer" can display every dataset from the study he imports in, unless he hasn't the CAN_SEE_ALL right resulting him no being able to check the data he has imported himself.
To improve that we could have, as a default behaviour without CAN_SEE_ALL, that would allow a user to see only the datasets acquired with an equipment he is explicitely linked with.
The impact on the UI would be an optional list of equipements to set in the user edit page. The impact on the backend model would be the creation of a n/n relationship between User and AcquisitionEquipment. That's a "cross microservices" relationship as User and AcquisitionEquipment are not in the same microservice. The implementaion would take the relationship between User and Study as an exemple.
Rights on hardware for experts
The question is : should expert users be allowed to create centers, coils, equipments, etc or should it be restricted to administrators ? Today they can, but it seems to have led to inconsistencies in the data due to unaware experts.
Technical specifications
Old data model
Center NiftiConverter
(1) (1)
| |
| |
(*) (*)
Study (1) ------------ (*) StudyCard (*) ------------ (1) AcquisitionEquipment
(1) (1)
| |
| |
----------------------- (*) (*)
| StudyCardAssignment | (*) ----- (1) StudyCardRule StudyCardElement (1) --- (*) StudyCardDicomCondition
----------------------- (1) (*)
|- id | | |
|- className | | |
|- field | (*) |
|- fieldClassName | ---------------------- (1)
|- value | | StudyCardCondition | (*) ----------------------- (1) ComparisonSign
----------------------- ----------------------
|- id |
|- dicomTag |
|- dicomValue |
----------------------
Evolutions
- StudyCardElement and StudyCardDicomCondition were not used and seem useless. They will be removed.
- The link with center may be useless as well, as the center is linked through AcquisitionEquipment.
- The comparison sign can be stored as a simple integer, like any other enum in Shanoir NG. We will call it 'operation'.
- StudyCardAssignment 'className' and 'fieldClassName' may be there to resolve Java reflection difficulties, hopefully we will get rid of them.
- DatasetAcquisition has a new StudyCard dependency.
New data model
NiftiConverter DatasetAcquisition
(1) (*)
| |
| |
(*) (1)
Study (1) ------------ (*) StudyCard (*) ------------ (1) AcquisitionEquipment
(1)
|
|
----------------------- (*)
| StudyCardAssignment | (*) ----- (1) StudyCardRule
----------------------- (1)
|- id | |
|- field | |
|- value | (*)
----------------------- ----------------------
| StudyCardCondition |
----------------------
|- id |
|- dicomTag |
|- dicomValue |
|- operation |
----------------------
Migration script
- Just copy the remaining fields, the other ones are not needed.
- DatasetAcquisition.studyCard will be null on migration.
- The comparison sign is now an enum but the ids are the same. So now its not a fk but a simple number, just copy it into 'StudyCardCondition.operation'.
Assignment fields
Assignment fields are now an enum. Here the conversion table from OLD to NG. The mentioned field from OLD refers to STUDY_CARD_ASSIGNMENT.FIELD in the OLD database.
MySQL [shanoirdb]> select distinct(FIELD), CLASS_NAME from STUDY_CARD_ASSIGNMENT;
+-------------------------------------------+-----+
| OLD | NG |
+-------------------------------------------+-----+
| refDatasetModalityType | 1 |
| protocolName | 2 |
| transmittingCoil | 4 |
| receivingCoil | 5 |
| refExploredEntity | 6 |
| refAcquisitionContrast | 7 |
| refMrSequenceApplication | 8 |
| refMrSequencePhysics | 9 |
| name | 10 |
| comment | 11 |
| mrSequenceName | 12 |
| refContrastAgentUsed | 13 |
+-------------------------------------------+-----+
Operation
STUDY_CARD_CONDITION.REF_COMPARISON_SIGN_ID will be copied into STUDY_CARD_CONDITION.OPERATION
Alternate data model
This would permit to manage complex condition expressions (with AND and OR).
NiftiConverter DatasetAcquisition
(1) (*)
| |
| |
(*) (1)
Study (1) ------------ (*) StudyCard (*) ------------ (1) AcquisitionEquipment
(1)
|
|
----------------------- (*)
| StudyCardAssignment | (*) ----- (1) StudyCardRule
----------------------- (1)
|- id | |
|- field | |
|- value | (*)
----------------------- ----------------------
| StudyCardCondition |
----------------------
|- id |
|- dicomTag |
|- dicomValue |
|- operation |
----------------------
(0-1)|
| ----------------
| | |
(1)| |(*) |
---------------------- |
| ConditionGroup | |
---------------------- |
| - id |-------
| - groupType |
| - reverse |
----------------------