Decisions Index: Workflow - crystalyragui/MARC2RDA GitHub Wiki
Reasons for not mapping a row/justifications for recording "Delete" should go in "Notes--Uncategorized" column, not in "Justification for Mapping" column.
I.A.1.a. "Not Mapped" status
Like "Delete" status, "Not Mapped" reasons for not mapping a row should go in "Notes--Uncategorized" column, not in "Justification for Mapping" column.
Add transformation notes whenever possible.
Do not comment on spreadsheets in Google Sheets. Comments, discussions, and issues should be located and tracked in GitHub.
I.A.4.a. Condition Layering
I.A.4.a.i. Multiple values for the same MARC subfield condition with OR relationships should be recorded in the same cell, with | as delimiter.
I.A.4.a.ii. Independent conditions get separate rows in the spreadsheet. Layered conditions are in multiple columns of the same row.
I.A.4.a.iii. New sets of MARCTagCondition/ConditionValue columns may be added as needed to create more layered conditions with AND relationships to one another.
I.A.4.b. Punctuation differences in label values for conditions should be ignored/treated as the same. For instance, “Based on (work)” and “Based on work” should be treated as the same string.
I.A.4.c. Updated Instructions to include formatting for MARCTagCondition cells and corresponding value cells.
I.A.4.d. Added table with prescribed syntax operators.
Mapping Spreadsheets that are finished should move rows with final statuses other than "done" to new sheets in the workbook to reduce redundancy and improve readability e.g. move deleted rows to a new “Deleted” sheet within the workbook. See issue.
The project will rely on GitHub’s versioning control, and refrain from adding new columns/notes to record different iterations of the mapping spreadsheets over time.
Draft versioning will rely on Google Sheets, with frequent semi-automated pushes to GitHub by University of Washington staff.
II.A.1.a. Current milestone: BSR. Minimal Viable Product (MVP) for Transform has been collapsed into BSR. Mapping Review has been abandoned. Publication has been collapsed into BSR or CSR as appropriate. Priority order for subsequent milestones: Post-Phase I Close-out, CSR [Phase II]. See discussion.
II.A.1.b. Old milestone: MVP for Transform. Priority order for subsequent milestones: BSR, CSR, Mapping Review, Publication [milestone for transform will be created once the project has MVP done].
II.B.1.a. The project will focus initial mapping of MVP and BSR Milestones on singleton Work-Expression entities (non-aggregates), with aggregates and serials integrated in subsequent phases.
II.B.1.b. The project will create a set of conditions to identify and exclude aggregates, serials, collections etc., adding them back and treating them appropriately in stages.
The project will use a semi-automated script to push changes directly to the Master branch. the project will not be utilizing pull requests.
When an issue is closed, if it isn't self-explanatory, a note and link will be included to the related decisions recorded in the decisions index at the time of issue-closing.
III.A.3.a. Project issues begin in the "to-do" category of the GitHub project, and are prioritized using Milestones.
III.A.3.b. Issues are self-assigned, and moved to "in progress" category at that time.
III.A.3.c. Once a first pass is complete, mappers move issue to "awaiting review".
III.A.3.d. If one or a few open questions are holding up the first pass, a mapper may move the issue to the "almost done" column, adding a note in the issue explaining what needs to be done in order to complete a first pass. The self-assigned mapper will finish the mapping when the open questions are answered, or will find someone else to adopt the issue. Crystal will monitor the "almost done" column to make sure nothing lingers unnecessarily. Once a first pass is complete, these issues move to "awaiting review". If a mapper believes the mapping is too complex for a single person to reasonably complete, and that the mapping would benefit from group review, the mapper will also label the issue "meeting discussion needed".
III.A.3.e. Any project participant except for the person who performed the initial mapping may elect to take on the role of "reviewer" for a section/field of the mapping. Reviewers self-assign issues from "awaiting review" to review mappings. Once review is complete, issues are moved to "ready for transform".
III.A.3.f. Transformers select issues from "ready for transform" and write code for them. Once code is complete, issue is closed and automatically moved to "done".
III.A.3.g. If changes are needed after an issue has been moved to "done", the issue is re-opened and re-assigned, and moved to the appropriate column. Notes should be made in the issue to indicate to mappers that transformation code needs to be adjusted/checked against any changes.
III.A.3.h. If a reviewer makes changes to the mapping and the mapping has a tag indicating a transformer is writing code for that mapping, ("coding-ar", "coded-ar", "coding-rip", "coded-rip", "coding-rft", or "coded-rft"), the reviewer should add the "code re-check" label to the issue to indicate that transformers need to update the code.
III.A.3.i. Once Transformers have re-checked issues with the "code re-check" label and made any needed updates to the code, remove the re-check label when done.
III.A.4.a. Transformation Review issues begin in the "Transformation Review To-Do" category of the GitHub project and are assigned an appropriate milestone, along with labels "Transformation Review To-Do" and "Input Data Needed". Other labels can be assigned as appropriate for sorting. Each issue includes links to the corresponding initial mapping and transformation issue, mapping spreadsheet, and input and output from the field-by-field testing done at the time of transformation by the coders.
III.A.4.b. Project team members: Crystal, Adam, or Deborah supply input test data to review issues either together or in groups.
III.A.4.b.i. Test data in the form of approximately ten records in MARC XML format goes into the MARCDatasets folder in the GitHub repository.
III.A.4.b.ii. Once a test dataset has been supplied, it is linked to in a comment in the Transformation Review issue(s) and a corresponding review discussion is created. Review discussions are also linked from the Transformation Review issues as comments. One discussion per review dataset may be used, to combine discussions on several tags at once.
III.A.4.b.iii. The label "input data needed" is replaced with "output data needed" and "needs coder attention".
III.A.4.c. Coders self-assign issues with the labels "input data needed" and "needs coder attention". Input data from the MARCDatasets folder is run through the current version of the transformation to produce outputs in the output data for review folder. At this point, the labels "need coder attention" and "output data needed" are replaced with "ready for transformation review"
III.A.4.d. Review
III.A.4.d.i. Once it is decided to review an issue, the "ready for transformation review" label is removed and "transformation review in progress" is added. Issues may be reviewed either asynchronously or during a meeting or both. "Asynchronous discussion needed" and "Meeting discussion needed" labels should be added to both issues and discussions as appropriate at this stage.
III.A.4.d.ii. If no problems are found with the code, reviewers should state this in the discussion and change the issue from "transformation review in progress" to "done" in the workflow.
III.A.4.d.iii. If problems are found in the code, discussions should be used for all feedback until a clear comment can be left in the issue directing coders on how to fix the problem. After a clear comment has been made, the labels "code re-check" and "needs coder attention" should be added to the issue. Once issues are resolved, the issue can be moved to "Transformation review complete" and closed.
III.A.4.d.iv. Issues with code may be found outside of this regular transformation review process. If that happens (say, for example, an issue with a field that isn't being reviewed yet is found during review for another field), create a new issue typed "Bug" with the labels "Transform" and "needs coder attention". Add the prefix [BUG] to the title. If possible, link to the area in the code that needs work. At least, link to the mapping spreadsheet and initial mapping and transformation issue related to the problem if they are relevant. Assign bugs to the to-do column of the project. Coders self-assign bugs by moving them into the "in progress" step in the workflow and assigning themselves to the issue. Once the bug is fixed, move the issue to "done".
Select field using the main project board and assign yourself to the issue on GitHub. Do not code fields in "To Do" or "In Progress." It is OK to code fields in "Awaiting Review" or "Review in Progress" if the procedure below (III.B.3) is followed. It is best to select fields from "Ready for Transform"; follow the procedure below in (III.B.4).
A field can be moved into "Done" on the main project board if two conditions are met:
(1) the coding is done;
(2) mappers have moved the field into "Ready for Transform."
III.B.3.a. Manually edit the issue by adding the appropriate label indicating the field's status on the main project board at the time of the coding. Either use "coding-ar" (for a field being coded while "Awaiting Review") or "coding-rip" (for a field being coded while "Review in Progress").
III.B.3.b. Add/Commit the code for the single field; include a commit message that includes the following: -m "Code for field XXX [awaitingReview OR reviewInProgress] #[issueNumber]." Example: "Code for field 100 awaitingReview #102."
III.B.3.c. When coding is complete, manually edit the issue by removing the "coding-ar" or "coding-rip" label and replacing it with the "coded-ar" or "coded-rip" label as appropriate.
III.B.4.a. Manually edit the issue by adding the label "coding-rft" (III.e. "We are coding this field in the main project board's category 'Ready for Transform").
III.B.4.b. Add/Commit the code for the single field; include a commit message that includes the following: -m "Code for field XXX #[issueNumber]." Example: "Code for field 100 #102."
III.B.4.c. When coding is complete, remove the "coding-rft" label and replace it with the "coded-rft" label.
III.B.5.a. Change the transformation-related tags from the appropriate "coding" tag to the matching "coded" tag. If the field was in "Ready for Transform", close the issue for the field.
Project team members will talk and listen in turns, using hand-raising functionality in Zoom interface to order discussions.
Agendas, set by Crystal and edited by anyone, will include time limits for items. Unused time can be rolled over to the next item.
Timekeeper and note-taker will be assigned at the start of each meeting (as separate duties). Note-taking should rotate between UW members. Notes may be amended by anyone after each meeting.
Off-topic but important items will be noted in a "Backburner" section of the notes/agenda in an effort to keep conversations focused.