Overall architecture

All of the Canvas interaction is held in the CanvasView.

The CanvasView contains a CodeCardView for each code card, and a PencilKitDrawingView for pencil interactions.

To allow the view to adapt to iPad orientation changes, you should pass in height and width to CanvasView from a geometry reader.

The CodeCardView is implemented using a simplified extension of SwiftTerm's TerminalView which is designed to handle static content only. This allows code cards to parse and display content in the same encoding it uses in the terminal as well as providing an identical visual experience to the terminal view used for live connections. These static terminals have been extended to handle resizing.

User interaction

Drawing mode vs. manipulation mode

The canvas has two interaction modes, to facilitate the combination of drawing and manipulating code cards. Our initial goal was to allow the user to always draw with their Apple pencil, and always be able to navigate and manipulate code cards with their finger. However, we struggled to integrate UIKit and SwiftUI in order to do this, so ultimately we settled on toggling between two modes.

In drawing mode, the user can draw with the Apple pencil. They can't interact with code cards.

In manipulation mode, the user can move, resize, and delete code cards. They can't draw.

In both modes, the user can navigate around the canvas by panning and zooming. To switch between modes, the user can press the mode toggle button or double tap with their finger.

Touch gesture interaction

Canvas interactions

• drag to pan around
• pinch to zoom in and out
• double tap to switch to drawing mode

Code card interactions

• drag to move and arrange cards on the canvas
• drag the resize icon to resize the card
• long press to bring up the option to delete the card

SwiftUI gestures and gestureStates are used for touch interactions. Whenever a gesture is completed, its respective value (magnification, x or y location, offset size) is saved in CoreData in the attributes described in the "Saving canvas and code card data" section.

Notably for code cards, the embedded terminal won't resize until the gesture is completed. This is due to limitations of passing GestureState as a binding to embedded UIKit components.

Apple pencil interaction

An embedded PKCanvasView allows the user to draw with the Apple pencil in drawing mode. All drawing tools such as pen, pencil, eraser, highlighter, cut, and color picker are implemented through PencilKit.

An UndoManager allows the user to undo and redo pencil drawings.

Saving canvas and code card data

When the user interacts with the Canvas component, information is saved in the Canvas and CodeCard entities through CoreData. The Canvas entity represents the canvas itself, and is linked to CodeCard entities, which represent each draggable card created from the terminal component.

Both Canvas and CodeCard entities have custom NSManagedObject subclasses, so that we have more control over required attributes and their defaults, and one-to-many relationships such as Canvases to CodeCards. The custom relationship-based attributes are noted below.

Canvas

General Attributes

Attribute	Type	Description
id	UUID	a unique identifier, hidden from the user
dateCreated	Date	date when canvas was created
title	String	a user-inputted name which will represent the canvas on the home page
drawingData	Data (binary data)	binary data version of any Apple pencil drawings made on the canvas
locX	Double	x location of the canvas (used for saving the user’s last position when panning around the canvas)
locY	Double	y location of the canvas (used for saving the user’s last position when panning around the canvas)
magnification	Double	how zoomed in the user last was when using the canvas

Custom relationship-related attributes

Attribute	Type	Description
cardArray	array of CodeCard	an array of code cards that exist on the canvas, sorted by descending zIndex

CodeCard

General Attributes:

Attribute	Type	Description
id	UUID	a unique identifier, hidden from the user
dateCreated	Date	date when card was created
locX	Double	x location of the card relative to the canvas
locY	Double	y location of the card relative to the canvas
zIndex	Double	stack order index for a card; higher zIndex cards will always show up on top of lower zIndex cards
text	String	the text output from a terminal that is displayed on the card
width	Float	width of the card
height	Float	height of the card

Custom relationship-related attributes

Attribute	Type	Functionality
origin	Canvas	the canvas that this card exists on