Evaluation Plan: rapid group testing of v3 (Week 11)

Our first plan (35–40 minute sessions in people's homes) was not realistic in the time left: participants could not give that much time in assessment season. Following the course advice to adapt methods rather than drop them, we switched to short, guerrilla-style group sessions. They keep the two rules for social and mobile evaluation:

1. Several people use it at once: every session has 2–3 participants on different devices.
2. As close to the real context as we can: the Dock sits on a table set up as a kitchen bench, and the Cart Unit is used beside a small "shelf" of real products with a fixed budget.

The evaluation is formative: we want to learn what works, what confuses people and what to change, not prove the design works. No SUS or TAM.

Who and where

• Who: UQ students who live with other people (share house, student accommodation, or flatmates). One screening question: "Do you live with housemates or flatmates?" Studio peers count only if they do, and we note it.
• Where: our lab and studio, campus (library, food court), and our own housemates and friends.
• How many: 4–6 groups of 2–3 people (about 10–15 participants), Thu 15 – Sun 18 Oct.
• Team roles: one facilitator and one note-taker per session. Everyone runs at least one.

The session (10–15 minutes)

Set-up: Testing mode → Reset to demo house. Dock on the "bench" (laptop or tablet), two phones on /cart, about 10 real products with prices on a table.

Min	Step	What the facilitator says or does	Group moment
1	Consent	Read the consent line (below)	–
1	First look	Point at the Dock: "What do you think this is for?" Note the answer; don't explain.	–
3	Plan	"You live together. Plan this week's shop using this." (vote on a meal, add a request)	G1, G2
4	Shop	Person B takes a phone: "You're going to the shops. Start the shop and get 3 things within $25." Partway through: "They're out of yoghurt. What do you do?" (Ask home is there if they find it.) Person A stays at the Dock.	G3, G4
1	Split (if 3 people)	Person C joins on the second phone and splits the list	G5
1	Back home	B taps Finish; everyone looks at the summary	G6
3	Questions	The 4 questions below	–

4 questions (ask exactly these, in this order):

1. What was the most useful part? What would you ignore?
2. When would you just text your housemate instead?
3. How did the cost split feel: fair, useful or awkward?
4. Was there anything you wouldn't want your housemates to see?

What we record (one sheet per group)

Group	Date	Facilitator	Who (codes)	Lives with others?	Said it's for…	Got stuck at…	Used Ask home?	Texted instead?	Cost split reaction	Best quote	Change to make
G-01			G01-P1, G01-P2

• Codes only (G01-P1…). No names, faces or recordings. Photos of hands and screens only, with permission.
• Within the day, add each group to the evidence register.

Consent line: "We're DECO3500 students testing a prototype, not testing you. It takes about 10 minutes. We'll take notes and maybe a photo of hands and screens, no faces or names. You can stop any time. OK?"

Analysis (Sun 18 – Mon 19)

1. Put every "got stuck", "texted instead" and quote on sticky notes, and group them.
2. For each group of notes: what happened, how often (e.g. 3 of 5 groups), one quote, and the change it leads to.
3. Make only the changes the evidence supports (v4), then update the poster panel, pitch and Design Process Overview section 6.

Tradeshow as a second round

At the tradeshow, visitors try the same 5-minute flow in pairs. The note-taker uses the same sheet, and the findings go into section 7 of the Design Process Overview.

Limitations

Short sessions show first use, not how a household behaves over weeks (Grudin). The setting is staged rather than a real kitchen and store. Participants are students we could reach quickly. We report these patterns as formative findings, not proof.

Results

Group	Date	Setting	Key observations	Changes made
G-01
G-02
G-03
G-04

Summary of findings: [after analysis: 4–6 bullet points, each with evidence]