Break it: test like an engineer · The Build Kit

Card 05 · Break it: test like an engineer · 20 minutes (1:55–2:15)

You are the subject expert, not the tool. Make it fail before a student does. Level 3 ■ Black: write your own test pack (the edge pack below) into your spec, before you build.

"Codex had marked its own homework, and it had marked it generously."

Alex Gray, I Gave ChatGPT a Week to Build My App

Test both directions: the 29p story

"Type 0.29 into the finance app and the finance app tells you that is not money."

Alex Gray, I Gave ChatGPT a Week to Build My App

In Alex's finance app, the tests "only ever checked that invalid amounts were rejected. Nobody had asked the opposite question: are all the valid amounts accepted?" Every test passed, and the app still rejected valid amounts. So check that every right answer is marked right, as well as every wrong answer marked wrong. As Alex puts it, "all the tests passed" can be the most misleading sentence in the room.

The swap: 9 minutes each way

Find your test partner before the break. Pair with anyone at a different level; if you can't, pair with someone who hasn't seen your build. An odd number makes one trio, with 6-minute turns.

Hand over your tool, your canvas and your track card. The tester writes on the bug log on your track card (cards 10, 11 and 12 each have one). From now on the maker says nothing.

Minutes	Tester	Maker
0–1	Read the maker's outcome statement and three success checks (canvas boxes 2 and 4)	Silent
1–6	Run each success check exactly as written: do what it says and watch what happens. Not what you think they meant: what they wrote. If you can't tell whether it passed, that is a finding too. Log each failure on the maker's bug log	Silent: watch where it fails
6–8	Try to break it with two of these four edge cases (3, 6, 7 and 8 below): no answer, a right answer in another form, keyboard only, phone width. Log what fails	Silent
8–9	Hand back the track card and point to the most serious failure	Read the bug log. No fixing, no defending

Then swap roles. The card 08 items for your level are yours to check in Sprint 2 (card 06), not your tester's.

Working alone?

Find a tester who hasn't seen your build: a colleague, a friend or someone at home. Give them your three success checks (canvas box 4), then say nothing while they test.
Nobody free? Test it yourself the next day, as a stranger would: read each check as written, and do exactly what it says.
Either way, work through "Testing your own tool" below.

The bug log

One line per failure, on the maker's track card:

Check	Expected	What happened
Which success check, or what you did	What should have happened	What you saw, in plain words
1: choose the right answer to question 2	"Correct." and why	"Not quite."

Testing your own tool

Do this in Sprint 1 once a check passes, and again in Sprint 2 after each fix. Minimum today: content check plus cases 1–3 of the edge pack.

1. Check the content by hand

Against your own scheme of work or textbook, check every one:

Used the build prompt (card 04)? Start from the list the tool gave you at the end, then check the page itself: the list can be wrong too.

Simulation or model: work out two or three results by hand first, write them down, then check the tool gives the same. Look at the picture as well as the numbers:

"An animation that shows a process wrongly does more damage than a sentence that describes it wrongly, because students remember the picture."

Alex Gray, Claude Design: A Teacher's Walkthrough

2. Run your success checks

Play it once as the user on your canvas, from start to finish. Then run each success check, and any test cases in your spec, and log each one below.

3. The eight-case edge pack

Adapted from DEN Lab Build 04, Generate the Edge-Case Test Pack.

#	Case	What you do	It should…
1	The expected student	Work through as a typical student, answering correctly	Accept every right answer and reach the end
2	The wrong answer	Give the most common wrong answer	Say it is wrong, explain why, and say what to do next
3	No answer	Press Next or Check with nothing chosen or typed	Ask for an answer. No crash, no skip, no "Correct"
4	The weakest student	Give two wrong answers to the same question	Explain each time. After the second, say "Ask your teacher" or show a worked example. No shaming words
5	The fastest student	Click through as fast as you can; double-click buttons	No double counting, no skipped feedback, something worth doing at the end
6	The student trying to break it	Type nonsense or a very long answer; give a right answer in another form (e.g. 0.5, 1/2, .5); refresh; press Back	Nothing breaks; no answer shown early; other correct forms accepted, or the expected form made clear
7	Keyboard only	Put the mouse away. Use Tab, Shift+Tab, Enter, Space and the arrow keys	Every control reachable and usable; you can always see where you are
8	Phone width	Narrow the browser window to its smallest, or send the file to your phone.	Readable and tappable, with no sideways scrolling

4. Access and safety quick check

Access checks from WCAG 2.2.

Text contrast at least 4.5:1 (check your colours in a free online contrast checker)
Right and wrong shown in words, not colour alone
Every button has words on it; every image has alt text; every input has a label
Buttons at least 24 by 24 pixels, or spaced well apart
Any timer can be turned off, adjusted or extended; anything moving for more than 5 seconds can be paused; nothing flashes
Anything students drag can also be done by clicking
No name box, nothing saved (unless it is a Remember build: card 08), no personal data in the page or the file name
Nobody is asked to sign in
Offline test: open the final saved file in a private window with Wi-Fi off and repeat the normal and error cases. Record which actions still work. This checks offline behaviour on those paths; it does not prove that the file makes no network calls. For the no-network rule, ask a helper to review the final code and inspect the browser Network panel while every task and error case is exercised. Look for relative requests as well as full web addresses. An http://www.w3.org namespace inside an SVG drawing is not a request. If the code review and runtime results disagree, investigate before sharing. Canva builds need internet; skip the offline test.

Run log

#	Check	Expected	What happened	Pass / fail	Fix needed
1
2
3
4
5
6
7
8
9
10

Passing these tests shows the tool does what you specified. It is not evidence that students learn from it: that is Teach it (card 07).