Teachers trusted the badge, and it changed nothing. So I changed what it said, and kept testing.

The situation

eduki is a marketplace where teachers buy and sell classroom materials. A buyer has a thumbnail, a title, and a preview to go on.

eduki's AI assessor scores materials against a quality framework, and the best ones get a badge: eduki Qualität.

The badge was A/B tested twice. Both times it came back slightly negative: GMV per session −1.63% and conversion −0.80%. It stayed live anyway, and that call was the PM's.

The constraint

A survey of 288 buyers explained why it wasn't landing.

• Teachers rated the badge's trustworthiness 3.66 out of 5.
• 81% said it changed nothing for them.
• Asked who decides which materials get the badge, respondents were split: no single answer got more than 4.1%. Of the 193 who answered the question about the criteria, one could name them.

Trusted, and inert. Both A/B tests had changed how the badge looked. Neither changed what it said.

The average was also hiding two opposite results.

One average: −1.63% GMV per session. Two opposite effects inside it.

1. Active buyers · 66% of sessions

   −2.10%

   GMV per session fell. Buyer interviews say these teachers already have their own way of judging a material.

   FixLeave their path alone. The preview already does this job.

2. Inactive and churned buyers · 30% of sessions

   +5.11% · +17.47%

   GMV per session rose: inactive buyers +5.11%, churned buyers +17.47%.

   FixDecide who sees the signal. A targeting call.

Combine the two mechanisms back into one result, or separate them

The thing teachers asked for second most often, knowing what concretely makes this material good (29.2%), is something a badge can't say.

Round 1: make it say something

The PM's hypothesis was to stop showing a yes/no mark and show the quality criteria. My part was everything a buyer would see.

Facts instead of verdicts. The existing module repeated the same five definitions on every badged material. The new one says what this material did for each criterion:

Not: "several clearly structured entry points accounting for different levels"

But: "Drei Stufen: Lücken füllen bis »gut – besser«". Three levels, from filling gaps up to "good – better".

A teacher can check the second one in the preview.

Rules before copy. I wrote the rules a generator would have to follow, and hand-wrote nine lines from real assessor output as the worked example:

• Only criteria rated high get a line.
• Two criteria that don't count towards the score never appear.
• No scores, ratings, or verdicts. Every line names something concrete in the file.
• One fact, one line. This rule came from a designer's review.

Predictions before data. I tested the one question a single group can answer: do teachers read the module as being about this material? The task never mentioned quality.

Before any data came in, I wrote down nine predictions, a pass threshold, and a fallback: if teachers only skim the collapsed peek and still read it as boilerplate, drop the module for simple tags.

What round 1 showed

The study ran in August 2026: 22 usable sessions.

The words worked. Asked what this section would say on another material with the same mark, 72.7% said something different. Only 9.1% said the same text, against a threshold of one in five set in advance.

Almost nobody used them.

• 31.8% said they noticed a quality mark.
• 6 of 22 opened the module.
• Asked what influenced their decision, 90.9% named the preview and 13.6% named the module.
• Nobody repeated a single fact from it.

One teacher judged the quality from the preview, then turned the material down on fit.

Four of my nine predictions were wrong.

What I recommended to the PM:

• Don't rewrite the copy. No rewording fixes "few open it, fewer use it".
• Don't fall back to tags. The fallback's first condition was met: most teachers only skimmed the collapsed peek. The second wasn't, because teachers read the lines as specific.
• Decide who it's for. The A/B and the study agree that active teachers don't need it. Whether lapsed buyers would use the module is untested. Who sees it is a targeting decision, which makes it a product call.

Round 2: the module we're testing now

What ships needs lines written from the assessor's output. So round 2 tests that version.

What changed.

• Shorter. I cut the sub-criterion titles and kept the structure teachers understood.
• Real copy. The new test material is a 52-page teaching unit. Its nine lines were written by the assessor team from the material's actual assessment, following our rules.
• One rule set, in one place. The rules had spread across documents, so I merged them into one file. "One fact, one line", the rule round 1 surfaced, is now in the PM's requirements for the prompt.
• Two things the data showed. Every badge-eligible material has at least 9 of its 12 criteria rated top, so a full module is the normal case. And the assessor can rate the same criterion differently from one run to the next. That went to the PM as a risk to solve before anything ships.
• Built fast, tested for real. I built the iterations with Claude Code and the Figma MCP: more than 30 versions, and the ones worth testing as working HTML prototypes. Teachers tested the real thing.

Two tests, with the bar set before launch.

1. Does real copy still read as specific? A website test with eduki teachers, using round 1's question. Pass: 14 or more of 20 expect different text on another material, and 4 or fewer expect the same.
2. Can it be seen? A five-second test: the page as it is, against the same page where the seal pops once as it loads.

Both studies are being set up now. The A/B comes after, and that call is the PM's.

Reflection

The instinct after two negative tests was a third redesign. The more useful move was to separate what the badge looked like from what it said, change only what it said, and test in small rounds, each asking one question with its pass bar written down first.

Round 1 answered its question: the words work. What it surfaced is harder. Teachers already have a method, and it's the preview. A quality signal can't answer a fit question, however well it's written.

What I'd do differently: check the stimulus against the plan before fielding. Round 1's comprehension question showed the expanded module. Most teachers saw the collapsed state. And I'd recruit the segment on purpose: 59.1% of the sample already knew the badge, against 39.2% in the survey. That skews towards the buyers the A/B says the badge hurts.

The work