Notes from work in progress: what I tried, how I measured it and what came out. Newest first.
Choosing the right AI model for each task can be automated, but not out of the box: my router was confidently wrong until I measured it and retrained it on my own tasks. It now gets 85% right, and it's switched off, because the hardware it needs would cost more than it saves.
My site works, it reads well, and nobody will remember it. I asked AI for something bold and got a poster from 1973, then three different styles on one skeleton. The problem was what I had given it.
Same brief, same six reference images, two ways to make the screens: an AI agent on its own, or me at the design tool. I scored twelve unlabelled variants against a rule written in advance.
Every test passed on a mock model. On the real one, 40% of answers fell back to an emergency mode. Four product decisions that fixed it, and what you can take from them.