Cognitive Mirror: an AI that says what it does not know

In one sentence

Cognitive Mirror is a short AI session that looks at how you learn and ends with one change to make.

Problem

Most people who drop an online course decide they lack discipline. More often the problem is a wrong picture of their own learning: people overestimate how much they have understood and repeat methods that don't work for them. Platforms answer with more content. Almost nobody looks at how a particular person actually learns.

What a session looks like

A conversation about a real, recent attempt to learn something. Then a short task where you say how sure you are of your answer before it is checked. At the end, a report: 5 dimensions, each with evidence from the conversation and a confidence level, plus one change to start with. When there is too little data, the report says so instead of guessing.

What I decided

• The model only collects observations. From the conversation it pulls only simple facts: whether something came up, how many times, which of a few options was chosen. A rule in code works out the score and the confidence level, and "high" confidence needs at least 2 independent signals. The model cannot decide on its own that it is sure.
• Ethical limits are a requirement, not a footnote. The product makes no clinical diagnosis and has an open path to a specialist. Spotting a situation where someone should be referred is a separate, tested step, because measurement showed it was the element most easily lost.
• Privacy is in the architecture. Conversations and reports are encrypted, consents are recorded, and the model receives only what it needs. I left the legal assessment to a lawyer and prepared a list of questions for them, instead of pretending code could replace it.
• The engine sits behind an API, so the model provider can change without touching the scoring rules.

Measurement instead of promises

The same conversation scored a second time could give a different result. Instead of fixing that by feel, I built a measurement layer on top of the frozen scoring rules and wrote up the results, the bad ones too:

• When a small local model analysed the conversation in one large call, it agreed with the production model on 20 of 26 points and missed a situation that needed a referral to a specialist. Split into a separate call for each dimension: 25 of 26, and the referral was caught. A different way of working helped; a bigger model was not needed.
• A paraphrase test, the same answer said in other words: the production model gave the same result in 39 of 40 cases, the small model in 37 of 42.
• On a repeat run with the model's randomness set high, the quality gate failed. It turned out the production version didn't set that parameter at all, so the main suspect was the configuration.

I deployed no fix automatically. Each one waits for a person's decision. The table of measurement properties, laid out on the COSMIN scheme, is generated straight from the measurement data.

Interface design

I design the interface in stages, and each stage ends with my decision. Done so far: foundations, the report, the session with escalation, consents, the account, payments and the public pages, in 4 languages: Polish, English, Ukrainian and Russian. Some decisions come straight from measurement integrity: the confidence field has no default value and no slider, so it doesn't suggest an answer, and withdrawing consent takes one click with an undo.

What I don't know yet

• The product has had no real users, and the instrument is neither empirically validated nor peer reviewed. The documentation says so in those words.
• Payments work in test mode only. The legal review and an independent security review wait for the decision whether this becomes a product or a study.
• The Ukrainian and Russian versions are screen designs, not a validated instrument in those languages.

This list is part of the evidence. It shows where what I have been able to check ends.

My role

I set the scope, the rules, the ethical limits and the acceptance gates for each phase. The model writes the code, the tests and the first drafts of the documentation.