HALA — and the question that stays after the answer

You work with an AI. You get a good answer, build a reasoning, solve a problem. Then you close the chat.

What stays?

HALA proposes comparing three learning conditions: no AI, an AI tutor, and a critical AI collaborator. In the proposed study, learners would take an independent baseline assessment, learn under one condition, then complete an independent post-test, a transfer task using an unfamiliar problem, and a retention assessment 7–14 days later. This design has not yet been tested with human participants.

That’s the question HALA starts from. Human–AI Learning Assessment — a name that sounds more definitive than it is. Right now it’s a repository, a proposal sent to the OECD, and a prototype with synthetic examples. No human study has been conducted. No measure has been validated. Just a question that won’t let go.

People learn with AI — but how do we know what actually stayed with them?

This isn’t an answer. It’s an attempt to turn that question into something we can investigate together.

This grew out of my longer exploration of learning with AI: prototypes, scripts, and a research proposal. On October 9, I submitted the proposal to the OECD as part of its call on AI-mediated assessment. I don’t know if it will go anywhere. Maybe it will, maybe it won’t.

But the work exists. It lives in a repository that someone might find useful — a script, an example, a way of documenting limitations. Or maybe a mistake to flag, a better path to suggest.

That’s why I’m sharing it.

What’s inside

The HALA repository holds:

• A study proposal — a proposed design for assessing learning with AI
• Working documents — notes, reasoning, design choices
• A prototype with synthetic examples — enough to see the direction, not enough to draw conclusions

The current prototype uses a scripted AI stand-in, and its retention step happens in the same session. It does not test the delayed-retention study described above.

Everything is still early. No human subjects, no external validation. HALA is currently a personal research project; there is no established research team or institutional partnership. Its value lies in making the design, tools and limitations available for others to examine and build on.

What it means to share unfinished work

I’ve spent years learning with AI, including working with Gino, an AI collaborator, and wondering whether I was actually learning or just surfing well-packaged answers. HALA is my attempt to turn that unease into something concrete and discussable.

The reviews documented so far were conducted by AI agents. Independent human review, especially from people with experience in psychometrics or learning sciences, is still missing.

Here is one specific question I would value help with: what transfer item would show that a learner can apply the idea to a genuinely new problem, rather than repeat the explanation? What would that item need to control for?

HALA isn’t a finished project. It’s a place to start.

Explore the HALA repository

Read the submitted proposal — PDF, 2 pages

This post is licensed CC BY-NC-SA 4.0. The repository’s text and figures are CC BY-SA 4.0, and its code is AGPL-3.0-only. See the repository’s licensing guide.

I don’t know if this will be useful. But keeping it closed would have been worse.