The word "self-improvement" is doing too much work

When people hear "self-improving AI," many imagine the same thing: a system quietly rewriting itself, changing its own goals, and accelerating beyond human oversight. Recent CNBC coverage of concerns raised by Anthropic and OpenAI researchers about recursive self-improvement taps into that fear. If AI starts improving the process of building more powerful AI, researchers worry, humans may eventually lose the ability to understand, predict, or control what comes next.

That is a serious conversation, and I do not want to dismiss it.

But as an engineer who has used AI-assisted workflows on real software migrations, I think the phrase "self-improving AI" also obscures a less dramatic, more practical distinction.

Self-improvement is only scary when it is unbounded, hidden, or misaligned with human intent.

In software work, "improving the agent" does not have to mean modifying model weights or letting a system rewrite its own core training procedures. More often it means updating documentation, adding examples, clarifying rules, improving prompts, and recording proven patterns in a controlled, versioned place. Boring, but useful. And in this context, "self-improvement" means instructing the agent to do all of that by it self as part of the process to accomplish some other goal.

I tried this during a large test migration.

The migration: 10,000 Karma/Enzyme tests to Jest and React Testing Library

We had a large codebase full of old frontend tests written with Karma and Enzyme. The goal was to migrate them to Jest and React Testing Library.

This was not a simple find-and-replace job. Enzyme tests often reached into component internals. They made assertions against implementation details, used shallow rendering in ways that did not survive well in a behavior-oriented testing model, mocked dependencies in inconsistent patterns, and tested event handling through internal props rather than through accessible user-facing behavior.

React Testing Library pushes you in a different direction: render components as users experience them, query by accessible roles and visible text, interact through realistic events, and assert on observable outcomes instead of private structure.

That sounds simple until you have thousands of tests in front of you.

A migration like this is full of one-off discoveries. We had to start reviewing what the components being tested were doing, document their behavior. We had to work out how to rewrite specific Enzyme assertions into accessible queries, decide on what mock strategy to follow to avoid over-mocking when the real integration is more valuable and without making the test fake the thing being verified, how to handle event dispatching correctly after moving away from Enzyme's `.simulate()` style and test async UI behavior with `findBy*` queries instead of synchronous assumptions, among many other complexities.

I researched and created a skill to migrate Karma/Enzyme to Jest and React Testing library which included a few general migration patterns.

After running the Codex agent on the first 10 tests, I noticed that it was wasting too much time and tokens doing similar replacements even when many clear patterns were apparent in the code. Patterns that were not general but specific to our code base. So, how could I take advantage of that?

I tried adding a simple document: `testing-patterns.md`. And modified the skill to update it after each migration with any new pattern it found out.

It was not fancy. It was a growing library of migration patterns. Every time the agent found a good way to convert a class of tests, it added it to the document. Every time the agent-assisted workflow solved a tricky case in a some way, it captured the rule and the example.

Sometimes, it captured not the best pattern. It was able to realize that ignoring tests could make the suite to pass and every time I tried to migrate more than 15 tests at once, it fell back to this pattern which messed up with the migration progress. So, I limited to sets of 5-8 tests each time.

The result was that our workflow became better at future migrations. The model weights did not change. The system did not secretly evolve. We learned with the agent unpredictable autonomous decisions. And it improved because the environment around the agent accumulated better memory, better instructions, better examples, and better constraints.

That is self-improvement in the practical engineering sense.

Two very different meanings of "self-improvement"

A lot of the fear around AI self-improvement comes from collapsing two different things into one phrase.

The first is the kind of self-improvement that is genuinely concerning at frontier scale. In that version, the system participates in improving itself in ways humans cannot easily inspect, approve, measure, or roll back. It might help redesign training objectives, propose architectural changes that are incorporated into the next model, or operate across a loop fast enough that human review becomes ceremonial rather than meaningful.

That is the kind of "recursive self-improvement" researchers worry about when they talk about loss of control, capability jumps, and misaligned autonomy.

The second is the kind of self-improvement that happens in ordinary software teams every day. In that version, the system does not modify itself in secret. It updates an external artifact: a rule in a Markdown file, an example in a prompt library, a checklist in a playbook, a snippet in a documentation page, a skill definition in a tooling repo, or a test-migration pattern in `testing-patterns.md`.

That change is visible. It can be reviewed, versioned in Git, discussed, tested, accepted, rejected, amended, or reverted.

Those two meanings of self-improvement are not morally equivalent. One is a frontier-control problem. The other is mostly a knowledge-management problem. If we blur them together, we risk missing an opportunity: safe, bounded, auditable self-improvement is not only possible. It is often one of the best ways to make AI systems more useful and predictable.

A concrete example from the migration

A mini scenario happened repeatedly.

1. The test-migration agent received a failing Karma migration.

2. It encountered an unfamiliar Enzyme pattern.

3. It found or proposed a solution.

4. If valid, the solution was recorded in `testing-patterns.md`.

5. Future migrations retrieved the relevant pattern from the document into the agent context before acting.

That is the whole loop. No model surgery, no silent change to weights, no autonomous goal drift. It was a human-supervised process where useful knowledge became reusable.

One recurring pattern was replacing Enzyme-style internal checks with accessible queries.

A naive Enzyme test might assert that an internal click handler exists:

```ts

expect(wrapper.find('button').props().onClick).toBeDefined();

```

That test does not really tell you whether the user can interact with the component in a meaningful way. It checks implementation plumbing. It survives refactors poorly and encourages brittle tests.

The React Testing Library migration often pushes you toward something like:

```ts

expect(screen.getByRole('button', { name: /save/i })).toHaveAttribute(

'aria-disabled',

'true'

);

```

This asks a user-centered question: what is the button called, and what state is it in? It is closer to what a user would observe. It is also less coupled to whether the component happens to pass an `onClick` prop into some internal node.

Once we saw the pattern recur, we captured it.

`testing-patterns.md` might accumulate a rule like this:

```md

Pattern: Replace Enzyme internal button checks with accessible role queries

Before:

Use `wrapper.find('button').props().onClick`

After:

Use `screen.getByRole('button', { name: /.../ })`

Use when:

- Migrating from Enzyme to React Testing Library

- The component renders a user-facing button

- The test asserts behavior, not implementation details

Why:

- More resilient to implementation changes

- Aligns with how users interact with the UI

```

That document entry is not glamorous. But it is the kind of improvement that matters in practice.

The next time the agent or a developer encounters a similar Enzyme test, they do not have to rediscover the principle from scratch. The workflow retrieves the pattern, applies it, and checks the result against tests.

Over time, the system becomes better at future migrations. "Better" here means humble: better context, more examples, clearer boundaries, less iterations and less tokens wasted, consequently cheaper and faster.

A patterns document turns one-off discoveries into reusable competence

In large migrations, the expensive part is rarely typing. It is decision-making.

Every test is slightly different. Every component has quirks. Every team has legacy conventions. Every assertion was written by a different person under a different set of assumptions. If you keep making those decisions from zero, migration velocity collapses.

The patterns document changed that. It turned one-off discoveries into shared competence.

For us, `testing-patterns.md` became a compact map of the migration territory. It recorded choices such as preferring `userEvent.click()` over `.simulate('click')`, rendering components with narrower fixtures or explicit mock boundaries when shallow rendering had been used to avoid child complexity, replacing CSS-class selector queries with accessible role and name queries, moving assertions from direct instance-method calls to rendered output and user behavior, using `findBy*` or `await waitFor` for async state expectations, and asking whether a test that checks internal props is actually testing behavior or just structure.

Each pattern reduced ambiguity. Each pattern made future decisions more consistent.

Because the document lived in the repository, it was part of the normal engineering workflow. Changes went through pull requests. Reviews happened in context. Tests provided feedback. If a pattern caused trouble, we could fix it, remove it, or narrow it.

That is what makes this form of self-improvement safe: it is treated like any other engineering artifact. It is not sacred, autonomous, or invisible. It is versioned.

Why this version of self-improvement feels good to work with

The key point is not that AI agents are harmless. They are not automatically harmless. Any tool that can act in your codebase needs guardrails.

The key point is that some forms of self-improvement are actively good because they stay inside the normal feedback loops of engineering.

In the migration example, the improvement was bounded to a specific task domain: converting tests from Enzyme to React Testing Library.

A document that says how to migrate button assertions is not the same thing as a system that rewrites its own training objectives. It has a narrow scope, a clear purpose, observable outputs, and testable behavior.

It is also transparent. Anyone can open `testing-patterns.md` and see what the system has learned. It is auditable. We can trace why a migration decision was made because the rule is written down. It is reversible. If a pattern turns out to be wrong, delete it, revise it, or constrain it.

The benefits were concrete: The agent did not become generally smarter. It became more competent at a known task because the environment had learned. That distinction matters.

The practical definition of safe self-improvement

If I were summarizing this for an engineering team, I would say that safe self-improvement is usually external memory plus human review.

It looks like an agent improving its prompts based on reviewed feedback, a workflow updating a checklist after repeated errors, a coding assistant recording a new pattern in a project-specific skill document, a support agent refining response examples after a supervisor approves the change, or a migration tool learning a new transformation rule after tests validate it and a human merges it.

It does not look like hidden self-modification, unreviewed objective changes, autonomous deployment of changes to the model itself, opaque capability increases that bypass inspection, or systems that can alter their own control mechanisms.

That is the line I care about. Software teams can stay on the right side of it if they treat AI improvement as a versioned engineering process rather than magical autonomous evolution.

The takeaway

The question is not whether AI should self-improve. The question is what exactly is improving, how it is measured, where it is stored, who approves the change, and how it can be audited or rolled back.

In our migration, the improvement lived in a Markdown file and in the judgment of the team. It was not hidden, autonomous, or unbounded. It was simply better engineering.

If more AI systems improve this way, through explicit artifacts, human review, and controlled versioning, "self-improving AI" may be one of the least frightening phrases in software.