The buzz about WikiSkill

Well, I don't know about you, but WikiSkill has been all over my LinkedIn recently, and I really wanted to know what the deal was. Apparently, this had nothing to do with Wikipedia.

I normally don't read research papers. Maybe it is the fear of having to explore every single node expansion of a knowledge graph, but there are some days when you just want to read something of the mildly enlightening nature. Today happens to be one of those days. So here is my slightly dumbed-down version of this new invention. Google, please don't sue me.

To understand WikiSkill, you first need to know what a "skill" is. A skill is basically a large contextual dump of important workflows and resources that can expand the abilities of agents. I used Claude to make a budget tracker the other day, and thanks to its .xlsx skill, it did a pretty good job!

The problem is that these skills still have to come from somewhere. They are often hand-written (kudos to all the devs using AI to write AI skills), so a natural question is whether agents can improve them automatically. Some clever workarounds centre around giving a model a set of challenges and then using successful and failed trajectories to iteratively optimise the skill. Some examples:

• EvoSkill
• Trace2Skill
• SkillOpt

WikiSkill belongs to this family of ideas, but before getting into why it is interesting, it is worth asking a more basic question: how good is it, really? Here's how it was evaluated.

• Good at maths? YAY
• Good at searching the web? YAYY
• Good at spreadsheets? (Just like Claude and my budget!) YAYYY
• Good at reading a big file and answering questions? YAYYY
• Good at interactive embodied tasks? YAYYYY (I totally understand this term)

They gave these problems to some Qwen, Gemma and Gemini models and looked at how much they improved after skill evolution. Spoiler: imagine you're a small model that evolved a skill yourself. A bigger model will generally still do better. But if you use a skill evolved by a bigger model, you can get some marginal gains! Sharing is caring!

That sharing does come with a few caveats, which they reveal near the end. The transferability of these skills depends on whether the skills themselves capture genuinely general strategies or something more model-specific. Some skills evolved by weaker models can drastically reduce bigger-model performance, whereas others do the opposite. Play with your strengths!

This brings us to the actual contribution of WikiSkill. Instead of treating the skill itself as the only thing worth preserving, WikiSkill introduces a separate "knowledge layer" between raw experience and the final skill:

• Raw Layer — Stores execution traces: each step-by-step interaction, observations, tool calls and final answers.
• Skill Layer — Skills as defined above.
• Wiki Layer — Maintains a structured knowledge base. It contains a patterns/ directory with a bunch of .md files documenting successful strategies and failures.

These layers are then connected by an update loop with four main parts:

• An Inference Agent that executes "rollouts" of the problem.
• A self-explanatory Wiki Maintainer.
• A Skill Proposer that utilises the other two layers to propose updates.
• A Gating and Rollback mechanism that retains the good updates.

If you want the slightly more formal version, the dataset of tasks, $\mathcal D$, is partitioned into training, validation and test tasks. The skill set is initially the empty set, $\emptyset$. On executing task $x_i$, the agent creates a trajectory containing observations $o_t$ and actions $a_t$. They use the score $f(\hat y_i, y_i) \in [0,1]$ to score an individual task, and the average of these scores, $\mathcal R(\mathcal T)$, to evaluate performance over a task split.

The system state is represented by $(S_k, W_k)$, with $S_k$ being the set of skills and $W_k$ the knowledge base at iteration $k$. Both begin empty, $(\emptyset, \emptyset)$, and evolve across iterations. In other words, the agent is not just trying to produce a better skill each round; it is also accumulating a persistent body of knowledge that can inform future skill updates.

They also study what happens when certain agents are restricted from accessing the Wiki, how performance changes across iterations, how compact the resulting skills are, and they examine a strange degradation in the Inference Agent when it has access to the Wiki. Read the paper for more cool stuff.

References

Tang, L., Rashtchian, C., Ferng, C.-S., Tomkins, A., Juan, D.-C., & Vu, T. (2026). WikiSkill: Compiling agent experience into persistent knowledge for skill evolution. arXiv. https://doi.org/10.48550/arXiv.2608.27454