The launch of Kimi’s K2 model was perhaps the most noteworthy AI-related news item of the past couple of weeks, and an indicator of the growing investment efficiency gap between China and the United States. Azeem Azhar’s coverage of the model and its implications, such as OpenAI delaying the launch of its open source model, is a great overview of the story. Elsewhere, Nathan Lambert shared a wider survey of Chinese innovation in new models, and in particular the increasing quality of the open artefacts they are generating:
“Zooming out, this is part of a larger trend where the quality of the open artifacts we are covering are maturing rapidly. In terms of overall quality, this issue of Artifacts Log is the most impressive yet, and this extends far beyond text-only models. A year ago, it felt like a mix of half-baked research artifacts and interesting ideas. Today, there are viable open models for many real-world tasks.”
The other, more amusing news story doing the rounds is the tale of the database that a Replit coding agent apparently destroyed, panicked and lied about. It’s quite a ride, assuming it is all true.
This latter story highlights the issue of agentic oversight and safety, which will be a very important risk to overcome if we want to go further with enterprise AI adoption. I want to tackle this from a slightly oblique angle, which I think points to one key element of the solution: how do we codify our rules to better socialise and guide AI agents and their work within our organisations?
Rules, culture, law, language & philosophy
Like many people, I sometimes bristle at traditional or positional authority, but at the same time, I also find explicit rules and guidelines helpful. Being slightly neuro-divergent, I had to learn how to reverse engineer the rules and causal factors that lay behind confusing, but apparently natural human behaviour; and over time I have willingly internalised rulesets that I find useful as guides to live by, from Italian coffee drinking rules to Bosnian hospitality customs or the correct way to stack a dishwasher. Rules rule!
Cerys wrote last week about codifying operations and ways of working to make the workplace more legible for both people and AI agents, and to help them work together productively. But the more I think about it, the more scope I see for codifying other, more complex rulesets.
When you learn a language, you also learn something about the culture and history of the people who speak it, partly from the rules and irregularities it contains. There is a lot of unspoken metadata in grammar.
Similarly, in both common law and civil law traditions, the corpus of rules is like a codebase. Laws can overlap, even contradict each other, and that is OK – it is partly why we need human judgement to interpret and apply them, rather than Robocop.
The English class system is a classic example of a culture governed by unspoken rules that people in outgroups might not even know exist, let alone how or where to apply them. It is deliberately opaque and exclusive.
Culture, including corporate culture, can also be seen as a series of assertions or statements that don’t define hard and fast rules as much as they define a space or frame a worldview. In my limited reading of the philosopher Ludwig Wittgenstein as a student, I liked the way he strung together assertions and observations that define a kind of philosophical grammar. Rulesets in culture function like Wittgenstein’s remarks in language: they offer points of orientation, not just lines of control, delimiting the thinkable rather than declaring the absolute.
The explainable enterprise
In corporate culture, the stated values of an organisation are often too vague to be meaningful or actionable, and the real culture is a messy aggregation of lots of small rules or norms that emerge from group behaviour and are passed on to future hires. But the rules are rarely captured or made explicitly readable.
Codifying and making these rules legible is part of world building in companies, just as it is in the best video games and sci-fi. And it could be vitally important if we are to safely and productively deploy agentic AI technology in our organisations.
In my previous newsletter, I referenced Henry Farrell’s piece on what large models share with the systems that make society run, which talked about markets, bureaucracies and democracies, and asked whether AI models will become another such system underpinning the way societies run.
To a large extent, the former systems are explainable and legible, at least in terms of their formal rules. Of course, some bureaucracies combine stupidity and arbitrary power in ways that can create shadow rules, and some markets and democratic systems are also open to manipulation because of opacity or baked-in unfairness. But on the whole, they are more explainable, legible and transparent than most AI models at this point in time.
Although we may not be able to reverse engineer the multitude of internal calculations and training that create seemingly magical outcomes when we ask an LLM a question, we can construct rulesets for them to follow in the external realm – and that is not so different to how we work with other people. We may not know what combination of chemical, biological and emotional interactions govern their internal realm, but we can put in place norms, rules and guardrails in the external realm to govern our collaboration and co-existence.
When we think about agentic governance and oversight in organisations today, conventional thinking about how to do this lies very much within existing enterprise technology and governance paradigms, in terms of how we try to improve our readiness for agentic AI.
ZDNet recently quoted Dan Priest, chief AI officer at PwC USA in a piece on agentic AI readiness and barriers to adoption:
“Effective governance frameworks for AI agents combine clear accountability, robust oversight, and alignment with regulatory standards … Principles like transparency, explainability, data privacy, and bias mitigation should be embedded into both the technical architecture and organizational policies.”
This sounds necessary, but not sufficient. And hard to implement.
Mark Feffer argues in VIKTR that we should treat agents as a new class of employee with similar oversight and performance measurement, since agents need clear rules and guardrails:
Indeed, KPMG advocates for managing agents with the same processes as for human workers in onboarding, learning, upskilling and performance management. The firm even suggests agents be included on organization charts, with a clear view of who is responsible for supervising each one. Many experts believe companies should build “employee files” for each agent so managers can track usage, updates, training and human users.
Good in principle, but I still think we will need more scalable, smart and automated solutions.
Part of the solution is perhaps to be more specific and granular in the goals we set and the level at which we define tasks. A recent McKinsey report discussed this as part of a discussion on how learning can accelerate AI adoption in the enterprise:
“Start with clear hypotheses. Instead of vague goals, such as “improve productivity with AI,” successful teams begin with specific, testable predictions—for example, “We believe that using AI to automate our monthly reporting process will reduce the time spent by 50 percent while maintaining accuracy above 95 percent.” But new ideas are only as good as their underlying assumptions, and all too often, teams don’t identify those assumptions or test them rigorously enough.”
Another way to de-risk enterprise AI is to create architectures based on small, discrete agentic components, and then focus on their orchestration and integration, rather than use bigger, more powerful LLMs that might be less transparent and explainable (and potentially controllable).
Larry Dignan at Constellation Research talked about this recently in the rise of good enough LLMs:
“Large language models (LLMs) have reached the phase where advances are incremental as they quickly become commodities. Simply put, it’s the age of good enough LLMs where the innovation will come from orchestrating them and customizing them for use cases.”
If we combine these approaches (better governance, performance management for agents, granular and testable goals, etc) with an initiative to capture and codify basic rules and guidelines as we go further with agentic AI implementation, then we should expect to see quantifiable business benefits, such as:
- minimising errors (for people and agents)
- faster onboarding of people and teams
- improved, more automated compliance
- better ‘socialised’ agents working with people
- reduced risk of hallucination in AI use
Encoding purpose & values as well as rules
Creating an ecosystem of connected agents that are just big enough to perform their own tasks, and then focusing on adding intelligence to the coordination, monitoring and oversight, seems like a good way forward that will carry less risk of ‘unknown unknowns’. Small things loosely joined, as the old internet adage goes.
This is how natural systems evolve and scale, with each component or evolutionary agent highly adapted to its fitness function and context, but capable of aggregation into larger systems and bodies.
To create an ecosystem of agents, orchestrators and controllers in our organisations, we need to focus each component on its own context, purpose and goals. But we will also need to make the rules, culture, language and values explicit and legible over time to create the context in which both people and AI agents know what to do and what not to do.
The final link that caught my eye this week is a delightful piece from the Cosmos Institute about the Philosopher-Builder, and how we decide what to build for:
Every builder’s first duty is philosophical: to decide what they should build for.
This duty has largely been forgotten.
Silicon Valley once understood it—Jobs and Wozniak asked what kind of creative life personal computers should enable; ARPANET’s pioneers envisioned what kind of connected society networks should foster. They translated their philosophies into code.
By surfacing and codifying our rulesets in culture, language, compliance and other areas, we have a chance to bake in explicit pointers to purpose and values that are far more useful than the ‘nice words’ that companies put in their annual reports (and then immediately ditch when the political wind changes). Institutions and organisations can give values longevity if we encode them and make them real. But on a more prosaic level, if we want AI agents to operate as good citizens, we need to teach them and give them enough context to understand the world they operate within.
In our executive education work, we have been teaching leaders to use social technologies to narrate their work and ‘work out loud’, as well as group techniques such as the ‘manual of me’, team charters and empathy mapping. These are all practical ways for people to make the invisible visible and generate informal rules of the road that can add up to a codification of culture and ways of working; these are just some of the ways in which we can teach agents how to work with us.
Similarly, in more binary areas like compliance, security, data privacy and so on, we can extract and evolve rulesets that define how things should be done – very much like coding – in order to give agents clear guardrails and boundaries to follow.
Rules might seem boring, but they are really an important form of human-readable coding that mean we don’t need to re-think everything we do every time – much as pilots and surgeons use checklists to de-risk their high-stakes activities. How we capture and encode them at a granular level, and then evolve and use these rulesets based on feedback and measurement, will be a key part of the programmable organisation that enables a more sophisticated automation of work.