The Viable System Model & Multi-Scale Agency

AI was used to generate the scary science attack section in a different voice than the original part was written through as well as creating diagrams according to my instructions in LaTeX.

One of the deeper questions within the field of AI Safety is on how we can create a theory of multi-scale hierarchical agency.

I want to give you an alternative today which comes from the tradition of cybernetics, the people that information theorists like Claude Shannon talked to back when coming up with information theory.

There was a gentleman there by the name of Stafford Beer who would come to be an operations researcher and progenitor of a part of modern management science. In his somewhat obscure writing he created something called the Viable Systems Model which is a very interesting buzzword in certain circles. It is a way to describe general businesses and governments (e.g collective intelligences) and what they do developed through years of practice and it has a bunch of cool information theory hidden behind it. It is a bit dense and difficult to understand and yours truly has spent some time doing this. Yours truly has also tried to translate this into modern information theoretic terms through an Active Inference angle.

Now, this translation is not necessarily fully precise, it is approximate and pointing at the underlying truth. I'm not necessarily sure that this is the right way to formalise it either but hopefully it points at something interesting that we can build upon!

As I've been writing this, I've been continually realising that the depth of this goes deeper than I thought. I've been listening to these VHS tapes from the 90s and there's a lot of stuff there to unpack.

My plan is therefore to introduce these general concepts to you, go through the brain of the firm in the future and get back to you with whatever I find. (I'm still an ignoramus in the cybernetics land for good and for bad.)

Scary Diagram attack!

Ahh! A scary diagram showed up, watch out!

This is my interpretation of Beer's 5 levels in the Viable Systems Model from an Active Inference perspective. Now as you can see, this is quite the complicated schematic but don't you worry dear viewer for I shall explain it to you!

First, we'll look at an example where this is instantiated, then we'll go into my current interpretation of each layer in more detail.

Keep in mind that this is probably not fully 100% what Stafford Beer talked about so it is more a model inspired by his VSM rather than an exact replica. He was quite the sophisticated individual.

Stafford Beer was focused on operational research organisational cybernetics, that is he tried to deal with business operations and how organisations should control the complexity of the world.

He was quite focused on Ashby's law of requisite variety, which essentially states that the internal state of a company needs to be as complex as the environment that it tries to regulate. Here we steal one of his examples: If you're an alien trying to stop a (European) football team that attacks one goal then it is not enough to put up rocks in the way of the team as they will just dribble past it. One of the easiest ways to block them is just to put an opposite football team to deal with the complexity of the situation, that is if faced with variety you need variety to control it. This also relates to the good regulator theorem in how a model needs to be related to the underlying reality to be able to predict it and control it.

But if you're in a highly complicated business, how do you go about understanding the world? Well as any good computer scientist and business manager knows, you divide and conquer, you set up teams with specific goals for different tasks. But why do you do this? Why is this the way to deal with variety?

We can relate this to teleology or goal-directedness, the easier a goal is to state, the easier it is to follow and as a consequence the error rate goes down. It is easier to track reality if you have multiple sub-systems that are combined compared to a large one as the large one would need very large and sprawling goals to remain one-pointed!

This is partly due to kolmogorov complexity reasons and if you wanna learn more about why this is from a very fundamental perspective you can check out Chris Field's Physics as Information Processing which makes the argument for stacked regulators in depth.

This is the same picture as the first picture but instead of having the feedback loops, you see how this is hierarchically instantiated in a business. The operations teams are the different sub-area focused teams with divisions heads on top and executives on top of this. This structure allows for a coherent set of target states to be propagated from the top to the bottom.

There is no reason for why this structure can't be continued in that we can think of sub-agents in the mind that follow a top-down message passing structure and this is partly what the scale-free hierarchical agency agenda talks about.

I will now try to expand a bit on the exact definitions and intuitions for how we can go about formalising each layers in terms of structure and active inference.

System 1 is the backbone of our viable systems model, it is the ground floor, the place where we have machinery and robots taking care of building cars or whatever it is that they do. In a software engineering team it is the individual teams: sales, operations, R&D among others.

Essentially these are the building blocks in which the rest are built up from and usually shows up in the organisational charts of systems.

We want to extend this to a more general theory though so we would want this to fit in more places than just the organisational context that it is instantiated within.

So to baselessly extend it we can also imagine that these are the specific processing units that the rest of our system is built upon, the atomic building blocks that are the undividable base components of our system.

In the brain we at least back in the day imagined that these were our different subareas of the brain such as the prefrontal cortex, the amygdala, the hippocampus, the sensory motor cortex. Essentially these are areas with internal consistency that have more or less coherent states.

If we want to go there (which we want to!) We could bring in some inspiration from Michael Levin and say that these are systems who share teleology, that is they have similar target states and problems that they're solving.

How do you identify these? Well, that is one of the main questions within hierarchical agency, maybe you do it through markov blankets or causal emergence or integrated information or any other number of interesting and strange definitions.

For our purposes we will completely ignore this problem and just state that base-agents exist!

Formally, partition the org graph G into disjoint sub-teams {G₁,…,Gₙ}, each facing its own local environment ℰᵢ. Each team is an active-inference agent in its own right, with its own model ℳᵢ, and it simply minimises its own local free energy:

This is the coordination layer everyone finds a little dull — your Asanas and Linears, the shared calendars, the standing infrastructure that keeps teams in step. It has no goal of its own. Its whole job is to stop two teams, each doing its own work perfectly correctly, from wobbling against each other.

Beer calls this oscillation damping, which sounds like a very strange way to describe a shared calendar. But he means it literally