Why Should Corrigible Agents Favor the Present?

A corrigible agent understands that it is flawed and seeks to empower its principal to correct those flaws. Many of the intuitive examples of corrigibility happen over a short period of time: the principal gives a command, and then the agent follows it. When there are contradictory commands given at different times, it seems like the agent should follow the most recent command, but it's unclear exactly why corrigible agents favor the present over the past. In this post, I will explore this open question from Max Harms’ CAST sequence, reproduced verbatim: Corrigibility clearly involves respecting commands given by the principal yesterday, or more generally, some arbitrary time in the past. But when the principal of today gives a contradictory command, we want the agent to respect the updated instruction. What gives the priority of the present over the past? As a simple example, which I will return to throughout the post, consider the following case. On Monday, the principal tells the agent to buy apples every week. On Tuesday, the principal tells the agent to buy oranges instead of apples. A corrigible agent should respect the newer command, but why? Is it just a preference that we have—that is, a corrigible agent could respect Monday's command, but it's more desirable to have an agent that gives priority to the present? Or is giving priority to newer commands an intrinsic part of what it means to be corrigible? I will argue that the answer is somewhere in between. Corrigibility doesn't seem to uniquely privilege the present: a corrigible agent can respond with some delay. However, corrigibility requires that the principal retain the ability to correct the agent. It is incorrigible for an agent to latch onto a command from the past principal and ignore attempts at correction. The present principal's authority might be best understood as a consequence of the ability to correct the agent rather than an independent feature of corrigibility. Can agents just be corrigible to the present principal? My first instinct is that agents are just corrigible to the present principal, since this at least explains what's going on in the fruit-buying example. However, an agent that's solely corrigible to the present principal would do nothing! All commands take some time to communicate, so agents only ever hear commands from the past. An agent that only cared about commands given by the present principal wouldn't respect any commands. Even if a time slice is slightly broader than one instant, so that being corrigible to the "present principal" includes the time it takes for a command to be communicated, such an agent would still be useless. It wouldn't follow the command even 30 seconds after it was given, since then it was given by the past principal. It's tempting to say that the agent can just assume the present principal still endorses their command because they haven't repealed it or issued a contradictory command. This starts to feel like incorrigible behavior, though. A corrigible agent doesn't try to guess what the principal wants and act accordingly, since estimating the principal's will leads to the problem of fully updated deference. Listening to commands isn't the best way to learn what the principal truly wants, so the agent might instead deconstruct the principal's brain to figure out precisely what they want. Solutions that rely on guessing whether the principal endorses a particular command, or more broadly, guessing what the principal wants, lead to incorrigible behavior. Differences between the past and present principal Each difference between the past and present principal is a possible way to answer the original question, since the answer must be grounded in some difference between the past and present principal. If they were literally indistinguishable, there would be no basis for prioritizing one over the other. Existence The most straightforward difference is that the present principal exists and the past principal doesn't, but there are two problems. First, it might not be true! It relies on a controversial view about the nature of time. B-theorists, for example, would disagree with the assertion that the past principal doesn’t exist. It would be strange if resolving the mundane fruit-buying case depended on disproving B-theory. Second, it implies that past commands don't have authority. If the past principal doesn't exist and existence is required to have authority, then no past command could have authority. The agent can't assume that the principal still endorses those commands for reasons explained above. Information Usually, the present principal knows more than the past principal. They've observed everything that the past principal has, plus whatever happened in the meantime. Perhaps newer commands deserve priority because they're more informed. For example, we might imagine that the command to buy oranges is because the principal was made aware of a sale that they didn't know about when th