This is my post for day 25 of the Inkhaven writing retreat.

In a previous post, I described the state of what I called “being a policy”. I contrasted this with habits, heuristics, and plain old conscious deciding. I think a reasonable response to this would be, why would one want to “be a policy”? Isn’t it better to just always be flexibly able to update your decision process to account for new information, et cetera?

Here are some reasons I know of for wanting to be a policy.

Disclaimer: none of what I’m saying here is novel, and all of it has been written about voluminously by others elsewhere, from Aristotle to Wei Dai.

It’s fast

The first reason is that it’s costly or impractical or impossible to calculate out the best action to take in all circumstances. Policies may be hard to follow in the motivation sense, but they are easy to follow in the cognitive sense. However, this reason is shared by habits and heuristics, and doesn’t justify why you’d want to install a policy instead.

To counter other biases

Our minds contain some built-in parts of the architecture that cannot be changed, at least not easily. If these parts influence us to take actions that are systematically working against our values, then we may be able to install a policy to counteract them.

For example, perhaps you’re addicted to checking your phone in the morning. If you leave the phone next to your bed, you will check it. So to counter this you set a policy of leaving your phone outside the room overnight.

A company might use a policy to counter implicit bias. If they’re hiring for a position where most candidates are men, they might require interviewing at least one woman before giving someone an offer.

To be reliably cooperative

There are endless situations where two interacting people would each be better off if they could cooperate on their actions, but where one party defecting leaves the cooperating party worse off (as encapsulated by the prisoner’s dilemma). Being a policy means you can’t change your actions to differ in specific circumstances. So if the policies could somehow know for sure that each other ran the appropriate policy, they could reliably cooperate, leading to better outcomes for everyone.

Unfortunately people cannot literally read each other’s brain states. But due to historical contingencies of evolution, people can discern with some degree of reliability whether other people are running certain policies. Or, to say that in a normal way, you can just kinda tell that Jane is an honest person. Lots of little signals like facial expressions, reaction times, and a consistent history of honesty can provide pretty good evidence of when someone is running honesty as a policy and not just re-deciding to tell the truth every time.

Terminally valuing being a policy

Last but not least: some people want to be a policy because they value that state in and of itself. Some people work hard because they like the results of their work, some people work hard because they want other people to think they’re a hard worker, and some people work hard because they like being a hard worker, period. I personally value having true beliefs, not because they’re useful for my other goals, not because it makes me more useful for other people, but because I value being a thing that has true beliefs. Those other benefits are huge, but they are in addition to my terminal value of having true beliefs. I do have other beliefs, and it is hypothetically possible that they will come in conflict one day such that I would prefer to have false beliefs. But I’m not even sure if I could choose that, because I’ve been running this other policy for so long.