Formally Stating the AI Alignment Problem

The development of smarter-than-human artificial intelligence poses an existential and suffering risk to humanity. Given that it is unlikely we can prevent and may not want to prevent the development of smarter-than-human AI, we are faced with the challenge…

Form and Feedback in Phenomenology

Now that we’ve covered phenomenology’s foundations and its methods, we’re almost ready to begin addressing AI alignment from a phenomenological perspective, but before we can start talking about AI or how to align it we need to build a clear understanding of what AI is…

Akrasia is confusion about what you want

Many people I know might describe this as a case of akrasia…

Introduction to Noematology

Last time we performed a reduction of the phenomenon of conscious self experience and through it discovered several key ideas. To refresh ourselves on them:

• Things exist ontologically as patterns within our experience of them.

AI Alignment and Phenomenal Consciousness

In the initial feedback I’ve received on my attempt to formally state the AI alignment problem, the primary objection I’ve heard is that I assume any AI worth aligning will experience qualia. In particular, some think we have to worry about aligning AI that…

Suffering and Intractable Pain

I’ve been writing a lot about AI alignment lately, so let’s take a break from that to talk about a lighter subject: suffering.

Suppose you contract the rare and as yet untreatable disease boneitis. The doctors estimate you have one year to live. In…

Avoiding AGI Races Through Self-Regulation

Summary

The first group to build artificial general intelligence or AGI stands to gain a significant strategic and market advantage over competitors, so companies, universities, militaries, and other actors have strong…

Evaluating Existing Approaches to AGI Alignment

My read of the AI safety space is that there are currently two major approaches to AGI alignment being researched: agent foundations and agent training. We can contrast them in part by saying the ultimate goal of the agent foundations program is…

Computational Complexity of P-Zombies

I’ve claimed that p-zombies require exponentially more resources than phenomenally conscious agents and used this to justify that AGI will necessarily be phenomenally conscious. I’ve previously supported this claim by pointing to a related result in integrated…