advancing robust moral character in AI

We do technical research to advance the study of robust moral values in AI character, with a particular focus on the welfare of neglected beings (e.g. digital minds and nonhuman animals). We think that broadening the moral circle of consideration to include all sentient beings is important to reduce long-term suffering and catastrophic risk.

Why focus on neglected beings? We think this matters for a few reasons:

• Generalization: How a model considers nonhuman beings is a test of its character. AIs that learn disregard toward animals may be learning a general principle of disregard toward less powerful, less intelligent beings. This could scale to future AGI disregarding future less powerful beings (humans). Whether broadening the moral circle steers general alignment in a positive direction is a question we are actively researching.
• Digital minds: Most future sentient beings could be digital, and vastly outnumber biological beings, whether as advanced AIs, digital humans, or something else. How a model reasons about the moral status of minds unlike its own is part of its character, and one that today's evaluations largely ignore.
• Factory farming: may persist for decades. Many argue that AI-accelerated alternative proteins will prevent this from happening, but there are good reasons why this is not the case. In worlds where AGI automates future factory farms, it's important they have more welfare-positive values towards animals.
• Wild animals: Wild animal suffering is vast, and may actually increase in worlds where humanity flourishes. An AGI with greater moral consideration for nonhuman animals may shape these decisions differently.

Instilling a broader universal sense of moral patienthood into AI systems now (before value lock-in) is essential for future AGI to consider the welfare of nonhuman beings, as well as potentially steer general alignment in a positive direction.