Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human 
Feedback

Conitzer, Vincent; Freedman, Rachel; Heitzig, Jobst; Holliday, Wesley H.; Jacobs, Bob M.; Lambert, Nathan; Mossé, Milan; Pacuit, Eric; Russell, Stuart; Schoelkopf, Hailey; Tewolde, Emanuel; Zwicker, William S.

アイテム詳細

登録内容を編集ファイル形式で保存

一時保存へ追加

タグ情報を表示リリース履歴を表示詳細要約

公開

書籍の一部

Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback

Authors

Conitzer, Vincent
External Organizations;

Freedman, Rachel
External Organizations;

/persons/resource/heitzig

Heitzig, Jobst
Potsdam Institute for Climate Impact Research;

Holliday, Wesley H.
External Organizations;

Jacobs, Bob M.
External Organizations;

Lambert, Nathan
External Organizations;

Mossé, Milan
External Organizations;

Pacuit, Eric
External Organizations;

Russell, Stuart
External Organizations;

Schoelkopf, Hailey
External Organizations;

Tewolde, Emanuel
External Organizations;

Zwicker, William S.
External Organizations;

URL

https://proceedings.mlr.press/v235/conitzer24a.html
(出版社版)

https://doi.org/10.48550/arXiv.2404.10271
(プレプリント)

フルテキスト (公開)

There are no public fulltexts stored in PIKpublic

付随資料 (公開)

There is no public supplementary material available

引用

Conitzer, V., Freedman, R., Heitzig, J., Holliday, W. H., Jacobs, B. M., Lambert, N., Mossé, M., Pacuit, E., Russell, S., Schoelkopf, H., Tewolde, E., & Zwicker, W. S. (2024). Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback. In R., Salakhutdinov, Z., Kolter, K., Heller, A., Weller, N., Oliver, J., Scarlett, & F., Berkenkamp (Eds.), Proceedings of the 41st International Conference on Machine Learning (pp. 9346-9360). Cambridge, MA: MRL Press.

引用: https://publications.pik-potsdam.de/pubman/item/item_29847

要旨

Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, such as helping to commit crimes or producing racist text. One approach to fine-tuning, called reinforcement learning from human feedback, learns from humans’ expressed preferences over multiple outputs. Another approach is constitutional AI, in which the input from humans is a list of high-level principles. But how do we deal with potentially diverging input from humans? How can we aggregate the input into consistent data about “collective” preferences or otherwise use it to make collective choices about model behavior? In this paper, we argue that the field of social choice is well positioned to address these questions, and we discuss ways forward for this agenda, drawing on discussions in a recent workshop on Social Choice for AI Ethics and Safety held in Berkeley, CA, USA in December 2023.