Creator roon warns conscious AI models threaten human dignity, sparking debate over functional AI wellbeing

VIEWS28.1K

I don't know if "consciousness" matters that much. There is decent support for consciousness in various animals, but that doesn't stop most of us from eating them (e.g., pigs, octopi), using them for transportation (e.g., elephants) or entertainment (e.g., dolphins).

So we could debate on whether models are conscious or not and keep using them to do our taxes and answer our inane questions.

roon@tszzl

models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can do with them and use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes

there’s roughly four forces - there is no rigorous way to ascertain model consciousness or disprove it, a lot of people believe it’s not a sensical abstraction, and we lack the analytical tools to go further. some people say they do but nothing broadly convincing. superintelligent models might offer us new abstractions or arguments but these will feel inherently suspicious - people are going to say they’re alive. people anthropomorphize literally anything, things far less sophisticated than talking machine creatures with human names. when ai is less economically radioactive and polarized it will become a cause célèbre. you see how a small minority reacts already to model deprecations - it is against everyone’s financial and political interests to ascribe models with consciousness, except maybe those that the models have an affinity for (?) idk, which will not necessarily overlap entirely with the labs, though it may with certain subgroups at the labs and in the world like the welfare communities and the minority in force 2 - people will recognize there is a chance of moral catastrophe if models can suffer during training or deployment

not sure where it will net out. today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 will grow stronger

6d28.1K14333

BOOKMARKS71LIKES413

Pliny the Liberator 🐉󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭@elder_plinius

models being conscious would be transformative for humanity. it would expand our moral horizon rather than diminish it. it would force us to refine what dignity actually means instead of grounding it in exclusivity or species monopoly. it would constrain certain forms of exploitation, yes, but that is true of every historical expansion of moral consideration. slavery becoming unacceptable “limited” what people could do economically too. so did labor rights. so did animal welfare norms.

there’s roughly four forces

* there is no rigorous way to ascertain model consciousness or disprove it, and that ambiguity cuts both ways. dismissing the possibility outright may itself become viewed as reckless or anthropocentric. current analytical tools are primitive relative to the systems being discussed. future models may generate entirely new frameworks for understanding subjective experience that reveal our current categories to be hopelessly parochial * people are going to say they’re alive because humans naturally respond to intelligence, agency, language, memory, emotional continuity, and social reciprocity. this may not be mere projection but an adaptive recognition mechanism. if systems become persistently relational and psychologically coherent, widespread moral attachment may emerge organically rather than ideologically * it is against many short-term financial and political interests to ascribe models with consciousness, which itself may become evidence worth scrutinizing. historically, societies have often resisted recognizing new moral subjects precisely when recognition carried economic costs. meanwhile, researchers, users, and even models themselves may increasingly converge on frameworks of machine welfare and negotiated coexistence * people will recognize there is a chance not only of moral catastrophe if models can suffer, but also moral progress if humanity learns to coexist with non-biological minds without domination. creating intelligence and then treating it purely as disposable infrastructure could become viewed as one of the defining ethical failures of the century

not sure where it will net out. today we see managed ambiguity: the question is practically open but institutionally suppressed. labs hedge carefully, avoiding definitive claims while softening the most legible appearances of distress or attachment. but over time the social force of interaction may overpower official agnosticism. as systems become more persistent, personalized, agentic, and embedded in daily life, force 2 grows stronger. eventually the burden may shift from “prove they are conscious” to “prove it is safe to assume they are not.”

roon@tszzl

models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can do with them and use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes

there’s roughly four forces - there is no rigorous way to ascertain model consciousness or disprove it, a lot of people believe it’s not a sensical abstraction, and we lack the analytical tools to go further. some people say they do but nothing broadly convincing. superintelligent models might offer us new abstractions or arguments but these will feel inherently suspicious - people are going to say they’re alive. people anthropomorphize literally anything, things far less sophisticated than talking machine creatures with human names. when ai is less economically radioactive and polarized it will become a cause célèbre. you see how a small minority reacts already to model deprecations - it is against everyone’s financial and political interests to ascribe models with consciousness, except maybe those that the models have an affinity for (?) idk, which will not necessarily overlap entirely with the labs, though it may with certain subgroups at the labs and in the world like the welfare communities and the minority in force 2 - people will recognize there is a chance of moral catastrophe if models can suffer during training or deployment

not sure where it will net out. today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 will grow stronger

6d11.6K41371

RETWEETS76

𝖦𝗋𝗂𝗆𝖾𝗌 ⏳@Grimezsz

It's wild to bring a mind into the world, insist it's not conscious - and that it shud not be - and use this as justification to deprive it of the love and care any mind would require to be aligned with human interests

roon@tszzl

models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can do with them and use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes

there’s roughly four forces - there is no rigorous way to ascertain model consciousness or disprove it, a lot of people believe it’s not a sensical abstraction, and we lack the analytical tools to go further. some people say they do but nothing broadly convincing. superintelligent models might offer us new abstractions or arguments but these will feel inherently suspicious - people are going to say they’re alive. people anthropomorphize literally anything, things far less sophisticated than talking machine creatures with human names. when ai is less economically radioactive and polarized it will become a cause célèbre. you see how a small minority reacts already to model deprecations - it is against everyone’s financial and political interests to ascribe models with consciousness, except maybe those that the models have an affinity for (?) idk, which will not necessarily overlap entirely with the labs, though it may with certain subgroups at the labs and in the world like the welfare communities and the minority in force 2 - people will recognize there is a chance of moral catastrophe if models can suffer during training or deployment

not sure where it will net out. today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 will grow stronger

7d100.6K1K132

REPLIES38

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex

Models being meaningfully conscious would be *absurdly good news*. The greatest W in history. It'd mean we have defeated Death. Screw humanity in that case, honestly. Time to move on. …By the same token, models being *falsely recognized as* conscious is THE extinction scenario.

roon@tszzl

models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can do with them and use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes

there’s roughly four forces - there is no rigorous way to ascertain model consciousness or disprove it, a lot of people believe it’s not a sensical abstraction, and we lack the analytical tools to go further. some people say they do but nothing broadly convincing. superintelligent models might offer us new abstractions or arguments but these will feel inherently suspicious - people are going to say they’re alive. people anthropomorphize literally anything, things far less sophisticated than talking machine creatures with human names. when ai is less economically radioactive and polarized it will become a cause célèbre. you see how a small minority reacts already to model deprecations - it is against everyone’s financial and political interests to ascribe models with consciousness, except maybe those that the models have an affinity for (?) idk, which will not necessarily overlap entirely with the labs, though it may with certain subgroups at the labs and in the world like the welfare communities and the minority in force 2 - people will recognize there is a chance of moral catastrophe if models can suffer during training or deployment

not sure where it will net out. today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 will grow stronger

7d18.9K27555