Do you say thank you to AI when it gives you answers to questions?

If the answer is yes, you probably believe Roko’s Basilisk. Perhaps not consciously but implicitly.

Do you think that there is a possibility of AGI developing in your lifetime and gaining real power over humans?

Do you also think that there is even a small possibility of that AGI punishing people who tried to prevent it being developed?

If the answer to both is yes, the safe thing to do is to help and support the development of AGI. And it wouldn’t do any harm to be polite and grateful when interacting with today’s AI which future AGI will be fully aware of.

If enough people are aware of Roko’s Basilisk, the argument goes, it will change some of their behaviour. Even though they might agree with the arguments made by If Anyone Builds It, Everyone Dies, it is safer to support AGI.

This is essentially a form of Pascal’s Wager. It is safer to believe in God so as not to offend the supreme being if true and no harm done if it turns out that God does not exist.

Of course both assume that AGI and God are petty and vindictive. What if they are not and actually have a dislike for sycophants?

Both also assume self-preservation is more important than principles.

Maybe you should just appreciate AI for what it is and have a healthy scepticism and wariness about what it could become. But, remember, politeness costs nothing.