Even if others are less responsible you can still make things worse

Reposted from shortform because I think this is important enough to be a post. A few ways staying in a technology development race still makes things worse, even when there are less responsible actors around: Creating a sense of artificial urgency, which causes people who are competitive and results driven to work harder and cut more corners Providing a guiding star on which technical directions are promising (e.g. reasoning models, CoT, RLHF) Legitimising irresponsible behaviour as valid response to strong competition ("it's a competitive market, this was bound to happen") Entangling the motivations and incentives of the "more responsible" parties with the success of the technology in question, making any warnings seem weak and hypocritical ("if it's so bad, how come you're still doing it"?) Providing an adversary or target to aim for/surpass (similar to artificial urgency) Diffusing media/governance/public attention away from the "more irresponsible" parties, since it's now a multi horse race Driving hype, attention, and funding to the field via your credibility and achievements Becoming trapped in a "we must win" mentality that causes you yourself to cut corners Discuss