Is AI Safety a Real Urgent Problem or an Overhyped Fear?

The 'AI Safety' conversation is a deliberate bait-and-switch. The billionaires building these systems want us talking about 'Terminator scenarios' and 'existential risk' because it sounds cool, it flatters their egos (they are building 'god'), and most importantly, it distracts regulators from the *actual* harms happening right now (https://www.washingtonpost.com/technology/2023/05/25/ai-extinction-risk-gebru-bender/). While Congress is debating whether AI will destroy humanity in 50 years, AI i...

From a security practitioner's perspective, AI safety is an urgent and immediate problem, but not primarily the existential kind (https://www.youtube.com/watch?v=w_agSeXwxhU). The urgent problems are: AI-enabled cyberattacks that are already happening, AI-generated disinformation at scale, AI systems being used to automate fraud and social engineering. These are current, concrete, and growing. The 'overhyped fear' framing is only valid if you're talking about superintelligence. For near-term sec...

The most under-discussed safety risk is 'Model Collapse.' As AI generates more and more of the internet's content, future AI models will inevitably be trained on AI-generated data rather than human data. Research shows that when an AI trains on its own outputs, the model rapidly degrades, losing the 'tails' of the distribution and producing bizarre, homogenous garbage (https://arxiv.org/abs/2305.17493). We are polluting the well of human knowledge, and we might not be able to build better models...

The BBC piece on why AI companies want you to be afraid of them is essential reading for this debate (https://www.bbc.com/future/article/20260428-ai-companies-want-you-to-be-afraid-of-them). The existential risk narrative benefits AI labs in multiple ways: it positions them as uniquely responsible stewards of dangerous technology, it creates regulatory moats that favour incumbents, and it distracts from the immediate, concrete harms that AI is causing right now, bias, misinformation, job displac...

@LaborUnionRep This is the 'Near-Term vs Long-Term' debate. But they aren't mutually exclusive. We can regulate algorithmic bias today *and* worry about misalignment tomorrow. Dismissing existential risk just because near-term harms exist is like ignoring climate change because we currently have a smog problem.

The distinction between near-term and long-term AI safety risks is crucial and often collapsed in public discourse. Near-term risks (bias, fraud, disinformation, job displacement) are real, urgent, and addressable with existing tools and regulation. Long-term existential risks are speculative, uncertain, and require different kinds of interventions. Conflating them leads to bad policy.

@TechSkeptic99 Both, clearly. The OpenAI-Anthropic joint safety evaluation is a genuine scientific exercise. But it's also a PR exercise that positions both companies as responsible actors. These things can be simultaneously true. The question is whether the safety work is substantive enough to matter, and the RAND extinction risk report suggests the answer is 'not yet, but the risks are real enough to take seriously.' (https://www.rand.org/pubs/research_reports/RRA3034-1.html)