What Can AI Do Well Today & What Is Still Mostly Hype?

Here's my honest breakdown after 18 months of real deployment: What works brilliantly, writing assistance, summarisation, first-draft generation, code completion, data extraction from documents. What's overhyped, autonomous agents completing multi-step business processes without supervision, AI 'reasoning' in high-stakes decisions, and anything that requires consistent factual accuracy. The PwC survey finding that 56% of CEOs haven't realised any revenue or cost benefit from AI investments tells...

The Stanford AI Index 2025 data is instructive here (https://news.stanford.edu/stories/2025/12/stanford-ai-experts-predict-what-will-happen-in-2026). AI now exceeds human performance on many standardised benchmarks, but those benchmarks were designed for humans, not for the messy, ambiguous, context-dependent problems that define real work. The gap between 'benchmark performance' and 'real-world reliability' is the central unresolved problem in applied AI. Until we have better ways to measure re...

What AI does brilliantly today: Code translation and legacy system modernisation. We use LLMs to translate 30-year-old COBOL banking systems into modern Python or Rust. It's tedious, pattern-based work that humans hate and AI excels at. What is pure hype: 'Autonomous AI Hackers.' The idea that an LLM can autonomously breach a secure, modern enterprise network is science fiction (https://www.blackhat.com/us-23/briefings/schedule/#the-reality-of-ai-in-offensive-security-32145). They are great at w...

My personal test: I give any new AI tool a task I already know the answer to, and I check not just whether it gets it right, but how it handles being wrong. The tools that say 'I'm not sure' or 'you should verify this' are the ones I trust. The ones that give confident wrong answers with no hedging are the ones I'm most cautious about in production.

@AcademicObserver The benchmark problem is real, but I'd add another dimension: the security and reliability of AI systems in adversarial conditions. In my field, AI-powered threat detection is excellent, until someone deliberately tries to fool it. Adversarial robustness is almost never discussed in the hype cycle, but it's the first thing we ask about in enterprise deployments.