Mankind faces problems. Innovations too pose challenges. It is paradoxical. Advanced mathematics is built on imaginary numbers. Blackholes have validated many laws, and still, they remain inscrutable. Similarly, AI too pages certain challenges. The basic challenge is how a superintelligent system of future would be controlled.

OpenAI has laid down Preparedness Strategy Framework in December 2023. It intends to adopt a scientific approach to assess catastrophic risk of any advanced AI system. The document describes processes to track, evaluate, forecast and protect against such risks.

If AGI is realized, it will require oversight.

AI sector has developed a concept of super alignment to deal with AGI. It is a holistic approach. It goes beyond technical specifications. It wants to consider the societal impact and ethical issues.

So far alignment was restricted to alignment of AI systems to human values (during training phase). Super alignment refers to continuous alignment throughout the life cycle of AI systems — including deployment, adaptation and evolution.

OpenAI suggests that a less potent LLM should serve as a proxy for human oversight of the more potent superintelligent AI.

OpenAI forecasts superintelligence could be a reality in the next 10 years.

OpenAI looked at how GPT 2 developed five years ago could supervise GPT-4, which is the latest LLM.

AI could be an existential threat to mankind. It is a dooms day scenario. It distracts from short-term risks of the present-day AI systems. Such risks include misinformation, bias, copyright violations and expensive compute. Industry should not be fixated upon doomsday scenarios. All such talk is highly hypothetical. The issue is how to deal with technology that currently exists. Of course, future possibilities cannot be ignored. But as Andrew Ng puts it — ‘Is there any engineering discipline where much attention is on hypothetical problems, rather than actual problems?’

AI is transformative. It has the potential to do much good.