AI of future, a decade hence, could be super-intelligent. OpenAI has formed a team led by Ilya Sutskever (chief scientist) to develop ways to steer and control such a system. Ilya Sutskever is a Russian-born Canadian computer scientist who cowrote the seminal paper Attention Is All You Need in 2017. He cofounded OpenAI.

AI of future may not be benevolent, thus making it necessary to exercise control over it. The system could go ‘rogue’.

At present, we use for a technique to align AI such as reinforcement learning from human feedback. This depends on human beings ability to supervise AI. How will they supervise AI that will be smarter than them?

We, therefore, need superintelligence alignment. The team formed will have access to 20 per cent compute the company has secured to date. The team will address the technical challenges to control superintelligent AI over the next four years.

They want to build ‘a human-level automated alignment researcher’. The aim is to train the AI systems using human feedback. Alternatively, one AI system can be trained to evaluate other AI systems. The ultimate aim is to build AI that can do ‘alignment research’ — alignment research refers to AI systems that achieve desired outcomes do not go off the rails. The assumption is AI can do the controlling job better than the humans can.

The future AI systems should be better aligned with humans. The alignment research will be reviewed by human researchers.

However, such alignment have potential to scale up biases and vulnerabilities. Is alignment related to engineering only? Still it is worth a try.

The team will share the results with others.