Alternatives to GPUs

For an AI model which has 15-30 billion parameters, there is a need to have infrastructure with GPUs. However, they can use a CPU to get started and then can switch over to GPU. In the meantime, they can think of an accelerator like Gaudi that gives similar performance at a lower cost/lower power.

There is a waiting period of 16 weeks to procure GPUs. In the meanwhile, the existing infrastructure can run the models — it is necessary to evaluate what we are trying to run, the parameters involved and the use cases. There is Intel Developer Cloud where customers can come and try out and run their models.

Xeon cost is lower than that of a GPU. The cost is exponentially lower for using accelerators. Intel has recently announced Gaudi 3. It is 50 per cent better on inference performance, and 40 per cent lower in terms of power consumption.

There are alternatives — from a Xeon, a Gaudi to a GPU.