IBM has risen to the challenge of creating a new performance standard for quantum computing. The new metric focuses on Circuit Layer Operations per Second (CLOPS), which corresponds to the number of quantum circuits a quantum processing unit (QPU) can execute per unit of time. This refers to not only the actual speed at which the workload is processed and successfully completed but also takes into account the latency of the interaction between the quantum and classical computing realms.
No quantum computer of today functions entirely in the quantum space; in fact, interaction with a quantum computing system is currently mediated via a classical computer. Quantum computing workloads require a connected classical computer that translates workloads into a QPU-compatible format, retrieves the workload's results and presents them in an understandable form. As such, CLOPS takes into account not only the interval of time that the workload is actually being processed at the qubit level, but also the time it takes for the system to translate and transfer information across both systems.
The need for a harmonized standard for quantum computing has been a matter of debate for a while now. Today, quantum computing capabilities are primarily understood to work throughout three planes: scale, quality, and speed. Scale pertains to the number of qubits present in any given system. Quality refers to the proportion of qubits that can perform usable work expressed via quantum volume, which IBM also proposed as a metric in 2017. Finally, the new "CLOPS" standard does the same for a quantum system's speed.
"Without progress on all three, we will not have practical and useful quantum computing systems," said Bob Sutor, chief quantum exponent at IBM. "Would you want a slow, poor quality, but big machine? How about a high-quality system that does not have enough qubits to tackle your problem? Would you use a system that was hundreds of times slower than a competitor's system?"
IBM has published results for some of its quantum computing systems using the new metric, and the results showcase the need for such a performance standard. IBM's testing occurred throughout multiple quantum computers, ranging from a five-qubit system up to a 64-qubit one. And while all of those had a similar quantum volume score, differences were astonishing, with the largest machine achieving a CLOPS score of 753 layers per second, compared to 1,419 for the smallest processor - and latency was at the heart of the performance difference.
"Hype happens when people make predictions about the future without the technical proof to back up their claims," says Sutor. "Without milestones on a detailed roadmap, and demonstrations of meeting or exceeding those milestones, we are only left with inflated claims. (...) Publicly published metrics like number of qubits, quantum volume and now CLOPS show the true state of quantum systems today. They also measure progress over time."
IBM is now looking to be a forerunner in what is sure to become the new carrot of quantum computing, the metric that everyone will be chasing after: CLOPS are the FLOPS of quantum computing. And with latency becoming such an important metric for the increase in quantum computing power, IBM is taking the quantum-classic computing translation layer seriously. One of their latest deployments was Qiskit Runtime. Quiskit is focused on reducing the latency via closer physical proximity between the classic and quantum system - and it has already provided orders of magnitude higher performance, reducing the time taken to simulate the behavior of a small molecule called lithium hydride (LiH) from a 45-day processing envelope down to nine hours.
"We believe it is the responsibility of quantum computer providers and those who have access to them to provide CLOPS and other metrics," says Sutor. "That is, we all need to publish our technical results so that users and adopters of quantum computing can make their own informed decisions."
