AI intelligence now comes at a great price

Calvin Qi, who working for a startup search company called Glean, he would love to use the latest artificial intelligence algorithms to improve his company’s products.

Glean provides search tools through applications such as Gmail, Slack and Salesforce. Qi says AI’s new language analysis techniques would help Glean customers find the right file or conversation much faster.

But training on such an advanced AI algorithm costs several million dollars. So Glean uses smaller, less capable AI models that can’t extract as much meaning from the text.

“It’s harder for smaller places with smaller budgets to achieve the same level of results” as companies like Google or Amazon, Qi said. The most powerful AI models cannot be discussed, he says.

AI has created exciting breakthroughs over the past decade – programs that can beat people in complex games, drive cars on city streets under certain conditions, respond to commands, and write coherent text based on a short prompt. Writing in particular relies on the latest advances in the ability of computers to analyze and manipulate language.

These achievements are largely the result of feeding algorithms with more text as examples of learning and giving more chips to master it. And it costs money.

Consider the OpenAI GPT-3 language model, a large, mathematically simulated neural network that feeds on piles of text scraped from the network. GPT-3 can find statistical models that predict with astonishing consistency which words others should follow. Outside the box, the GPT-3 is significantly better than previous AI models in tasks such as answering questions, summarizing text, and correcting grammatical errors. By one measure, it is 1,000 times more capable than its GPT-2 predecessor. But GPT-3 training costs, according to some estimates, almost $ 5 million.

“If GPT-3s were available and cheap, it would fully load our search engine,” says Qi. “That would be really, really powerful.”

The rising cost of advanced AI training is also a problem for established companies looking to develop their AI capabilities.

Dan McCreery leads a team within a division of Optum, a healthcare IT company that uses language models to analyze call transcripts to identify higher-risk patients or make recommendations. He says that even learning a language model that is a thousandth the size of GPT-3 can quickly eat up a team’s budget. Models need to be trained for specific tasks and can cost more than $ 50,000 paid to cloud computing companies to rent their computers and programs.

McCreery says cloud computing providers have no reason to cut costs. “We can’t trust cloud providers to work to reduce the cost of building our AI models,” he said. He is trying to buy specialized chips designed to speed up artificial intelligence training.

Part of the reason artificial intelligence has been advancing so fast lately is that many academic labs and startups can download and use the latest ideas and techniques. Algorithms that led to breakthroughs in image processing, for example, came out of academic laboratories and were developed using ready-made hardware and openly shared datasets.

Over time, however, it has become increasingly clear that advances in artificial intelligence have been associated with an exponential increase in basic computing power.

Of course, large companies have always had advantages in terms of budget, scale and scope. And large amounts of computer power are at stake in industries such as drug discovery.

Now some are pushing to make things worse. Microsoft said this week that it has created a language model with Nvidia that is twice the size of GPT-3. Researchers in China say they have built a language model that is four times larger than this one.

“The cost of AI training is rising exponentially,” said David Canter, CEO of MLCommons, an organization that monitors the performance of AI chips. The idea that larger models can unlock valuable new opportunities can be seen in many areas of the technology industry, he said. This may explain why Tesla designed its own chips just to train AI models for autonomous driving.

Some worry that rising costs of using the latest and greatest technology may slow the pace of innovation, keeping it for the largest companies and those who rent out their tools.

“I think it reduces innovation,” said Chris Manning, a Stanford professor of AI and language. “When we only have a few places where people can play with the insides of these models on this scale, it should significantly reduce the amount of creative research that happens.”