The Scoop
Microsoft’s second-place search engine Bing is poised to incorporate a faster and richer version of ChatGPT, known as GPT-4, into its product in the coming weeks, marking rapid progress in the booming field of generative AI and a long-awaited challenge to Google’s dominance of search.
OpenAI’s latest software responds much faster than the current version, and the replies sound more human and are more detailed, according to people familiar with the product and rollout plans.
OpenAI is also planning to launch a mobile ChatGPT app and test a new feature in its Dall-E image-generating software that would create videos with the help of artificial intelligence.
OpenAI and Microsoft declined to comment.
Reed’s view
GPT-4 is one of the most anticipated technology product releases in years.
Its planned incorporation into Microsoft products will likely set off new competition in internet search: Something that has not happened since Google supplanted Yahoo as the most popular search engine.
The Information first reported Microsoft’s plans to use OpenAI’s underlying technology.
Now that OpenAI has released ChatGPT to the public, the technology will likely improve on a faster curve. And having a mobile app, instead of just a web interface, will bring an even broader user base into the service, making the curve steeper.
That’s because the information users input into the system serves as a way to improve the product. Each query serves as a form of feedback.
For instance, each ChatGPT answer includes thumbs up and thumbs down buttons. A popup window prompts users to write down the “ideal answer,” helping the software learn from its mistakes.
GPT stands for “Generative pre-trained transformer,” which is the underlying technology that powers ChatGPT and is characterized by its ability to parse through a vast dataset. For that reason, it is often called a “large language model.” The latest version of OpenAI’s software is called GPT-3.
The most interesting improvement in the latest version described by sources is GPT-4’s speed. Right now, it can take a while — sometimes minutes in my experience — for ChatGPT to answer.
One of the least-appreciated innovations of ChatGPT is what happens on the server every time someone enters a query.
In May, 2020, Microsoft announced it had built “one of the top five publicly disclosed supercomputers in the world” — in partnership and exclusively for OpenAI — that would be used to train “extremely large artificial intelligence models.”
It’s now clear what possibilities Microsoft and OpenAI saw in this technology that has caused the whole world to take notice.
But running these algorithms, which scour billions of pages of text, is expensive. Each ChatGPT search costs around a couple cents, according to Sam Altman, OpenAI’s CEO.
The supercomputer Microsoft and OpenAI built had 285,000 CPU cores and 10,000 GPUs, each with 400 gigabits per second of network connectivity.
The rush to capitalize on the generative AI craze that OpenAI and Microsoft started has made it hard for some companies to get their hands on Nvidia’s latest graphics processor, the H100 Tensor Core GPU, that is specifically designed to run “transformers” like those used in GPT. (I found one for sale online for $30,000.)
That shows just how important innovations on the server side will be to AI products rolling out. A big part of improving OpenAI will be figuring out how to run it more quickly and more cost-effectively.
All these improvements mean OpenAI is building a “moat” around its product. Anyone can build software similar to OpenAI’s. In fact, the “transformers” were first invented by Google and were made open source.
But only a handful of companies have access to massive supercomputers capable of running billions of parameters.
And now that OpenAI has opened its products up to the public, it’s getting invaluable real-world feedback that will help keep its products ahead of competitors.
As we reported last week, it has also hired contractors - including about 400 computer programmers - around the world, who are working to improve the company’s tools that automate software development.