The Scoop
Lightning AI, the Nvidia-backed AI infrastructure startup, is launching a sui generis GPU rental marketplace that allows customers to compare processors from hyperscalers like Google alongside neoclouds like Lambda, the company exclusively told Semafor.
Companies shopping for GPUs have traditionally had to choose between the two. Cloud providers like Microsoft, Google, and AWS offer graphics units for rent and machine learning services that help streamline training and deployment, but typically at a higher cost than the emerging neocloud sector, which includes firms like CoreWeave and Lambda. While cheaper, neoclouds don’t have the same additional tooling as Big Tech firms, requiring customers to build their own infrastructure on top. Besides offering a side-by-side comparison — which customers could previously piece together with some legwork — Lightning provides its own machine learning platform for training and scaling models, mirroring what hyperscalers provide and closing the gap for neoclouds.
“The goal has always been to be the ‘iOS’ of AI development platforms,” chief product officer Priya Shivakumar told Semafor about the Tuesday launch.
With graphics processors in high demand to support AI development, the offering intends to make the technology more accessible and simplify the sales process. The startup claims its platform could cut AI costs by 70% by presenting more affordable GPU rental options and making it easy to switch between units, reducing the amount of paid-for but unused capacity.
The marketplace operates within Lightning’s existing AI development platform rather than as a separate service. Users can access the technology for free, starting with the lowest level subscription tier. Small teams typically pay $140 per user per month, and enterprises receive custom pricing.
Step Back
Big Tech could benefit from exposure to smaller, fast-moving software developers that wouldn’t typically start their workloads with hyperscalers. Meanwhile, aggregating the GPU ecosystem into one platform could more significantly highlight the price disparity between the options and drive customers towards the cheaper neoclouds. It also gives new credibility to those nascent companies that have thus far been a risky bet for investors.
Know More
The 60-person startup began gaining recognition around 2021 after its PyTorch Lightning deep learning framework rose to prominence in the research community. To date, it has been downloaded 200 million times, Shivakumar said.
The company wouldn’t disclose its valuation, but PitchBook pegs it at around $315 million. Investors include Cisco, JPMorgan, and Nvidia — which offers a GPU rental service similar to what Lightning launched, called DGX Cloud Lepton. Lightning’s marketplace differs by providing more resources for developers, Shivakumar said.
“It’s not created any kind of conflict or anything like that,” she said, adding that the companies routinely collaborate, including working on a PyTorch extension called Thunder.
Notable
- Among neoclouds, CoreWeave stands out as highly competitive with hyperscalers, because of its focus on GPUs and advanced capabilities — including automated health checks and triage systems — that, according to Artemis Research’s June report, Big Tech providers offer only in rudimentary form.
- During CoreWeave’s first quarter earnings call, CEO Michael Intrator addressed the disparities between existing cloud services and his own firm: “These clouds were built to host websites and run SaaS applications, not to run high-performance training and inference workloads. The exponential growth and success of our customers’ AI products are driving demand for our differentiated services.”