The News
Startup VAST Data has come up with a new way for businesses to employ AI models more efficiently and securely, which also leverages corporate information in internal products like chatbots and other tools.
The new product, announced on Tuesday and dubbed the VAST InsightEngine, could help smooth the way for more companies to take advantage of large generative AI models.
The software aims to solve a key pain point in one of the most common business use cases for AI: chatbots that help employees find and make sense of the enormous troves of data their companies have amassed.
VAST’s service doesn’t require companies to do anything with its information. It’s able to read all forms of data and permissions, and then make it understandable to AI models.
“The enterprise doesn’t need to do anything differently than what they did before,” VAST founder and CEO Renen Hallak told Semafor. “They can start leveraging these new abilities without breaking their compliance.”
The InsightEngine is designed to work with Nvidia software called NIMS, which leverages the hardware acceleration in Nvidia’s AI chips, making chatbots respond faster.
“It makes it possible for every enterprise to be able to engage AI, transform their data into their own digital intelligence and connect it into a flywheel that sits on top of VAST and Nvidia,” Nvidia CEO Jensen Huang said in a recorded interview with Hallak.
In this article:
Know More
When it works properly, corporate chatbots can be enormously helpful. Employees at a business consultancy, for instance, can ask one to read past reports in the company’s databases and then spit out a draft for a new client, using the correct format and style. The most common method of accessing that data is known as “Retrieval Augmented Generation,” or RAG.
The problem is that most companies don’t keep their data in one place, which creates challenges for making it quickly accessible in one AI prompt. And each employee at a company has permission to access different slices of that information.
There are existing methods for handling these problems, but because of the nature of generative AI, there are risks. For instance, employees at some companies have used complex prompts to “trick” AI models into accessing company data that is prohibited — a so-called “prompt injection attack.”
VAST InsightEngine acts as part traffic cop and part translator. To AI models, a company’s enormous mix of different data types appears as a single source. But even if an AI model falls victim to a prompt injection attack, it will be unable to access data that a user isn’t supposed to see.
Step Back
VAST is an example of a class of companies that have been able to profit from the rise of AI by providing valuable technology and services used to make complex algorithms run more efficiently.
While chipmaker Nvidia is the best known of these companies, there are hundreds of lesser known players building businesses around the many complicated aspects of AI development.
Some, like VAST, saw the AI wave coming many years ago and began investing early in a future that they knew would one day arrive.
Hallak told Semafor he founded VAST when the most impressive thing AI could do was recognize cat videos on the internet. “But it was already clear that the way it could recognize cats is not because we thought of a brand new algorithm, but because we gave the old algorithms a lot more data and a lot more processing power,” he said.
Hallak realized that one of the big bottlenecks in AI development was the way data was stored. It needed to be organized differently and accessed more quickly.
The change required new hardware that didn’t yet exist, so Hallak began building software that wouldn’t actually be usable for at least a couple of years. “Everything came together in the 2018 and 2019 time frame when we could assemble the hardware pieces and our new architecture,” he said.
At first, VAST’s customers were hedge funds, life science institutes, autonomous driving companies and government agencies. Today, the potential customers span all industries.
On Tuesday, VAST also announced it was starting a new organization called Cosmos, which aims to bring together AI companies to share information aimed at advancing the technology.
The initial membership includes major players like Nvidia, Cisco, xAI, CoreWeave, Deloitte and Core42, among others.
Reed’s view
VAST is a good reminder of the way technology really works in modern times.
Long gone are the days when someone like Thomas Edison could tinker in a workshop and come up with world changing inventions. Technology like AI takes thousands of little inventions and breakthroughs that all add up something monumental.
You need people like Hallak who see a technology trend and start thinking about where the bottlenecks are that might slow it down. Then they need to have the conviction to devote all of their time to removing that bottleneck.
If they get venture capitalists to believe in their vision and trust in their ability to execute, they are proven right only years later.