VAST Data has built software to find, catalog, and capture file and object data in other suppliers’ storage and data management systems, bringing it into VAST’s private, AI-focused universe.
VAST says SyncEngine is “a universal data catalog and high-performance onboarding solution.” It is a so-called universal router, providing a real-time, searchable catalog across file systems, object stores, and SaaS apps like SharePoint, Google Drive, and Salesforce. This makes billions, even trillions, of files and objects in petabyte-to-exabyte data sets instantly discoverable.
Co-founder Jeff Denworth said: “The future of AI belongs to those who can harness all of their data, not just what’s conveniently available. Data sprawl is the silent killer of enterprise AI strategies, and SyncEngine ends that problematic era.” The company declares: “SyncEngine eliminates the friction of discovering and mobilizing distributed unstructured datasets and enterprise SaaS platforms, so organizations can simplify their infrastructure and move faster from raw data to real-world AI outcomes.”
It seems self-evident that AI models or agents can provide better responses to an organization’s user requests if they have access to as much of the organization’s data as possible. We commented on VAST’s need for this capability here. Ideally, there needs to be an all-encompassing and single pool of data, virtual or physical. VAST wants it to be physical, and stored in its own disaggregated, shared-everything storage system and software environment, the VAST AI OS as it’s now called. This includes VAST’s InsightEngine and DataEngine for feeding and supporting AI pipelines.
The company has now built its own data vacuum cleaner, capable of discovering and ingesting data stored in competitors’ systems. It says the biggest constraint to using AI at scale “isn’t just model complexity or GPU supply, but the data itself – scattered across outdated file and object systems, siloed inside enterprise SaaS applications, and invisible to AI pipelines.” Invisible to VAST AI pipelines, but Dell, HPE, NetApp, Komprise, Hammerspace, and enterprise SaaS app suppliers like Salesforce may well argue it’s not invisible to their AI pipelines.
The SyncEngine utility’s discovery and migration replaces external and standalone cataloging and migration tools, such as those from, we understand, Datadobi, Data Dynamics, Diskover, Komprise, and other suppliers. The discovered files and objects are cataloged in the VAST database where they can be searched at the hundreds-of-trillions level. It says the catalog has multiple namespaces and “deep metadata indexing.”
The cataloged files and objects can be migrated to the VAST Data system and kept synchronized with the sources. Bi-directional data flow is included, implying the VAST system can feed data to the discovered source systems. The migration does not need external extract, transform, and load (ETL) procedures nor custom scripts. That functionality is built-in and includes data integrity verification and automated recovery.
AI pipelines on the VAST system can be fed with data from this migration, with it being chunked and vectorized for RAG and made available for agentic workflows, including any LangChain-based application.
VAST claims that SyncEngine offers the world’s fastest data migration services, “delivering record-breaking speeds from ingest to insight,” but without quoting numbers. There is unlimited ingest throughput, it says, by scale-out node addition, with network line rate performance.
What target systems are covered by SyncEngine? VAST mentions legacy POSIX and S3-compatible sources, as well as enterprise apps such as Confluence, Microsoft SharePoint, Google Drive, and Salesforce.
It asserts that SyncEngine: “replaces expensive standalone tools and complex open source options, dramatically simplifying data cataloging and migration, with integrated intelligence. Customers benefit from a consolidated stack with no additional licensing fees, reducing complexity, operational burden, and total cost of ownership.”
Denworth said: “Legacy IT created silos, and we’re tearing them down. Whether your data is buried in on-prem systems or hidden in SaaS apps, SyncEngine makes it all accessible, visible, and valuable. We’re giving customers a direct path from where their data lives today to where AI transformation begins, inside the VAST AI Operating System.”
SyncEngine is available at no charge to VAST customers.
Find out more at VAST’s blog site and product webpage.
Bootnote
VAST’s SyncEngine appears not to be connected to the SyncEngine open source data synchronization and API integration manager.
