The Center for Data Innovation spoke with Krishna Subramanian, co-founder and COO of Komprise, a data management platform based in San Francisco. Subramanian explained the challenges of working with unstructured data and discussed the future of data management and mobility.
This interview has been edited.
Morgan Stevens: What inspired you to start Komprise, and what is the company trying to achieve?
Krishna Subramanian: Back in 2014, my co-founders and I saw that unstructured data was growing like crazy and there wasn’t much innovation to address this challenge and opportunity. We talked to customers whom we had worked with at previous companies and the sentiment was that they were all drowning in unstructured data. Not only did they not know much about this data but they didn’t know what to do about it. It was expensive to store and backup and a lack of visibility was creating risk from both security and compliance and loss of market opportunity for reusing data over time.
IT leaders told us that not only did they need insights about their data but they wanted to take action based on these insights. This was the impetus for the founding of Komprise: we saw an enormous market opportunity for a no lock-in, independent way to manage unstructured data for both cost savings and long-term value. We realized that we needed to do something vendor-agnostic since most enterprises have silos of data that were increasingly spread across disparate storage technologies and clouds. Our customer advisors also emphasized that they didn’t want a solution that sat in front of their data. So the vision from day one was to deliver deep analytics on all customer file and object data in storage regardless of vendor and the ability to easily search for and tag files and create policies and plans and take action on this data through the full lifecycle of data: copy, tier, migrate and delete. Don’t get in the hot data path and don’t disrupt user or application access to data at any time. Today, we’re excited to bring unstructured data management to meet the next seminal challenge and opportunity: managing data for AI. This requires visibility, governance and automated workflows—all of which Komprise is perfectly suited to address.
Stevens: Can you explain the technology behind Komprise’s platform?
Subramanian: Komprise scales to handle billions of files and tens of petabytes of data with a lightweight distributed architecture. This scale-out architecture is important to highlight, but from a customer’s perspective, it’s all about ease of use and ease of deployment. Komprise is a SaaS service that is easy for customers to adopt and scale, and we provide a hybrid model where virtual machines can be deployed at data centers if the source storage is at a data center. This simplifies deployment and scaling while delivering the best performance and security.
The three core components of the Komprise platform and architecture are the Global File Index (GFI), our patented Transparent Move Technology (TMT), and Elastic Data Migration.
The GFI is a scalable, distributed database containing records for each file in each customer’s entire multi-site deployment. The GFI delivers holistic visibility into all data in storage and powers our Deep Analytics capability, which allows users to drill down into multiple dimensions of metadata to find just the files that interest them. Additionally, Komprise allows customers to provide custom tags for the files to further enrich the data and deliver precise search across data silos.
TMT provides a zero-disruptive experience to users when their data is tiered or archived to secondary storage. We accomplish this through the use of standard symlinks, which point to the destination yet the user accesses their data in the same location as they always have. TMT also moves the data in native form to secondary storage such as in the cloud, delivering two ways to access the data: as a file at the original location or directly in the cloud for use in cloud-native services, such as data lakes and AI applications.
Elastic Data Migration achieves migration between primary storage systems at 25 to 27 times the speed of typical migration tools. Our migration capabilities are continually expanding, with more options for customers as announced in our latest product update. As well, Komprise ensures data integrity and reliability through the following features: auto retry if network or storage is unavailable; migrate with all file permissions, access control and data integrity intact; manage chain of custody reporting with checksums and integrity reporting per file; and thwart ransomware attacks by not using network access to cloud storage during migration.
Stevens: How does Komprise adapt its data management and mobility solutions to cater to the specific needs of different industries?
Subramanian: Komprise is a solution that applies to any industry with petabyte-scale data environments. Our top industries are healthcare, life sciences, higher education, public sector, media and entertainment, oil and gas, and manufacturing. While the types of files may vary across industries, the need for data management is horizontal across industries. However, certain things are industry-specific, especially when looking into the contents of the data or creating data workflows to industry-specific applications. Komprise provides APIs and smart data workflows to simplify such industry-specific data processing. Customers can connect to third-party solutions, such as file content analysis, machine learning for analyzing data stores on specific file types like lab instrument data or PII data for a healthcare organization, or for genomic analysis in the cloud.
Stevens: What differentiates Komprise from other data management and mobility platforms?
Subramanian: Komprise is an analytics-driven SaaS platform to manage and mobilize all your unstructured data. Komprise is different because we are standards-based, storage-agnostic, and always keep data in your control in native format without lock-in. Our solution delivers a central interface to easily analyze and search unstructured data across silos to save money and serve departmental needs for data access and analytics. Komprise customers save on average 70 percent on storage, backup and cloud costs with intelligent data tiering. They can move to the cloud 27 times faster with smart data migration and see a dramatic reduction in time spent preparing data for analytics workflows with our Global File Index.
Stevens: How do you foresee the future of data management and mobility, and how is Komprise positioning itself for that future?
Subramanian: Data will continue to grow, especially unstructured data. Data is now a top strategic asset: organizations will need to optimize infrastructure, storage and data protection costs. Equally important is the need to quickly generate value from data and use it for positive business outcomes. With the rise of AI and machine learning, organizations that can easily find, enrich and glean intelligence from their data will have a competitive advantage.
Our 2023 industry survey on unstructured data management found that IT organizations are looking for solutions that enable monitoring and alerting for capacity issues and anomalies, policy-based automation such as moving data to cold storage or confinement for deletion and self-service access for line of business IT teams and researchers.
Komprise makes it easy for organizations to optimize data costs while extracting data value. We are continuing to add capabilities that make it easier to securely find the right data and feed it to AI/ML, enrich data with contextual information, and know what data to keep and where to keep it for optimizing costs, carbon footprint and sustainability. Komprise has also introduced new capabilities for self-service by authorized departmental users and our investments in delivering integrated data and storage metrics are facilitating better real-time decision-making to keep data secure and available for all stakeholders.