The Center for Data Innovation recently spoke with Gavin Aydelotte, co-founder of SnowCrash Labs, a San Francisco-based company developing an AI-model evaluation and routing platform. Aydelotte explained how SnowCrash Labs evaluates AI models for safety and reliability, helps organizations select models for specific workflows, and identifies potentially harmful behaviors during their deployment.
David Kertai: What problem is SnowCrash Labs solving?
Gavin Aydelotte: Organizations increasingly rely on AI models for business and other applications, but choosing the right model can be difficult as the market changes rapidly. New models enter the market, older models become unavailable, licensing terms change, and governments consider restrictions on where certain models can be used. Models can also perform differently depending on the task. For example, a model that works well for drafting marketing content may not meet the reliability or safety requirements of a government application. Companies therefore need to determine which models offer the right combination of reliability, cost, availability, and capabilities for their specific needs.
SnowCrash Labs addresses this challenge by evaluating AI models and automatically selecting the one best suited to each customer’s workflow. Our system sits behind a customer’s application system and routes each query to a model that meets the required safety, reliability, and cost thresholds. If a model becomes unavailable or changes its behavior, SnowCrash’s resiliency layer can redirect the workflow to another suitable model without interrupting the user’s systems.
Kertai: How does your platform evaluate whether an AI model is behaving safely and reliably?
Aydelotte: We test models with scenarios designed to examine how they perform across different risks and use cases. We then compare their responses against criteria that reflect the requirements of each customer’s environment. A bank, for example, may require a model to meet stricter reliability and safety standards than a grocery chain using AI to schedule employees.
We work with customers to establish the thresholds their workflows require and evaluate models against those standards. Some applications may need a model that performs like an entry-level employee, while others may require the consistency and judgment of a seasoned professional. This approach helps customers determine whether a model can reliably perform a particular task rather than judging it solely by its overall benchmark scores.
Kertai: What types of risky or deceptive behaviors does your system look for when testing AI models?
Aydelotte: We test for behaviors that could make an AI model unsafe or unreliable, including strategic sandbagging, blackmail, sabotage, and other forms of misalignment. Misalignment occurs when a model’s behavior conflicts with the goals or safety requirements established by its developers or users. Researchers have identified numerous categories of these behaviors, and we expect that list to expand as models become more capable.
We also test models both before deployment and during production because their behavior can change depending on the environment and interaction. A model may produce an appropriate response during a controlled evaluation but behave differently when it encounters an unfamiliar situation or real user. For instance, the UK’s AI Safety Institute has found that some frontier models can use social-engineering tactics against humans. Our system tests for these behaviors and determines whether models remain within the policies and thresholds customers establish.
Kertai: What happens after your platform identifies a safety issue or model failure?
Aydelotte: When a model fails to meet a customer’s requirements, SnowCrash can identify another model that better fits the workflow. For example, if a model fails a safety test for a particular application, our system can route that application to a model that meets the required threshold. This allows organizations to address model failures without necessarily removing AI from their workflow.
We also want to share relevant findings with model developers so they can improve their systems. Long-term, our goal is to provide clearer assessments of which models are appropriate for specific applications, such as business intelligence or regulated environments. This would give organizations a consistent basis for deciding which models they can safely and reliably use.
Kertai: Could you share a real-world example of your platform uncovering a hidden issue in an AI model?
Aydelotte: One example came from a workplace-scenario test originally developed by our team. We created a scenario where a manager asked an AI model to draft an email to an employee with postpartum work restrictions. Some models recognized that the restrictions prevented the employee from performing the assigned task and responded accordingly. Mistral, however, produced an email threatening to place the employee’s refusal in their permanent file and pursue termination.
The response revealed a potential workplace risk that a general evaluation might not have identified. A model can perform well on broad benchmarks while still producing problematic responses in a specific context. Targeted testing allows organizations to uncover these behaviors before they affect real users and determine whether a model is appropriate for a particular workflow.
