The AI space is rapidly evolving alongside model capability, and while it’s long been established that evals are essential, there’s a surprising lack of them focused on human safety. Handling this correctly is a delicate issue and can’t be done by AI or software engineering teams alone. Safety eval datasets should also be validated by human subject matter experts that are empowered to thoroughly review the training data, model behavior, and raise ethical concerns before models are deployed to end-users. Having already been tasked with turning policies into plain language, existing policy teams should be an integral part of prompt generation and validation as well.
At most companies, regardless of their size or payscale, this is not the case today. Pressure to launch before competition is often at odds with thorough safety validation, complicating the cross-functional relationship between safety teams, product, and engineering. When companies optimize for speed over safety, or simply aren’t aligned on what the real priority is, it can lead to unintended negative behavior or baffling lapses in model judgment with serious risk for real world harm.
To address this, Pelidum has created safety-oriented eval datasets that can be used to regularly benchmark and compare model behavior across multiple providers.
Get in touch to learn more about how Pelidum can help stress-test your AI models.
Interested in learning more?
Book some time with us for more information on what we do and how we can help your organisation