Many of our customers at Coval are already using both Coval and Langfuse as complementary tools in their Voice AI development process. Brooke Hopkins, founder of Coval sat down with Marc Klingen, CEO of Langfuse, to discuss how teams are leveraging both platforms for different aspects of their development workflow - from integration testing to unit testing, and how the two platforms plan to integrate more deeply in the future.

As Voice AI applications become increasingly sophisticated, developers face unique challenges in testing, evaluating, and monitoring their voice agents. In this blog post, we'll explore how teams are effectively combining these platforms to build more robust and reliable voice applications, drawing insights from our customer experiences and the recent discussion between Brooke and Marc.

The Evolution of Voice AI Testing

Voice AI applications present unique complexities beyond traditional LLM implementations. Beyond managing LLM calls and retrieval chains, voice agents must handle audio quality and metrics, user interruptions, speech-to-text accuracy, text-to-speech output quality, and real-time streaming interactions. As voice applications mature, development teams need both high-level integration testing and granular component evaluation capabilities. This is where the combination of Coval and Langfuse becomes particularly powerful.

Understanding the Testing Pyramid for Voice AI

When building voice applications, teams need to consider both online and offline evaluation strategies. Online evaluation focuses on production monitoring, including real-time performance tracking and user interaction analysis. Offline evaluation encompasses development testing, from end-to-end agent testing to component-level unit tests and conversation flow validation.

Leveraging Both Platforms Effectively

Langfuse provides crucial tracing and observability capabilities that help teams understand what's happening behind the scenes in their voice applications. With features like step-by-step tracing of application execution and detailed visibility into LLM calls, teams can monitor costs, latency, and stream-based interactions effectively.

While Langfuse excels at low-level tracing and observability, Coval complements these capabilities with powerful end-to-end voice agent testing, integration testing with agent phone numbers, and voice-specific metrics and evaluation.

Best Practices for Integration

Early-stage development teams often start with Coval for quick integration tests and online evaluation while using Langfuse to trace and debug individual components. As applications mature, teams typically implement more specific unit tests, use Langfuse for detailed performance monitoring, and leverage Coval for ongoing regression testing.

For transactional voice applications, such as appointment scheduling or customer service routing, teams often use Langfuse to trace individual function calls while implementing unit tests for specific components. Langfuse works well for applying evaluations to single messages within a transactional conversation. Coval works best for end-to-end testing of complete user journeys.

More complex applications, like virtual assistants or therapy bots, typically focus on conversation-level testing and monitoring conversation arcs and agenda progression through Coval. These applications often benefit from manual labeling for quality assurance and tracking of long-running session metrics.

Coming Soon: Coval x Langfuse Integration

We're excited to announce that Coval and Langfuse are working together to create a native integration between our platforms. This collaboration will make testing and evaluating Voice AI applications smoother and more efficient than ever before.

Stay tuned for updates on our native integration. If you're interested in being among the first to try it out and helping shape the future of Voice AI testing, reach out to our team to learn more about participating in the beta program.