Langfuse
Helicone AI
LangSmith
LangChain
Openlayer
Braintrust.dev
Portkey
LastMile AI
Testim
Selenium
Cypress.io
mabl
RainforestQA
Katalon
TestRail
Testsigma
Langfuse is an open-source LLM engineering platform designed to empower developers by providing insights into user interactions with their LLM applications. We offer tools that help developers understand usage patterns, diagnose issues, and improve application performance based on real user data. By integrating seamlessly into existing workflows, Langfuse streamlines the process of monitoring, debugging, and optimizing LLM applications. Our platform's robust documentation and active community support make it easy for developers to leverage Langfuse for enhancing their LLM projects efficiently. Whether you're troubleshooting interactions or iterating on new features, Langfuse is committed to simplifying your LLM development journey.
Testim gives you the flexibility to create and manage tests your wayโcodeless, coded, or both. - Quickly click through UI scenarios, add validation steps, create reusable groups, or export to code and edit in your IDE. - Run suites or test plans in parallel, across multiple browsers, and report results. - Configure validations, modify conditions, or insert custom code or data to test any scenario. - Connect to your CI, version control, collaboration, bug capture, or 3rd party testing grids. - Development Kit - export your tests to code or write them in your IDE using the Testim JavaScript library, API commands, and example code. - Self-healing - Smart Locators learn with each run to stabilize tests, maintaining test stability even when code changes. - Root Cause Analysis - errors are aggregated giving you quick insight into where tests are failing. View rich data including HTML/DOM and before/after screenshots see how attributes changed
Langfuse TestimTestim is recommended for development teams, QA engineers, and product managers who face frequent test maintenance challenges and those who work within Agile and DevOps environments. It is especially beneficial for organizations seeking to automate frontend tests for web applications and needing a tool that scales with the complexity and size of their products.
Based on our record, Langfuse should be more popular than Testim. It has been mentiond 28 times since March 2021. We are tracking product recommendations and mentions on various public social media platforms and blogs. They can help you identify which product is more popular and what people think of it.
In this project we will build a Python banking assistant agent using Strands Agents and make it observable and continuously evaluated using Langfuse โ step by step. - Source: dev.to / 11 days ago
Langfuse is the open-source standard for LLM observability. It traces every LLM interaction โ prompts, completions, latency, token usage, cost โ and provides the tooling to debug, evaluate, and optimize LLM applications in production. Think of it as "Datadog for LLM calls" with a focus on prompt engineering workflows. - Source: dev.to / 30 days ago
You're monitoring production traffic. You need Langfuse / Phoenix / Helicone / Braintrust for that. Online eval is a different problem class: implicit feedback, drift detection, hallucination rates on your data, not on HellaSwag. - Source: dev.to / about 1 month ago
Gateway or proxy attribution. A reverse proxy in front of the model-provider API records the request, computes the cost, and exposes per-customer breakdowns. Open-source options include Helicone, LiteLLM, Langfuse, and OpenLLMetry. Hosted equivalents serve as the AI cost observability layer for teams that want centralized visibility: LangSmith, Datadog LLM Observability, Arize Phoenix. Adds a network hop.... - Source: dev.to / about 1 month ago
Same approach works with Langfuse, Phoenix, Braintrust, or your existing OTel pipeline โ the metadata.userId pattern is the universal part. - Source: dev.to / about 2 months ago
Automated Testing: Platforms like Testim and Selenium use AI to automate the testing process, reducing the time and effort needed for manual testing. - Source: dev.to / almost 2 years ago
Does anyone have any experience with testim.io? A In my company, they created a focus group/team to research this tool and I am part of the team? If you have 1st hand experience using it, please share your feedback. Source: about 4 years ago
Last month, I started exploring solutions for the above problems and one day landed with testim.io. Testim not only automates the flow but also automates code generation for the test scripts. Letโs take a look together. - Source: dev.to / about 4 years ago
I'm seeing the comments and a lot of great suggestions. Curious if anyone has had any experience using testim.io, and how that compares to the most popular one in this thread test cafe. Source: about 5 years ago
Helicone AI - Open-source LLM Observability for Developers
Selenium - Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that.
LangSmith - Build and deploy LLM applications with confidence
Cypress.io - Slow, difficult and unreliable testing for anything that runs in a browser. Install Cypress in seconds and take the pain out of front-end testing.
LangChain - Framework for building applications with LLMs through composability
mabl - Agentic Test Automation Platform