LMSYS’ Chatbot Arena is widely regarded as one of, if not, the most reliable open benchmarks for LLMs. Real users provide prompts to chatbots and then blindly pick the best response. The only drawback is that the leaderboard is restricted to the most popular models and, even then, it can take a while for new models to be added. This is understandable given the considerable ongoing costs associated with... - Source: Hacker News / 4 months ago

I've recently updated the LLM Explorer (https://llm.extractum.io) - a directory that houses more than 13,000 LLMs. For those, who are new to the project: it offers a user-friendly interface for searching and filtering models by categories, comparing them and swiftly identifying superior alternatives (which is notably more convenient and faster than browsing HuggingFace). Source: 6 months ago

It's been some time since I announced LLM Explorer. I've taken into account all of your comments and feedback on the project, made the necessary fixes, and incorporated new features. Now, I'm excited to present the new version (v2.0) of the service: https://llm.extractum.io. Source: 7 months ago

The original link to the directory is llm.extractum.io. Thank you! Source: 10 months ago

Using the Burr UI to monitor is not the only way. You can integrate your own by leveraging lifecycle hooks, enabling you to log data in a custom format to, say, datadog, langsmith, or langfuse. - Source: dev.to / about 1 month ago

Langfuse (YC W23) | https://langfuse.com | Full-Time | Berlin, Germany | on-site | LLM Observability and Analytics Langfuse is open source [1] observability and analytics tool for LLM applications — think Amplitude and Datadog for LLM apps. Our users use Langfuse to understand what happens in production and use our insights to improve their applications. We have built a number of... - Source: Hacker News / 7 months ago

Langfuse makes tracing and analyzing LLM applications accessible. It is an open-source project under MIT license. - Source: dev.to / 9 months ago