Using the Burr UI to monitor is not the only way. You can integrate your own by leveraging lifecycle hooks, enabling you to log data in a custom format to, say, datadog, langsmith, or langfuse. - Source: dev.to / about 2 months ago

Langfuse (YC W23) | https://langfuse.com | Full-Time | Berlin, Germany | on-site | LLM Observability and Analytics Langfuse is open source [1] observability and analytics tool for LLM applications — think Amplitude and Datadog for LLM apps. Our users use Langfuse to understand what happens in production and use our insights to improve their applications. We have built a number of... - Source: Hacker News / 8 months ago

Langfuse makes tracing and analyzing LLM applications accessible. It is an open-source project under MIT license. - Source: dev.to / 9 months ago

- Conversational simulation is an emerging idea building on top of model-graded eval” - AI Startup Founder Things to consider when comparing options: “Types of metrics supported (only NLP metrics, model-graded evals, or both), level of customizability; supports component eval (i.e. Single prompts) or pipeline evals (i.e. Testing the entire pipeline, all the way from retrieval to post-processing)” “+method of... - Source: Hacker News / 9 months ago

Humanloop (YC S20) | London (or remote) | https://humanloop.com We're looking for exceptional engineers that can work at varying levels of the stack (frontend, backend, infra), who are customer obsessed and thoughtful about product (we think you have to be -- our customers are "living in the future" and we're building what's needed). Our stack is primarily Typescript, Python, GPT-3. Please apply at... - Source: Hacker News / over 1 year ago

https://humanloop.com/ Find the prompts users love and fine-tune custom models for higher performance at lower cost. - Source: Hacker News / over 1 year ago