Helicone takes the simplest possible approach to LLM monitoring: it's a proxy. Change your OpenAI base URL from api.openai.com to oai.helicone.ai, add your Helicone API key as a header, and every LLM request is logged — latency, tokens, cost, prompts, and completions. No SDK integration, no code changes beyond a URL swap. - Source: dev.to / 24 days ago

You're monitoring production traffic. You need Langfuse / Phoenix / Helicone / Braintrust for that. Online eval is a different problem class: implicit feedback, drift detection, hallucination rates on your data, not on HellaSwag. - Source: dev.to / about 1 month ago

For many teams, especially those starting out or with simpler needs, commercial solutions like Portkey, Helicone, OpenPipe, or LiteLLM Proxy offer off-the-shelf capabilities that cover many common proxy use cases (caching, logging, cost tracking). NeuroLink itself can be seen as an SDK that complements these, allowing you to integrate with them or build similar features on top. - Source: dev.to / 3 months ago

TL;DR: Go with Langfuse if you want open-source and self-hosted. Pick Helicone if you want the fastest setup (2 minutes, no SDK). Stick with LangSmith if your stack already runs on LangChain. And if your org already pays for Datadog, their LLM module slots right in. - Source: dev.to / 4 months ago

Hey HN, we're Justin and Cole, the founders of Helicone (https://helicone.ai) or self-deploy with our new fully open-source helm chart (https://helicone.ai/selfhost). Yet even with detailed traces, probabilistic systems are notoriously hard to debug at scale. So, we released evaluators (either via LLM-as-judge or custom Python evaluators leveraging the CodeSandbox SDK - https://codesandbox.io/docs/sdk/sandboxes).... - Source: Hacker News / over 1 year ago

The web service supports generating embeddings from OpenAI and Ollama AI models. It also provides a fallback for users without access to AI models running on a remote server through PostgresML. - Source: dev.to / over 1 year ago

That's outside of the database, though. This is more like what I had in mind -- I just found it: https://postgresml.org/. - Source: Hacker News / about 2 years ago

Some excellent tools were created to represent these tasks "naturally" in SQL and even let most of the computation happen inside the database. PostgresML is a great example. It's built above PostgreSQL and provides a set of functions that allow you to train and use machine learning models with SQL. Here's how you can train a classification model for the classic handwritten digit recognition problem:. - Source: dev.to / over 2 years ago

PostgresML | You know Postgres. Now you know machine learning – PostgresML. - Source: dev.to / over 2 years ago

You can swap in almost any open-source model on Huggingface. HuggingFaceH4/zephyr-7b-beta, Gryphe/MythoMax-L2-13b, teknium/OpenHermes-2.5-Mistral-7B and more.If you haven't seen us here before, we're PostgresML, an open-source MLOps platform built on Postgres. We bring ML to the database rather than the other way around. Source: over 2 years ago