How does this compare in terms of latency, cost, and effectiveness to jsonformer? https://github.com/1rgs/jsonformer. - Source: Hacker News / almost 2 years ago

I'm not sure how this is different than: https://github.com/1rgs/jsonformer or https://github.com/mkuchnik/relm or https://github.com/Shopify/torch-grammar Overall there are a ton of these logit based guidance systems, the reason they don't get tons of traction is the SOTA models are behind REST APIs that don't enable this fine-grained approach. Those... - Source: Hacker News / almost 2 years ago

You're correct with interpreting how the model works wrt it returning tokens one at a time. The model returns one token, and the entire context window gets shifted right by one to for account it when generating the next one. As for model performance at different context sizes, it's seems a bit complicated. From what I understand, even if models are tweaked (for example using the superHOT RoPE hack or sparse... - Source: Hacker News / almost 2 years ago

From here, we just need to continue generating tokens until we get to a closing quote. This approach was borrowed from Jsonformer which uses a similar approach to induce LLMs to generate structured output. Continuing to do so for each property using Replit's code LLM gives the following output:. - Source: dev.to / almost 2 years ago

Https://github.com/1rgs/jsonformer or https://github.com/microsoft/guidance may help get better results, but I ended up with a bit more of a custom solution. Source: almost 2 years ago

You can easily scale this to 100K+ entries, integrate it with a local LLM like LLama - find one yourself on huggingface. ...or deploy it to your own infrastructure. No cloud dependencies required 💪. - Source: dev.to / 6 days ago

Compatibility with standard tools: Functions with OCI-compliant registries such as Docker Hub and integrates with widely-used tools including Hugging Face, ZenML, and Git. - Source: dev.to / 14 days ago

Hugging Face's Transformers: A comprehensive library with access to many open-source LLMs. https://huggingface.co/. - Source: dev.to / about 1 month ago

Hugging Face provides licensing for their NLP models, encouraging businesses to deploy AI-powered solutions seamlessly. Learn more here. Actionable Advice: Evaluate your algorithms and determine if they can be productized for licensing. Ensure contracts are clear about usage rights and application fields. - Source: dev.to / about 1 month ago

There are lots of open-source models available on HuggingFace that can be used to create vector embeddings. Transformers.js is a module that lets you use machine learning models in JavaScript, both in the browser and Node.js. It uses the ONNX runtime to achieve this; it works with models that have published ONNX weights, of which there are plenty. Some of those models we can use to create vector embeddings. - Source: dev.to / about 2 months ago