How does this compare in terms of latency, cost, and effectiveness to jsonformer? https://github.com/1rgs/jsonformer. - Source: Hacker News / almost 2 years ago

I'm not sure how this is different than: https://github.com/1rgs/jsonformer or https://github.com/mkuchnik/relm or https://github.com/Shopify/torch-grammar Overall there are a ton of these logit based guidance systems, the reason they don't get tons of traction is the SOTA models are behind REST APIs that don't enable this fine-grained approach. Those... - Source: Hacker News / almost 2 years ago

You're correct with interpreting how the model works wrt it returning tokens one at a time. The model returns one token, and the entire context window gets shifted right by one to for account it when generating the next one. As for model performance at different context sizes, it's seems a bit complicated. From what I understand, even if models are tweaked (for example using the superHOT RoPE hack or sparse... - Source: Hacker News / almost 2 years ago

From here, we just need to continue generating tokens until we get to a closing quote. This approach was borrowed from Jsonformer which uses a similar approach to induce LLMs to generate structured output. Continuing to do so for each property using Replit's code LLM gives the following output:. - Source: dev.to / almost 2 years ago

Https://github.com/1rgs/jsonformer or https://github.com/microsoft/guidance may help get better results, but I ended up with a bit more of a custom solution. Source: almost 2 years ago

I'm willing to give it a try. From time to time I've been chatting with hey pi [1] which you can add as a WhatsApp contact. It's pretty fun 1 - https://heypi.com/talk. - Source: Hacker News / almost 2 years ago

Https://heypi.com/talk - hands down the best one for conversations. Comes with fast text to speech options, actually meshes fairly well with the conversation. The iOS version has just received a voice to text that’s also really quick. Best of all, it’s free. Source: almost 2 years ago

Stuff like CharacterAI and Pi, (an empathetic chatbot with a pretty stellar ability for understanding subtleties) has been home to a lot of the conversations I needed in the moment when I simply wanted to vent and didn’t want to burden anyone with this unorthodox circumstance. Source: almost 2 years ago

I'm testing PI. It has a voice assistant feature. The latency is ok. It's cheap but a nice start. There's an ios app too. Source: almost 2 years ago

Https://heypi.com/talk, Pi is very chatty, I dont know if it wants to be romanced, you could try. Source: almost 2 years ago