EditMode mentions (0)

Jsonformer mentions (9)

• Show HN: LLMs can generate valid JSON 100% of the time

  How does this compare in terms of latency, cost, and effectiveness to jsonformer? https://github.com/1rgs/jsonformer. - Source: Hacker News / 10 months ago

• Show HN: LLMs can generate valid JSON 100% of the time

  I'm not sure how this is different than: https://github.com/1rgs/jsonformer or https://github.com/mkuchnik/relm or https://github.com/Shopify/torch-grammar Overall there are a ton of these logit based guidance systems, the reason they don't get tons of traction is the SOTA models are behind REST APIs that don't enable this fine-grained approach. Those... - Source: Hacker News / 10 months ago

• Ask HN: Explain how size of input changes ChatGPT performance

  You're correct with interpreting how the model works wrt it returning tokens one at a time. The model returns one token, and the entire context window gets shifted right by one to for account it when generating the next one. As for model performance at different context sizes, it's seems a bit complicated. From what I understand, even if models are tweaked (for example using the superHOT RoPE hack or sparse... - Source: Hacker News / 10 months ago

• LLMs for Schema Augmentation

  From here, we just need to continue generating tokens until we get to a closing quote. This approach was borrowed from Jsonformer which uses a similar approach to induce LLMs to generate structured output. Continuing to do so for each property using Replit's code LLM gives the following output:. - Source: dev.to / 11 months ago

• Doesn't a 4090 massively overpower a 3090 for running local LLMs?

  Https://github.com/1rgs/jsonformer or https://github.com/microsoft/guidance may help get better results, but I ended up with a bit more of a custom solution. Source: 12 months ago