Jsonformer Reviews and details

Social recommendations and mentions

• Show HN: LLMs can generate valid JSON 100% of the time

  How does this compare in terms of latency, cost, and effectiveness to jsonformer? https://github.com/1rgs/jsonformer. - Source: Hacker News / over 1 year ago

• Show HN: LLMs can generate valid JSON 100% of the time

  I'm not sure how this is different than: https://github.com/1rgs/jsonformer or https://github.com/mkuchnik/relm or https://github.com/Shopify/torch-grammar Overall there are a ton of these logit based guidance systems, the reason they don't get tons of traction is the SOTA models are behind REST APIs that don't enable this fine-grained approach. Those... - Source: Hacker News / over 1 year ago

• Ask HN: Explain how size of input changes ChatGPT performance

  You're correct with interpreting how the model works wrt it returning tokens one at a time. The model returns one token, and the entire context window gets shifted right by one to for account it when generating the next one. As for model performance at different context sizes, it's seems a bit complicated. From what I understand, even if models are tweaked (for example using the superHOT RoPE hack or sparse... - Source: Hacker News / over 1 year ago

• LLMs for Schema Augmentation

  From here, we just need to continue generating tokens until we get to a closing quote. This approach was borrowed from Jsonformer which uses a similar approach to induce LLMs to generate structured output. Continuing to do so for each property using Replit's code LLM gives the following output:. - Source: dev.to / over 1 year ago

• Doesn't a 4090 massively overpower a 3090 for running local LLMs?

  Https://github.com/1rgs/jsonformer or https://github.com/microsoft/guidance may help get better results, but I ended up with a bit more of a custom solution. Source: over 1 year ago

• OpenAI Function calling and API updates

  As they are accepting a JSON schema for the function calls, it is likely they are using token biasing based on the schema (using some kind of state machine that follows along with the tokens and only allows the next token to be a valid one given the grammar/schema). I have successfully implemented this for JSON Schema (limited subset) on llama.cpp. See also e.g. This implementation: - Source: Hacker News / over 1 year ago

• Learning sources on working with local LLMs

  Jsonformer is another example of applying constraint to the output to 100% get valid JSON. Source: over 1 year ago

• Adding GPT to a web app. The real experience.

  I can see some specific problems there, like malformed json (or json not matching intended schema being generated). Approaches like https://github.com/1rgs/jsonformer and https://github.com/newhouseb/clownfish could be interesting there, as well as approaches to validate outputs like https://medium.com/@markherhold/validating-json-patch-requests-44ca5981a7fc (references jsonpatch which could be interesting as... - Source: dev.to / over 1 year ago

• (1/2) May 2023

  Jsonformer: Generate structured output from LLMs (https://github.com/1rgs/jsonformer). Source: over 1 year ago