Aquarium will try to run things in a docker container. Source: about 1 year ago

Aquarium is pretty cool IMO. It makes a Ubuntu docker container and does whatever the request was to the best of its abilities. Source: about 1 year ago

- Conversational simulation is an emerging idea building on top of model-graded eval” - AI Startup Founder Things to consider when comparing options: “Types of metrics supported (only NLP metrics, model-graded evals, or both), level of customizability; supports component eval (i.e. Single prompts) or pipeline evals (i.e. Testing the entire pipeline, all the way from retrieval to post-processing)” “+method of... - Source: Hacker News / 9 months ago

Humanloop (YC S20) | London (or remote) | https://humanloop.com We're looking for exceptional engineers that can work at varying levels of the stack (frontend, backend, infra), who are customer obsessed and thoughtful about product (we think you have to be -- our customers are "living in the future" and we're building what's needed). Our stack is primarily Typescript, Python, GPT-3. Please apply at... - Source: Hacker News / over 1 year ago

https://humanloop.com/ Find the prompts users love and fine-tune custom models for higher performance at lower cost. - Source: Hacker News / over 1 year ago