The $0.47 bill seems reasonable for an experiment, but imagine someone doing a task of this complexity as a daily job - let's say 100x times, or a little more than 4 hours - the bill would be $47/day. It feels like there's still an opportunity for a cheaper solution. Have you or someone else experimented with e.g. https://localai.io/ ? - Source: Hacker News / 6 months ago

We're using LocalAI https://localai.io/ for inference on the back end amongst other tools. Source: 8 months ago

We recently added support to use open-source models by integrating with LocalAI (https://localai.io). With LocalAI, we can run open-source models like Llama2 and seamlessly build LLM applications using LLMStack and run everything on-prem. Source: 9 months ago

- Ability to use local open-source LLMs like Llama2 etc using LocalAI (https://localai.io) Background: We started as a closed source prompt management platform early this year (trypromptly.com) and eventually landed as an Enterprise LLM apps platform. In the process, we learned how hard it is to sell a horizontal SaaS platform. That combined with the concerns around data privacy (both with us hosting data as well... - Source: Hacker News / 9 months ago

LocalAI https://localai.io/ and LMStudio https://lmstudio.ai/ both have fairly complete OpenAI compatibility layers. llama-cpp-python has a FastAPI server as well: https://github.com/abetlen/llama-cpp-python/blob/main/llama_... (as of this moment it hasn't merged GGUF update yet though). - Source: Hacker News / 10 months ago

AgentGPT was an early agent framework designed to create, configure, and deploy autonomous AI agents. It mostly relies on looping OpenAI's GPT models like GPT-3.5 and GPT-4. AgentGPT allows users to set a goal for the AI, which autonomously plans, executes, and refines strategies to achieve it. This platform allows for both web browser access and local operation via Docker, or server deployment. - Source: dev.to / about 1 month ago