At Scale we built a tool for model debugging in computer vision called Nucleus (scale.com/nucleus) designed exactly for this, which is free try out if you're curious to see where your model predictions are most at odds with your ground truth. Source: over 2 years ago

To address your point about gathering edge cases, which can also be defined as cases of low model fidelity for our use cases, there is active learning and tools such as Aquarium Learning and Scale Nucleus which make it easy to implement into workflows. Source: almost 3 years ago

For the inference part, you can dockerise your model and use https://banana.dev for serverless GPU. They have examples on github on how to deploy and I’ve done it last year and was pretty straightforward. - Source: Hacker News / about 1 month ago

I want to first check the user's ID and only if the user has an active subscription then the request will be forwarded to my API on banana.dev else the request will be blocked at the middleware itself. Should I use Express JS for the middleware i.e. Authentication and forwarding requests? Is there any other better way to improve my project structure? Currently it looks like:. Source: 6 months ago

Hey! Would love to have you try https://banana.dev (bias: I'm one of the founders). We run A100s for you and scale 0->1->n->0 on demand, so you only pay for what you use. I'm at erik@banana.dev if you want any help with it :). - Source: Hacker News / about 1 year ago

CAN you do this in AWS? Of course, do they have a service that does exactly what this banana.dev does? Probably not. Source: about 1 year ago

I've been using banana.dev for easily running my ML models on GPU in a serverless manner, and interacting with them as an API. Although the principle of the service is sound, it is currently too buggy to take into production (very long cold boots, errorring requests, always hitting capacity). Source: about 1 year ago