At Scale we built a tool for model debugging in computer vision called Nucleus (scale.com/nucleus) designed exactly for this, which is free try out if you're curious to see where your model predictions are most at odds with your ground truth. Source: over 2 years ago

To address your point about gathering edge cases, which can also be defined as cases of low model fidelity for our use cases, there is active learning and tools such as Aquarium Learning and Scale Nucleus which make it easy to implement into workflows. Source: almost 3 years ago

Maybe similar in spirit as idle games: I used to enjoy BoxCar2D where you just sit back and watch cars evolve using genetic algorithm. The original version[1] requires Flash, but looks like someone ported it to HTML5[2]: [1] http://boxcar2d.com/ [2] https://rednuht.org/genetic_cars_2/. - Source: Hacker News / about 2 months ago

Watch this for a while, it randomly generates a bunch of triangles and circles, the ones that make it furthest to the right have their code passed onto the next generation with random mutations. In almost no time you'll have a bunch of cars that are evolved specifically for the niche of that track. No guidance needed. Source: 6 months ago

You can actually produce a better analogy with a genetic algorithm. In the linked example, you can watch the evolution of simulated cars. Just take note that while the simulation is a good illustration of evolution, it's also, by necessity, vastly simplified. (I guess running for about 4 billion years would incur off-putting cloud costs.). Source: 6 months ago

This really doesn't require a nural net. You can run similar learning algorithms in your browser. Source: about 1 year ago

I learned about genetic algorithms: https://rednuht.org/genetic_cars_2/. Source: about 1 year ago