Prodigy VS Evidently AI

This is really cool. The annotation-to-testing-to-annotation-etc. Feedback loop makes a ton of sense, and I'd encourage others who may be confused on this post to look at the Automotus case study https://encord.com/customers/automotus-customer-story/ for the annotation side, but my understanding is the relationship between model outputs and annotation steering is out of scope for that project - do you know of... - Source: Hacker News / 3 months ago

Spacy [0] is a state-of-art / easy-to-use NLP library from the pre-LLM era. This post is the Spacy founder's thoughts on how to integrate LLMs with the kind of problems that "traditional" NLP is used for right now. It's an advertisement for Prodigy [1], their paid tool for using LLMs to assist data labeling. That said, I think I largely agree with the premise, and it's worth reading the entire post. The steps... - Source: Hacker News / 8 months ago

Prodigy AI - Offers software engineers career coaching, skill assessment, and job matching. Visit Prodigy AI. - Source: dev.to / 9 months ago

I essentially want to use a Combo of OCR + NER to attempt to identify this, but I'm not sure NER is well suited for this, as it is not natural language, so there is little context to go off of. I was thinking of perhaps using Prodigy, a data annotation tool, to annotate Candidate Names, Races, etc, and perhaps it will be able to learn off of image data alone wheat these fields tend to look like. Source: about 1 year ago

I come from a similar application area, where I try to tag (annotation/label) a taxonomy of products iteratively. You are trying something slightly different, AFAIU, labeling a flat set of songs, each song with a set of tags from ontology (directed graph)From an application point of view, this is what taxonomists often do, when migrating products from one catalog to another: mapping one taxonomy to another. There... Source: over 1 year ago

It is doable. However the main focus of MLFlow is in experiment tracking. I would suggest for you to look into another monitoring tools such evidentlyai . You can track more things than performance (e.g.data drift). Which may be helpful in a production setting. Source: over 1 year ago

Evidently is an open-source Python library that analyzes and monitors machine learning models. It generates interactive reports based on Panda DataFrames and CSV files for troubleshooting models and checking data integrity. These reports show model health, data drift, target drift, data integrity, feature analysis, and performance by segment. - Source: dev.to / over 2 years ago