Metaflow Reviews and details

Social recommendations and mentions

• What are some open-source ML pipeline managers that are easy to use?

  I would recommend the following: - https://www.mage.ai/ - https://dagster.io/ - https://www.prefect.io/ - https://metaflow.org/ - https://zenml.io/home. Source: 12 months ago

• Needs advice for choosing tools for my team. We use AWS.

  1) I've been looking into [Metaflow](https://metaflow.org/), which connects nicely to AWS, does a lot of heavy lifting for you, including scheduling. Source: about 1 year ago

• Selfhosted chatGPT with local contente

  Even for people who don't have an ML background there's now a lot of very fully-featured model deployment environments that allow self-hosting (kubeflow has a good self-hosting option, as do mlflow and metaflow), handle most of the complicated stuff involved in just deploying an individual model, and work pretty well off the shelf. Source: about 1 year ago

• [OC] Gender diversity in Tech companies

  They had to figure out video compression that worked at the volume that they wanted to deliver. They had to build and maintain their own CDN to be able to have a always available and consistent viewing experience. Don’t even get me started on the resiliency tools like hystrix that they were kind enough to open source. I mean, they have their own fucking data science framework and they’re looking into using neural... Source: over 1 year ago

• Going to Production with Github Actions, Metaflow and AWS SageMaker

  Github Actions, Metaflow and AWS SageMaker are awesome technologies by themselves however they are seldom used together in the same sentence, even less so in the same Machine Learning project. - Source: dev.to / over 1 year ago

• AWS Summit 2022 Australia and New Zealand - Day 2, AI/ML Edition

  As a result of their new DS framework (based on a Metaflow - a DS framework built at Netflix and AWS SageMaker Pipelines), they were able to free up their DS resources so that Software Developers were now trained and equipped to tackle their normal DS projects, at a ratio of 70% DS/ML work was now completed by developers. This leaves the 30% meatier and more difficult problems for the Data Scientists to tackle. - Source: dev.to / almost 2 years ago

• DevOps Fundamentals for Deep Learning Engineers

  MLOps is a HUGE area to explore, and not surprisingly, there are many startups showing up in this space. If you want to get it on the latest trends, then I would look at workflow orchestration frameworks such as Metaflow (started off at Netflix, is now spinning off into its own enterprise business, https://metaflow.org/), Kubeflow (used at Google, https://www.kubeflow.org/), Airflow (used at Airbnb,... Source: about 2 years ago

• Netflix Metaflow

  The project predates Facebook’s name-change to Meta. That’s gotta be irksome. Project site: https://metaflow.org. - Source: Hacker News / over 2 years ago

• Best job scheduler in 2022? (Airflow / Dagster / Prefect / Luigi / other)

  Can I give a plug for Metaflow. It's particularly well suited to data science and ML workflows, with great tooling that's basically just annotations on python functions that gives you: - DAG orchestration - parallelism - cloud integration - data flow through DAGs — very very useful imo for data science teams trying to migrate their existing scripts to (and write new ones on) Metaflow. Source: over 2 years ago

• Help on understanding mlops tools.

  Reading up on TFX (https://www.tensorflow.org/tfx/guide): it is written in python and thus (IMO) cannot cover infrastructure aspects. I feel that it somewhat compares to Metaflow (https://metaflow.org/). As I have read more about Metaflow than TFX I'll keep going with Metaflow. It's a python sdk to streamline your ml pipeline wrapping your python annotated code in nodes of a DAG that can be scheduled on your... Source: over 2 years ago

• A few reasons why internal product management is awesome and not a downgrade

  Some examples of internal products: Airbnb's Minerva, Netflix's Metaflow, Lyft's Amundsen, LinkedIn's Kafka. Internal PMs can oversee data products, machine learning platforms, security products, devex products, devops products, marketing software, CRM platforms and many more. Source: over 2 years ago

• [D] What’s the simplest, most lightweight but complete and 100% open source MLOps toolkit? -> MY OWN CONCLUSIONS

  Metaflow . I love this framework for pipelining. Source: about 3 years ago