Apache Airflow Reviews

AWS Snowball Reviews

Apache Airflow mentions (67)

• 10 Open Source Tools for Building MLOps Pipelines

  An integral part of an ML project is data acquisition and data transformation into the required format. This involves creating ETL (extract, transform, load) pipelines and running them periodically. Airflow is an open source platform that helps engineers create and manage complex data pipelines. Furthermore, the support for Python programming language makes it easy for ML teams to adopt Airflow. - Source: dev.to / 3 days ago

• AI Strategy Guide: How to Scale AI Across Your Business

  Level 1 of MLOps is when you've put each lifecycle stage and their intefaces in an automated pipeline. The pipeline could be a python or bash script, or it could be a directed acyclic graph run by some orchestration framework like Airflow, dagster or one of the cloud-provider offerings. AI- or data-specific platforms like MLflow, ClearML and dvc also feature pipeline capabilities. - Source: dev.to / 29 days ago

• The 2024 Web Hosting Report

  For the third, examples here might be analytics plugins in specialized databases like Clickhouse, data-transformations in places like your ETL pipeline using Airflow or Fivetran, or special integrations in your authentication workflow with Auth0 hooks and rules. - Source: dev.to / 4 months ago

• Best ETL Tools And Why To Choose

  Apache Airflow is an open-source platform to programmatically author, schedule, and monitor workflows. The platform features a web-based user interface and a command-line interface for managing and triggering workflows. Source: 7 months ago

• Simplifying Data Transformation in Redshift: An Approach with DBT and Airflow

  Airflow is the most widely used and well-known tool for orchestrating data workflows. It allows for efficient pipeline construction, scheduling, and monitoring. - Source: dev.to / 7 months ago

AWS Snowball mentions (21)

• WD Ultrastar Transporter: Briefcase with 368TB of NVMe SSD, 128GB RAM, 1300W PSU

  This is still a common thing. Amazon has AWS Snowball [1] (up to 210 TB), Backblaze has Fireball [2] (96 TB), and I'm sure there are others. [1]: https://aws.amazon.com/snowball/ [2]: https://www.backblaze.com/cloud-storage/features/fireball-data-migration. - Source: Hacker News / about 2 months ago

• AWS To Provide Free Data Transfer Out To Internet

  If you have extremely large amounts of data to transfer, such as hundreds of petabytes or into exabytes, AWS Snowmobile can move up to 100PB at once via a ruggedized shipping container. The ruggedized shipping container is tamper-resistant, water-resistant, temperature controlled, and GPS-tracked. The service was announced in 2016, and one of the trucks shown during a presentation that year at AWS re-Invent:. - Source: dev.to / 3 months ago

• AWS To Provide Free Data Transfer Out To Internet

  AWS Snowball has a number of devices optimized for edge computing and data transfer. The service allows you to order a ruggedized device that can hold multiple terrabytes to petabytes of data, to transfer to AWS. - Source: dev.to / 3 months ago

• AWS To Provide Free Data Transfer Out To Internet

  If you have extremely large amounts of data to transfer, such as hundreds of petabytes or into exabytes, AWS Snowmobile can move up to 100PB at once via a ruggedized shipping container. The ruggedized shipping container is tamper-resistant, water-resistant, temperature controlled, and GPS-tracked. The service was announced in 2016, and one of the trucks shown during a presentation that year at AWS re-Invent:. - Source: dev.to / 3 months ago

• Can they email between Pegasus and Earth?

  I'm sure once they got the BC-304s making regular round trips they dropped of a couple of AWS snowmobile trailers. For boosting their computing power and make copies of the Atlantis database to send back to Earth for more in depth analysis. Source: 6 months ago