Azure Event Hubs VS Apache Airflow

Azure Event Hubs Reviews

Apache Airflow Reviews

Azure Event Hubs mentions (4)

• Anyone routing firewall logs to Microsoft Event Hubs?

  We're looking into some sort of cloud-based solution to route our Palo Alto firewall logs to across our customer base. I'm with an MSP that manages over a hundred PA firewalls. I was intrigued by the Event Hubs (https://azure.microsoft.com/en-us/products/event-hubs/) solution as a way to push logs to it and then ingest them from there into our SIEM, without having to deal with challenges of multi-tenancy and... Source: over 1 year ago

• Microsoft Releases Stream Analytics No-Code Editor into General Availability

  Microsoft released Azure Stream Analytics no-code editor, a drag-and-drop canvas for developing jobs for stream processing scenarios such as streaming ETL, ingestion, and materializing data to data into general availability. The no-code editor is hosted in the company’s big-data streaming platform and event ingestion service, Azure Event Hubs. Interestingly, the offering follows up after Confluent's recent release... Source: over 1 year ago

• Infrastructure as code (IaC) for Java-based apps on Azure

  Sometimes you don’t need an entire Java-based microservice. You can build serverless APIs with the help of Azure Functions. For example, Azure functions have a bunch of built-in connectors like Azure Event Hubs to process event-driven Java code and send the data to Azure Cosmos DB in real-time. FedEx and UBS projects are great examples of real-time, event-driven Java. I also recommend you to go through 👉 Code,... - Source: dev.to / over 1 year ago

• Setting up demos in Azure - Part 1: ARM templates

  For event infrastructure, we have a bunch of options, like Azure Service Bus, Azure Event Grid and Azure Event Hubs. Like the databases, they aren't mutually exclusive and I could use all, depending on the circumstance, but to keep things simple, I'll pick one and move on. Right now I'm more inclined towards Event Hubs, as it works similarly to Apache Kafka, which is a good fit for the presentation context. - Source: dev.to / about 3 years ago

Apache Airflow mentions (65)

• The 2024 Web Hosting Report

  For the third, examples here might be analytics plugins in specialized databases like Clickhouse, data-transformations in places like your ETL pipeline using Airflow or Fivetran, or special integrations in your authentication workflow with Auth0 hooks and rules. - Source: dev.to / 3 months ago

• Best ETL Tools And Why To Choose

  Apache Airflow is an open-source platform to programmatically author, schedule, and monitor workflows. The platform features a web-based user interface and a command-line interface for managing and triggering workflows. Source: 6 months ago

• Simplifying Data Transformation in Redshift: An Approach with DBT and Airflow

  Airflow is the most widely used and well-known tool for orchestrating data workflows. It allows for efficient pipeline construction, scheduling, and monitoring. - Source: dev.to / 6 months ago

• Share Your favorite python related software!

  AIRFLOW This is more of a library in my opinion, but Airflow has become an essential tool for scheduling in my work. All our ML training pipelines are ordered and scheduled with Airflow and it works seamlessly. The dashboard provided is also fantastic! Source: 7 months ago

• Ask HN: What is the correct way to deal with pipelines?

  I agree there are many options in this space. Two others to consider: - https://airflow.apache.org/ - https://github.com/spotify/luigi There are also many Kubernetes based options out there. For the specific use case you specified, you might even consider a plain old Makefile and incrond if you expect these all to run on a single host and be triggered by a new file... - Source: Hacker News / 8 months ago