Apache Flink Reviews

Mage AI Reviews

Apache Flink mentions (30)

• Show HN: Restate, low-latency durable workflows for JavaScript/Java, in Rust

  Restate is built as a sharded replicated state machine similar to how TiKV (https://tikv.org/), Kudu (https://kudu.apache.org/kudu.pdf) or CockroachDB (https://github.com/cockroachdb/cockroach) since it makes it possible to tune the system more easily for different deployment scenarios (on-prem, cloud, cost-effective blob storage). Moreover, it allows for some other cool things like seamlessly moving from one log... - Source: Hacker News / 10 days ago

• Array Expansion in Flink SQL

  I’ve recently started my journey with Apache Flink. As I learn certain concepts, I’d like to share them. One such "learning" is the expansion of array type columns in Flink SQL. Having used ksqlDB in a previous life, I was looking for functionality similar to the EXPLODE function to "flatten" a collection type column into a row per element of the collection. Because Flink SQL is ANSI compliant, it’s no surprise... - Source: dev.to / 30 days ago

• Show HN: An SQS Alternative on Postgres

  You should let the Apache Flink team know, they mention exactly-once processing on their home page (under "correctness guarantees") and in their list of features. [0] https://flink.apache.org/ [1] https://flink.apache.org/what-is-flink/flink-applications/#building-blocks-for-streaming-applications. - Source: Hacker News / about 1 month ago

• Top 10 Common Data Engineers and Scientists Pain Points in 2024

  Data scientists often prefer Python for its simplicity and powerful libraries like Pandas or SciPy. However, many real-time data processing tools are Java-based. Take the example of Kafka, Flink, or Spark streaming. While these tools have their Python API/wrapper libraries, they introduce increased latency, and data scientists need to manage dependencies for both Python and JVM environments. For example,... - Source: dev.to / 2 months ago

• Choosing Between a Streaming Database and a Stream Processing Framework in Python

  Other stream processing engines (such as Flink and Spark Streaming) provide SQL interfaces too, but the key difference is a streaming database has its storage. Stream processing engines require a dedicated database to store input and output data. On the other hand, streaming databases utilize cloud-native storage to maintain materialized views and states, allowing data replication and independent storage scaling. - Source: dev.to / 4 months ago

Mage AI mentions (26)

• Looking for an open-source project

  Want to work on Mage, a modern replacement for Airflow? Source: over 1 year ago

• Is there an alternative for Airflow for running thousands of dynamic tasks?

  You could also check mage. https://github.com/mage-ai/mage-ai It is developed by former engineers of AirBnb too. Source: over 1 year ago

• ETL tool

  Disclaimer: I worked at Airbnb for 5+ years working on data tools like Airflow and I helped start Mage almost 2 years ago. Source: over 1 year ago

• Can we take a moment to appreciate how much of dataengineering is open source?

  Here is an easy to use data pipeline tool (free) with a user friendly UI: https://github.com/mage-ai/mage-ai. Source: over 1 year ago

• Show HN: Mage, Fivetran alternative for ELT and data integrations

  You can also check out the repo here: https://github.com/mage-ai/mage-ai. - Source: Hacker News / over 1 year ago