Apache Flink, known initially as Stratosphere, is a distributed stream processing engine initiated by a group of researchers at TU Berlin. Since its initial release in May 2011, Flink has gained immense popularity in both academia and industry. And it is currently the most well-known streaming system globally (challenge me if you think I got it wrong!). - Source: dev.to / 13 days ago

Apache Iceberg defines a table format that separates how data is stored from how data is queried. Any engine that implements the Iceberg integration — Spark, Flink, Trino, DuckDB, Snowflake, RisingWave — can read and/or write Iceberg data directly. - Source: dev.to / 18 days ago

The last decade saw the rise of open-source frameworks like Apache Flink, Spark Streaming, and Apache Samza. These offered more flexibility but still demanded significant engineering muscle to run effectively at scale. Companies using them often needed specialized stream processing engineers just to manage internal state, tune performance, and handle the day-to-day operational challenges. The barrier to entry... - Source: dev.to / 22 days ago

Apache Flink: Flink is a unified streaming and batching platform developed under the Apache Foundation. It provides support for Java API and a SQL interface. Flink boasts a large ecosystem and can seamlessly integrate with various services, including Kafka, Pulsar, HDFS, Iceberg, Hudi, and other systems. - Source: dev.to / about 1 month ago

In conclusion, Apache Flink is more than a big data processing tool—it is a thriving ecosystem that exemplifies the power of open source collaboration. From its impressive technical capabilities to its innovative funding model, Apache Flink shows that sustainable software development is possible when community, corporate support, and transparency converge. As industries continue to demand efficient real-time data... - Source: dev.to / 2 months ago

The open-sourcing of the X's feed algorithm is a typical Musk's smokescreen. First, while the code was open sourced, the configs and data that determines its behavior wasn't. Any researcher that has some knowledge of the matter will tell you this. Second, that code was last updated in July 2023 [1]. News feed algorithms are updated regularly, especially around the time of US elections [2]. [1] - Source: Hacker News / 6 months ago

Sure it's open source but that doesn't mean the public repo is in any way up to date. Especially given that the repos only have a handful of commits from the few months after the repos were published and they haven't been updated in well over a year. https://github.com/twitter/the-algorithm. - Source: Hacker News / 6 months ago

> Wasn’t the tweet recommendation system “open sourced” as well? Does this guy know the difference between open source and “open source”? What do you mean? There exists only one binding definition of open source > https://opensource.org/osd and either some product does satisfy it, or it doesn't. As far as I am aware > https://github.com/twitter/the-algorithm... - Source: Hacker News / about 1 year ago

Yes, and it's here: https://github.com/twitter/the-algorithm If e.g. Amazon open sources some part of its software infrastructure should they also open source the data it uses or their configuration files? - Source: Hacker News / about 1 year ago

And I believe the source for that was effectively opened up to the world: https://github.com/twitter/the-algorithm. Source: almost 2 years ago