The Algorithm Reviews

Apache Spark Reviews

The Algorithm mentions (22)

• X's algorithm was changed in mid-July 2024 to boost right-leaning accounts

  The open-sourcing of the X's feed algorithm is a typical Musk's smokescreen. First, while the code was open sourced, the configs and data that determines its behavior wasn't. Any researcher that has some knowledge of the matter will tell you this. Second, that code was last updated in July 2023 [1]. News feed algorithms are updated regularly, especially around the time of US elections [2]. [1] - Source: Hacker News / 6 months ago

• X's algorithm was changed in mid-July 2024 to boost right-leaning accounts

  Sure it's open source but that doesn't mean the public repo is in any way up to date. Especially given that the repos only have a handful of commits from the few months after the repos were published and they haven't been updated in well over a year. https://github.com/twitter/the-algorithm. - Source: Hacker News / 6 months ago

• "xAI will open source Grok"

  > Wasn’t the tweet recommendation system “open sourced” as well? Does this guy know the difference between open source and “open source”? What do you mean? There exists only one binding definition of open source > https://opensource.org/osd and either some product does satisfy it, or it doesn't. As far as I am aware > https://github.com/twitter/the-algorithm... - Source: Hacker News / about 1 year ago

• "xAI will open source Grok"

  Yes, and it's here: https://github.com/twitter/the-algorithm If e.g. Amazon open sources some part of its software infrastructure should they also open source the data it uses or their configuration files? - Source: Hacker News / about 1 year ago

• Twitter sends Meta cease-and-desist letter over new Threads app: Sources

  And I believe the source for that was effectively opened up to the world: https://github.com/twitter/the-algorithm. Source: almost 2 years ago

Apache Spark mentions (70)

• Every Database Will Support Iceberg — Here's Why

  Apache Iceberg defines a table format that separates how data is stored from how data is queried. Any engine that implements the Iceberg integration — Spark, Flink, Trino, DuckDB, Snowflake, RisingWave — can read and/or write Iceberg data directly. - Source: dev.to / 17 days ago

• How to Reduce Big Data Analytics Costs by 90% with Karpenter and Spark

  Apache Spark powers large-scale data analytics and machine learning, but as workloads grow exponentially, traditional static resource allocation leads to 30–50% resource waste due to idle Executors and suboptimal instance selection. - Source: dev.to / 19 days ago

• Unveiling the Apache License 2.0: A Deep Dive into Open Source Freedom

  One of the key attributes of Apache License 2.0 is its flexible nature. Permitting use in both proprietary and open source environments, it has become the go-to choice for innovative projects ranging from the Apache HTTP Server to large-scale initiatives like Apache Spark and Hadoop. This flexibility is not solely legal; it is also philosophical. The license is designed to encourage transparency and maintain a... - Source: dev.to / about 2 months ago

• The Application of Java Programming In Data Analysis and Artificial Intelligence

  [1] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach. Pearson, 2020. [2] F. Chollet, Deep Learning with Python. Manning Publications, 2018. [3] C. C. Aggarwal, Data Mining: The Textbook. Springer, 2015. [4] J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communications of the ACM, vol. 51, no. 1, pp. 107-113, 2008. [5] Apache Software Foundation, "Apache... - Source: dev.to / 2 months ago

• Automating Enhanced Due Diligence in Regulated Applications

  If you're designing an event-based pipeline, you can use a data streaming tool like Kafka to process data as it's collected by the pipeline. For a setup that already has data stored, you can use tools like Apache Spark to batch process and clean it before moving ahead with the pipeline. - Source: dev.to / 3 months ago