Google Cloud Dataflow VS ClickHouse

Google Cloud Dataflow Reviews

ClickHouse Reviews

Google Cloud Dataflow mentions (14)

• How do you implement CDC in your organization

  Imo if you are using the cloud and not doing anything particularly fancy the native tooling is good enough. For AWS that is DMS (for RDBMS) and Kinesis/Lamba (for streams). Google has Data Fusion and Dataflow . Azure hasData Factory if you are unfortunate enough to have to use SQL Server or Azure. Imo the vendored tools and open source tools are more useful when you need to ingest data from SaaS platforms, and... Source: over 1 year ago

• Here’s a playlist of 7 hours of music I use to focus when I’m coding/developing. Post yours as well if you also have one!

  This sub is for Apache Beam and Google Cloud Dataflow as the sidebar suggests. Source: over 1 year ago

• How are view/listen counts rolled up on something like Spotify/YouTube?

  I am pretty sure they are using pub/sub with probably a Dataflow pipeline to process all that data. Source: over 1 year ago

• Best way to export several GCP datasets to AWS?

  You can run a Dataflow job that copies the data directly from BQ into S3, though you'll have to run a job per table. This can be somewhat expensive to do. Source: over 1 year ago

• Why we don’t use Spark

  It was clear we needed something that was built specifically for our big-data SaaS requirements. Dataflow was our first idea, as the service is fully managed, highly scalable, fairly reliable and has a unified model for streaming & batch workloads. Sadly, the cost of this service was quite large. Secondly, at that moment in time, the service only accepted Java implementations, of which we had little knowledge... - Source: dev.to / about 2 years ago

ClickHouse mentions (43)

• The 2024 Web Hosting Report

  For the third, examples here might be analytics plugins in specialized databases like Clickhouse, data-transformations in places like your ETL pipeline using Airflow or Fivetran, or special integrations in your authentication workflow with Auth0 hooks and rules. - Source: dev.to / 3 months ago

• Choosing Between a Streaming Database and a Stream Processing Framework in Python

  Online analytical processing (OLAP) databases like Apache Druid, Apache Pinot, and ClickHouse shine in addressing user-initiated analytical queries. You might write a query to analyze historical data to find the most-clicked products over the past month efficiently using OLAP databases. When contrasting with streaming databases, they may not be optimized for incremental computation, leading to challenges in... - Source: dev.to / 3 months ago

• Real-Time Data Enrichment and Analytics With RisingWave and ClickHouse

  To achieve seamless real-time data ingestion, transformation, and analytics, a powerful combination to explore is RisingWave and ClickHouse. RisingWave is a PostgreSQL-compatible database specifically designed for stream processing. It excels at ingesting real-time data streams, performing diverse transformations, and enabling instant querying of results. ClickHouse® is a high-performance, column-oriented SQL... - Source: dev.to / 5 months ago

• Ask HN: Is there a Hacker News takeout to export my comments / upvotes, etc.?

  You can export the whole dataset as described here: https://github.com/ClickHouse/ClickHouse/issues/29693

      curl https://clickhouse.com/ | sh.

  - Source: Hacker News / 5 months ago

• Best alternative

  Nowadays I am looking at the clickhouse and how it might help me maybe you can check it out: https://clickhouse.com/. Source: 6 months ago