Software Alternatives & Reviews
Table of contents
  1. Social Mentions

Google Cloud Dataflow

Google Cloud Dataflow is a fully-managed cloud service and programming model for batch and streaming big data processing.

Big Data Data Dashboard Big Data Tools

Google Cloud Dataflow Reviews and details

Google Cloud Dataflow Landing Page
Google Cloud Dataflow Landing Page

Social recommendations and mentions

We have tracked the following product recommendations or mentions on Reddit and HackerNews. They can help you see what people think about Google Cloud Dataflow and what they use it for.
  • How do you implement CDC in your organization
    Imo if you are using the cloud and not doing anything particularly fancy the native tooling is good enough. For AWS that is DMS (for RDBMS) and Kinesis/Lamba (for streams). Google has Data Fusion and Dataflow . Azure hasData Factory if you are unfortunate enough to have to use SQL Server or Azure. Imo the vendored tools and open source tools are more useful when you need to ingest data from SaaS platforms, and... - Source: Reddit / 4 months ago
  • Here’s a playlist of 7 hours of music I use to focus when I’m coding/developing. Post yours as well if you also have one!
    This sub is for Apache Beam and Google Cloud Dataflow as the sidebar suggests. - Source: Reddit / 8 months ago
  • How are view/listen counts rolled up on something like Spotify/YouTube?
    I am pretty sure they are using pub/sub with probably a Dataflow pipeline to process all that data. - Source: Reddit / 8 months ago
  • Best way to export several GCP datasets to AWS?
    You can run a Dataflow job that copies the data directly from BQ into S3, though you'll have to run a job per table. This can be somewhat expensive to do. - Source: Reddit / 8 months ago
  • Why we don’t use Spark
    It was clear we needed something that was built specifically for our big-data SaaS requirements. Dataflow was our first idea, as the service is fully managed, highly scalable, fairly reliable and has a unified model for streaming & batch workloads. Sadly, the cost of this service was quite large. Secondly, at that moment in time, the service only accepted Java implementations, of which we had little knowledge... - Source: / about 1 year ago
  • Google Cloud Reference
    Cloud Dataflow: Stream/batch data processing 🔗Link 🔗Link. - Source: / 9 months ago
  • Composer out of resources - "INFO Task exited with return code Negsignal.SIGKILL"
    What you are looking for is Dataflow. It can be a bit tricky to wrap your head around at first, but I highly suggest leaning into this technology for most of your data engineering needs. It's based on the open source Apache Beam framework that originated at Google. We use an internal version of this system at Google for virtually all of our pipeline tasks, from a few GB, to Exabyte scale systems -- it can do it all. - Source: Reddit / 9 months ago
  • Pub/Sub parallel processing best practices
    The go-to recommendation is to use Dataflow to write your pipeline instead of disjoint functions. You can do something like this:. - Source: Reddit / 10 months ago
  • Data processing issue
    With that, the best way to maximize processing and minimize time is to use Dataflow or Dataproc depending on your needs. These systems are highly parallel and clustered, which allows for much larger processing pipelines that execute quickly. - Source: Reddit / over 1 year ago
  • Google Pub/Sub client library for R
    Stream data into Dataflow pipelines from R. - Source: Reddit / over 1 year ago
  • Noob question: Data Factory, but Google cloud?
    I'm not 100% sure, but perhaps Google Cloud Dataflow is similar to Azure Data Factory. - Source: Reddit / over 1 year ago
  • Best Practices to Become a Data Engineer
    Apache Beam - Apache Beam is a scalable framework that allows you to implement batch and streaming data processing jobs. It is a framework that you can use in order to create a data pipeline on Google Cloud or on Amazon Web Services. - Source: / about 2 years ago
  • Ecosystem: Haskell vs JVM (Eta, Frege)
    Dataflow is Google's implementation of a runner for Apache Beam jobs in Google cloud. Right now, python and java are pretty much the only two options supported for writing Beam jobs that run on Dataflow. - Source: Reddit / about 2 years ago
  • Google Cloud just posted this on their Twitter
    "Google Cloud’s databases and analytics products such as BigQuery, Dataflow, Pub/Sub and Firestore brought Theta Labs unlimited scale and performance, allowing them to: ...". - Source: Reddit / about 2 years ago

Do you know an article comparing Google Cloud Dataflow to other products?
Suggest a link to a post with product alternatives.

Generic Google Cloud Dataflow discussion

Log in or Post with

This is an informative page about Google Cloud Dataflow. You can review and discuss the product here. The primary details have not been verified within the last quarter, and they might be outdated. If you think we are missing something, please use the means on this page to comment or suggest changes. All reviews and comments are highly encouranged and appreciated as they help everyone in the community to make an informed choice. Please always be kind and objective when evaluating a product and sharing your opinion.