Google Cloud Pub/Sub Reviews and details

Social recommendations and mentions

• Event-Driven Architecture 101

  Secondly, Go is incredibly easy to learn and in my opinion, maintain. This means that if you're a growing company and expect to onboard new teams and team members, having Go as a basis for your systems should mean that new engineers can get up to speed quickly.  Below is a small sample application that can connect to Google PubSub, subscribe to a topic, send an event and then clean up. In total, its 82 lines of... - Source: dev.to / 7 months ago

• Top 6 message queues for distributed architectures

  Google Cloud Pub/Sub is a fully-managed, globally scalable and secure queue provided by Google Cloud for asynchronous processing messages. Cloud Pub/Sub has many of the same advantages and disadvantages as SQS due to also being cloud hosted. It has a free and paid tier. - Source: dev.to / 12 months ago

• Job Scheduling on Google Cloud Platform

  Cloud Pub/Sub: A global messaging service for event-driven architectures. - Source: dev.to / about 1 year ago

• Effortlessly Scale Your Applications with FaaS: Learn How Functions as a Service Can Help You Grow and Thrive

  Google Cloud Functions is a FaaS offering from Google Cloud Platform (GCP). It allows developers to run their code in response to events, such as changes in a database or the arrival of a message in a Pub/Sub topic. Like AWS Lambda, Google Cloud Functions can be used to build a variety of applications, including serverless websites, data processing pipelines, and real-time data streams. - Source: dev.to / over 1 year ago

• Mixing GCloud and F#

  That gets triggered when a Pub/Sub topic is fired (from the webhook function). - Source: dev.to / over 1 year ago

• Kafka on GKE cluster security guidelines

  I'm curious - given your limited knowledge, is there a reason you're looking to self host this in your own cluster rather than using a managed service like https://cloud.google.com/find-a-partner/partner/confluent-inc?redirect= or just native Google PubSub https://cloud.google.com/pubsub/ ? Source: over 1 year ago

• Moving to Google Cloud managed services, from a FinOps point of view

  Pub/Sub, the GCP managed service for message queuing, has two levels of services on standard and one Lite. Standard is the high availability version of it and Lite could be a zonal or regional service with infrastructure managed by the client. Obviously, the model pricing will be very different with a x10 between Standard and zonal Lite. However, the model pricing is the same is based on throughput for message... - Source: dev.to / over 1 year ago

• Why we don’t use Spark

  The Kafka-Dataproc combination served us well for some time, until GCP released its own message queue implementation: Google Cloud Pub/Sub. At the time, we investigated the value of switching. There is always an inherent switching cost, but what we had underestimated with Kafka is that there is a substantial overhead in maintaining the system. This is especially true if the ingested data volume increases rapidly.... - Source: dev.to / almost 2 years ago

• The cloud-agnostic-architecture illusion

  We can mitigate lock-in by following first principles. Loosely coupled architectures are popular for a reason. We can build systems that allow replacing single components if we follow this principle. E.g., if we move our dependency to GCP's Pub/Sub to a specific component, moving to AWS requires replacing that component with a new AWS SNS version. - Source: dev.to / almost 2 years ago

• Kubernetes Reinvented Virtual Machines (in a good sense)

  I have come at this problem from a bit of a different angle by asking what is the closest I can possibly get to the hypothetical dream state of everything is automated, autoscaling blah blah blah as possible with the absolute smallest budget in terms of not only actual costs but time budget as well. I only know the GCP ecosystem kind of well so I don't fully know to what extent these things exist in AWS and Azure... - Source: Hacker News / over 1 year ago

• A notification architecture for social networks — send millions of notifications per day

  Pub/Sub by google cloud - receiver-selection-q, builder-q, storage-q and instant-notification-q are all pub-sub queues powered by GCP Pub/Sub. If you want to use this, check the dead-letter queue and exponential backoff setting as well. - Source: dev.to / almost 2 years ago

• Can someone help me understand why data batch processing and data streaming processing pose such different challenges in data management?

  Because of these and other differences, the tools used are also different. With batch processing, data might be read from large files, processed, and stored in an OLTP (Online Transaction Processing) database (like MySQL) or OLAP (Online Analytical Processing) system (like Google BigQuery). But these would not be good solutions for streaming applications, because they are not optimized for high throughput on a lot... Source: about 2 years ago

• Distributed messaging with NATS

  Recently, I was building an application where I wanted to take event driven approach for async communication between the microservices. Usually I would use Apache Kafka, RabbitMQ, Redis Streams or a managed solution like AWS SNS, Google Cloud Pub/Sub but this time I wanted to keep my implementation easy and cost minimal yet not affect my scalability or increase technical debt for the future. After few hops on... - Source: dev.to / over 2 years ago

• What Is Event Sourcing?

  And these operations could -and probably should- be asynchronous as long as you push domain events to a queue system like Google PubSub or AWS SQS or RabbitMQ and then you pull it from a daemon. Bear in mind you should manage order and duplication. - Source: dev.to / over 2 years ago

• Google Pub/Sub client library for R

  I recently had the need to implement advanced interactivity in a Shiny dashboard based on Google Pub/Sub messaging. I couldn't find any library out there so I ended up wrapping my own one (you can find it here). Even though I didn't have the chance to give it a full spin, this might open up the opportunity to:. Source: over 2 years ago