Apache Kafka Reviews and details

Social recommendations and mentions

• Empowering Real-Time Data Pipelines: Leveraging Apache Kafka and Rudderstack

  In today’s fast-paced digital landscape, effective data management and analysis are essential for businesses aiming to stay ahead of the curve. Fortunately, modern tools like Apache Kafka and RudderStack have revolutionized the way we handle and derive insights from large datasets. In this blog post, we’ll explore our experience implementing the Kafka Sink Connector to facilitate seamless event data transfer to... - Source: dev.to / 28 days ago

• Choosing Between a Streaming Database and a Stream Processing Framework in Python

  Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a... - Source: dev.to / 2 months ago

• How to Use Reductstore as a Data Sink for Kafka

  Apache Kafka is a distributed streaming platform capable of handling high throughput of data, while ReductStore is a databases for unstructured data optimized for storing and querying along time. - Source: dev.to / 2 months ago

• 🦿🛴Smarcity garbage reporting automation w/ ollama

  *Push data *(original source image, GPS, timestamp) in a common place (Apache Kafka,...). - Source: dev.to / 3 months ago

• How to Build & Deploy Scalable Microservices with NodeJS, TypeScript and Docker || A Comprehesive Guide

  RabbitMQ comes with administrative tools to manage user permissions and broker security and is perfect for low latency message delivery and complex routing. In comparison, Apache Kafka architecture provides secure event streams with Transport Layer Security(TLS) and is best suited for big data use cases requiring the best throughput. - Source: dev.to / 3 months ago

• Easy Guide to Integrating Kafka: Practical Solutions for Managing Blob Data

  Apache Kafka is a distributed streaming platform to share data between applications and services in real-time. - Source: dev.to / 3 months ago

• Go concurrency simplified. Part 4: Post office as a data pipeline

  Also, this knowledge applies to learning more about data engineering, as this field of software engineering relies heavily on the event-driven approach via tools like Spark, Flink, Kafka, etc. - Source: dev.to / 4 months ago

• FastStream: Python's framework for Efficient Message Queue Handling

  Our journey with FastStream started when we needed to integrate our machine learning models into a customer's Apache Kafka environment. To streamline this process, we created FastKafka using AIOKafka, AsyncAPI, and asyncio. It was our first step in making message queue management easier. - Source: dev.to / 6 months ago

• The Complete Microservices Guide

  Inter-Service Communication: Middleware provides communication channels and protocols that enable microservices to communicate with each other. This can include message brokers like RabbitMQ, Apache Kafka, RPC frameworks like gRPC, or RESTful APIs. - Source: dev.to / 7 months ago

• Handling computer vision events in real-time with Python, Kafka and Pipeless

  This article explains how you can generate and process computer vision events in real-time using Pipeless and Kafka. Pipeless is an open-source computer vision framework to build and deploy apps in minutes. Kafka is a popular OSS distributed event streaming platform. - Source: dev.to / 7 months ago

• Track every PostgreSQL data change using Debezium

  Whenever a new row is added, a row is updated, or a row is deleted, Debezium notices it immediately. It then packages up these changes and sends them as a continuous stream of events by leveraging the power of Apache Kafka. - Source: dev.to / 8 months ago

• The Role of Queues in Building Efficient Distributed Applications

  As shown above, we are using Apache Kafka as the messaging queue. Each URL will be pushed into the queue as a single event from the URLInputLambda. Next, the ScraperLambdaget each event from the queue to be processed. - Source: dev.to / 8 months ago

• Ingesting Data into OpenSearch using Apache Kafka and Go

  Scalable data ingestion is a key aspect for a large-scale distributed search and analytics engine like OpenSearch. One of the ways to build a real-time data ingestion pipeline is to use Apache Kafka. It's an open-source event streaming platform used to handle high data volume (and velocity) and integrates with a variety of sources including relational and NoSQL databases. For example, one of the canonical use... - Source: dev.to / 9 months ago

• Analyzing Real-Time Movie Reviews With Redpanda and Memgraph

  In recent years, it has become apparent that almost no production system is complete without real-time data. This can also be observed through the rise of streaming platforms such as Apache Kafka, Apache Pulsar, Redpanda, and RabbitMQ. - Source: dev.to / 9 months ago

• Visualize Real-Time Data With Python, Dash, and RisingWave

  We know that real-time data is data that is generated and processed immediately, as it is collected from different data sources. Sources can be typical databases such as Postgres or MySQL, and message brokers like Kafka. A real-time data visualization consists of a few steps, first we ingest, then process, and finally show this data in a dashboard. - Source: dev.to / 11 months ago

• Delayed events pattern, no more crons

  Although, there are no limitations in using a pure event streaming technology like Kafka. - Source: dev.to / 10 months ago

• JR, quality Random Data from the Command line, part II

  In the first part of this series, we have seen how to use JR in simple use cases to stream random data from predefined templates to standard out and Apache Kafka on Confluent Cloud. - Source: dev.to / 11 months ago

• Exploring Async PHP

  The use of queues such as Amazon SQS, RabbitMQ or Apache Kafka has been a widely accepted solution for some time. - Source: dev.to / 11 months ago

• Best way to schedule events and handle them in the future?

  The second approach is to use a message queue, as some others have suggested. The most powerful of these is probably Kafka, but it's almost certainly overkill. (Technically, Kafka is an event log, not a message queue, but that's semantics at this point). Source: 11 months ago

• Top 6 message queues for distributed architectures

  Apache Kafka is an open-source, distributed event streaming platform with message communication and storage capabilities. Although Kafka is not technically a message queue, it has the functionality of a message queue using a topic partition. - Source: dev.to / 11 months ago

• Amazon Ditches Microservices for Monolith: Decoding Prime Video's Architectural Shift

  When it comes to the limitations of AWS Step Functions, let us look at what it was doing. Step Functions handled communication between the different steps of their stream quality architecture and error handling. When it comes to communication between services, tools like Kafka exist and can be used to transfer data (or state) between services. Kafka uses a pub/sub (publish and subscribe) messaging model that... - Source: dev.to / 11 months ago