The sidecar pattern in Kubernetes describes a single pod containing a container in which a main app sits. A helper container (the sidecar) is deployed alongside a main app container within the same pod. This pattern allows each container to focus on a single aspect of the overall functionality, improving the maintainability and scalability of apps deployed in Kubernetes environments. From gathering metrics to... - Source: dev.to / 11 days ago

Dapr provides a set of building blocks that abstract concepts commonly used in distributed systems. This includes secured synchronous and asynchronous communication between services, caching, workflows, resiliency, secret management and much more. Not having to implement these features yourself eliminates boilerplate, reduce complexity and allows you to focus on developing your business features. - Source: dev.to / about 2 months ago

Diagrid Catalyst is a Developer API platform providing a brand-new approach to distributed application development. Using the Catalyst APIs, powered by the Dapr open source project, developers can overcome the complexity of rewriting common software patterns and achieve higher productivity by offloading infrastructure concerns from their code to Catalyst. - Source: dev.to / about 2 months ago

The following two examples are open-source projects maintained by Fermyon with contributions from companies like Microsoft and SUSE. The first is Spin, which allows us to use WebAssembly to create Serverless applications. The second, SpinKube, combines some of the topics I'm most excited about these days: WebAssembly and Kubernetes Operators :) The official website says, "By running applications in the Wasm... - Source: dev.to / 2 months ago

Speaking of this has anyone had much experience with Dapr (https://dapr.io/) before? I always thought this was a particularly interesting approach from Microsoft where they use this pattern to essentially take the complexity of micro services and instead try and keep it as simple as a normal .NET application but (and I think this is the clever part) in both a vendor and language neutral way. But all of a sudden it... - Source: Hacker News / 7 months ago

Restate is built as a sharded replicated state machine similar to how TiKV (https://tikv.org/), Kudu (https://kudu.apache.org/kudu.pdf) or CockroachDB (https://github.com/cockroachdb/cockroach) since it makes it possible to tune the system more easily for different deployment scenarios (on-prem, cloud, cost-effective blob storage). Moreover, it allows for some other cool things like seamlessly moving from one log... - Source: Hacker News / 9 days ago

I’ve recently started my journey with Apache Flink. As I learn certain concepts, I’d like to share them. One such "learning" is the expansion of array type columns in Flink SQL. Having used ksqlDB in a previous life, I was looking for functionality similar to the EXPLODE function to "flatten" a collection type column into a row per element of the collection. Because Flink SQL is ANSI compliant, it’s no surprise... - Source: dev.to / 29 days ago

You should let the Apache Flink team know, they mention exactly-once processing on their home page (under "correctness guarantees") and in their list of features. [0] https://flink.apache.org/ [1] https://flink.apache.org/what-is-flink/flink-applications/#building-blocks-for-streaming-applications. - Source: Hacker News / about 1 month ago

Data scientists often prefer Python for its simplicity and powerful libraries like Pandas or SciPy. However, many real-time data processing tools are Java-based. Take the example of Kafka, Flink, or Spark streaming. While these tools have their Python API/wrapper libraries, they introduce increased latency, and data scientists need to manage dependencies for both Python and JVM environments. For example,... - Source: dev.to / 2 months ago

Other stream processing engines (such as Flink and Spark Streaming) provide SQL interfaces too, but the key difference is a streaming database has its storage. Stream processing engines require a dedicated database to store input and output data. On the other hand, streaming databases utilize cloud-native storage to maintain materialized views and states, allowing data replication and independent storage scaling. - Source: dev.to / 4 months ago