Apache Parquet Reviews

Amazon Kinesis Reviews

Apache Parquet mentions (19)

• [D] Is there other better data format for LLM to generate structured data?

  The Apache Spark / Databricks community prefers Apache parquet or Linux Fundation's delta.io over json. Source: 6 months ago

• Demystifying Apache Arrow

  Apache Parquet (Parquet for short), which nowadays is an industry standard to store columnar data on disk. It compress the data with high efficiency and provides fast read and write speeds. As written in the Arrow documentation, "Arrow is an ideal in-memory transport layer for data that is being read or written with Parquet files". - Source: dev.to / about 1 year ago

• Parquet: more than just "Turbo CSV"

  Googling that suggests this page: https://parquet.apache.org/. Source: about 1 year ago

• Beginner question about transformation

  You should also consider distribution of data because in a company that has machine learning workflows, the same data may need to go through different workflows using different technologies and stored in something other than a data warehouse, e.g. Feature engineering in Spark and loaded/stored in binary format such as Parquet in a data lake/object store. Source: over 1 year ago

• Pandas Free Online Tutorial In Python — Learn Pandas Basics In 5 Lessons!

  This section will teach you how to read and write data to and from a variety of file types, including CSV, Excel, SQL, HTML, Parquet, JSON etc. You’ll also learn how to manipulate data from other sources, such as databases and web sites. Source: over 1 year ago

Amazon Kinesis mentions (23)

• Event-Driven Architecture on AWS

  Event Consumers: Services that actively listen for events and respond accordingly. These consumers can be easily implemented using microservices, AWS Lambda or Amazon Kinesis (for ingesting, processing, and analyzing streaming data in real-time). - Source: dev.to / about 1 month ago

• AWS DEV OPS Professional Exam short notes

  When you see Amazon Kinesis as an option, this becomes the ideal option to process data in real time. Amazon Kinesis makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information. Amazon Kinesis offers key capabilities to cost effectively process streaming data at any scale, along with the flexibility to choose the tools that best suit... - Source: dev.to / 3 months ago

• Querying microservices in real-time with materialized views

  RisingWave is an open-source streaming database that has built-in fully-managed CDC source connectors for various databases, also it can collect data from other sources such Kafka, Pulsar, Kinesis, or Redpanda and it allows you to query real-time streams using SQL. You can get a materialized view that is always up-to-date. - Source: dev.to / about 1 year ago

• How Streaming database differs from a Traditional database?

  For example, RisingWave is one of the fastest-growing open-source streaming databases that can ingest data from Apache Kafka, Apache Pulsar, Amazon Kinesis, Redpanda, and databases via native Change data capture connections or using Debezium connectors to MySQL and PostgreSQL sources. Previously, I wrote a blog post about how to choose the right streaming database that discusses some key factors that you should... - Source: dev.to / about 1 year ago

• Query Real Time Data in Kafka Using SQL

  RisingWave is an open-source distributed SQL database for stream processing. RisingWave accepts data from sources like Apache Kafka, Apache Pulsar, Amazon Kinesis, Redpanda, and databases via native Change data capture connections to MySQL and PostgreSQL sources. It uses the concept of materialized view that involves caching the outcome of your query operations and it is quite efficient for long-running stream... - Source: dev.to / about 1 year ago