Apache Parquet VS Amazon RDS for PostgreSQL

The Apache Spark / Databricks community prefers Apache parquet or Linux Fundation's delta.io over json. Source: 5 months ago

Apache Parquet (Parquet for short), which nowadays is an industry standard to store columnar data on disk. It compress the data with high efficiency and provides fast read and write speeds. As written in the Arrow documentation, "Arrow is an ideal in-memory transport layer for data that is being read or written with Parquet files". - Source: dev.to / 12 months ago

Googling that suggests this page: https://parquet.apache.org/. Source: about 1 year ago

You should also consider distribution of data because in a company that has machine learning workflows, the same data may need to go through different workflows using different technologies and stored in something other than a data warehouse, e.g. Feature engineering in Spark and loaded/stored in binary format such as Parquet in a data lake/object store. Source: about 1 year ago

This section will teach you how to read and write data to and from a variety of file types, including CSV, Excel, SQL, HTML, Parquet, JSON etc. You’ll also learn how to manipulate data from other sources, such as databases and web sites. Source: about 1 year ago

Yay! We have now deployed our Django web application with ECS Service + Fargate on AWS. But now it works with SQLite file database. This file will be recreated on every service restart. So, our app cannot persist any data for now. In the next article we’ll connect Django to AWS RDS PostgreSQL. - Source: dev.to / about 1 month ago

Today, AWS announces the general availability of pgactive: Active-active Replication Extension for PostgreSQL, available for Amazon Relational Database Service (RDS) for PostgreSQL. Pgactive lets you use asynchronous active-active replication for streaming data between database instances to provide additional resiliency and flexibility in moving data between database instances, including writers located in... Source: 7 months ago

Best practice would definitely be setting up a separately hosted database (I swear I'm not an AWS shill) for production as this ensures much better data integrity. Plus it manages backups etc. For you. Source: about 1 year ago

For Postgres I’d use RDS for Postgres and for your Node app well I mean you’ve got a plethora of options. Elastic Beanstalk, ECS, App Runner, EC2, etc. If you really want to go the 0 managed hardware approach I’d go with App Runner if your application is already containerized and if not then Elastic Beanstalk. Source: over 1 year ago

How cash strapped? Personally, I would just use something managed like AWS's RDS for PostgreSQL https://aws.amazon.com/rds/postgresql/ Then you don't need to worry too much about administrative tasks. As a bonus, you can start out small and easily scale as you grow, versus self-managed. It doesn't have to be AWS. You can find similar offerings from pretty much any cloud provider. - Source: Hacker News / over 1 year ago