To fully leverage the data is the new oil concept, companies require a special database designed to manage vast amounts of data instantly. This need has led to different database forms, including NoSQL databases, vector databases, time-series databases, graph databases, in-memory databases, and in-memory data grids. Recent years have seen the rise of cloud-based streaming databases such as RisingWave, Materialize,... - Source: dev.to / 4 months ago

Some recent solutions to this problem include Differential Dataflow and Materialize. It would be neat if postgres adopted something similar for live-updating materialized views. https://github.com/timelydataflow/differential-dataflow. - Source: Hacker News / 8 months ago

Materialize | Full-Time | NYC Office or Remote | https://materialize.com Materialize is an Operational Data Warehouse: A cloud data warehouse with streaming internals, built for work that needs action on what’s happening right now. Keep the familiar SQL, keep the proven architecture of cloud warehouses but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that... - Source: Hacker News / 9 months ago

Materialize | EM (Compute), Senior PM | New York, New York | https://materialize.com/ You shouldn't have to throw away the database to build with fast-changing data. Keep the familiar SQL, keep the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. That is Materialize, the only true SQL... - Source: Hacker News / about 1 year ago

Materialize | NY, NY | https://materialize.com/ The Cloud Database for Fast-Changing Data. We put a streaming engine in a database, so your team can build real-time data products without the cost, complexity, and development time of stream processing. Cloud team openings: https://grnh.se/0ad6ab6b4us Senior PM openings: https://grnh.se/415c267f4us. - Source: Hacker News / about 1 year ago

I have lived through the hype of Big data it was a time of HDFS+HTable I guess and Hapoop etc. One can't go wrong with DuckDB+SQLite+Open/Elasticsearch either with 6 to 8 even 10 TB of data. [0]. https://duckdb.org/. - Source: Hacker News / 18 days ago

More than once, I have been in a situation where I needed to query CloudTrail logs but was working in a customer environment where they weren’t aggregated to a search interface. Another similar situation is when CloudTrail data events are disabled for cost reasons but need to be temporarily turned on for troubleshooting/audit purposes. While the CloudTrail console offers some (very) limited lookups (for management... - Source: dev.to / 25 days ago

DuckDB: An in-process SQL OLAP database management system. While not a traditional OLAP database, DuckDB is designed to execute analytical queries efficiently, making it suitable for analytical workloads within data-intensive applications. - Source: dev.to / 4 months ago

Easiest way to practically use SIMD table scan database is try out DuckDB: https://duckdb.org/. - Source: Hacker News / 4 months ago

Duckdb so we can make OLAP like queries on the data. - Source: dev.to / 6 months ago