Analyzing multi-gigabyte JSON files locally

1

OctoSQL

OctoSQL[0] or DuckDB[1] will most likely be much simpler, while going through 10 GB of JSON in a couple seconds at most. Disclaimer: author of OctoSQL [0]: https://github.com/cube2222/octosql.

#Databases #Big Data #Relational Databases 22 social mentions

2

DuckDB

OctoSQL[0] or DuckDB[1] will most likely be much simpler, while going through 10 GB of JSON in a couple seconds at most. Disclaimer: author of OctoSQL [0]: https://github.com/cube2222/octosql.

#Databases #Big Data #Relational Databases 13 social mentions

3

LNAV

LNAV (https://lnav.org) is ideally suited for this kind of thing, with an embedded sqlite engine and what amounts to a local laptop-scale mini-ETL toolkit w/ a nice CLI. I've been recommending it for the last 7 years since I discovered this awesome little underappreciated util.

#Monitoring Tools #Log Management #Realtime Analytics 60 social mentions