For anyone interested in exploring the data yourself, here are a few tools https://dadroit.com/: desktop tool for processing large JSON files.There’s a free version so you can get pretty far with it if you’re patient https://www.dolthub.com/repositories/dolthub/quest: they’re running a bounty program for some healthcare providers to process the machine readable files, and have some useful code snippets in case... - Source: Hacker News / over 1 year ago

Note for people who don't know much about FreePascal. It is a full-featured and very fast compiler. The resulting program is a rival for the best output of C/CPP compilers. It can be used in the style of simpler languages like Go and is almost as safe as Rust in a much faster manner. It has a great but old-looking IDE, Lazarus. It has been under active development for decades and is used for proper projects like:... - Source: Hacker News / over 1 year ago

The files are insanely large. Eventually the only way I've been able to open them is using the DADROIT large JSON viewer: http://dadroit.com (but even this only worked when I got an M1 Mac). Source: over 1 year ago

For those looking for the ability to (locally) open and query very large JSON files, Dadroit is great: https://dadroit.com. It's been a while since I used it last, but it was a life saver for me when working with JSON lines files in S3 and Kafka log dumps. A side tidbit, IIRC, it was written in Pascal. - Source: Hacker News / over 1 year ago

You can review all the contents of it, but generally, there is one big 5GB JSON file — yelp_academic_dataset_review.json it contains 6,9 🍋 JSON records. Use Dadroid to review massive JSON files. - Source: dev.to / over 1 year ago

Yes. A Lightroom catalog file is, after all, just a SQLite database. (Srsly, make a copy of your catalog file, rename it whatever.sqlite and use your favorite SQLite GUI to rip it open and look at the tables and fields). It's just storing the pathame to the RAW file for that file's record in the database. Source: about 1 year ago

I use visidata with a playback script I recorded to open the sheet to a specific Excel tab, add a column, save the sheet as a csv file. Then I have a sqlite script that takes the csv file and puts it in a database, partitioned by monthYear. Source: about 1 year ago

Use the most-used database in the world: https://sqlite.org/index.html. Source: over 1 year ago

With this in mind, I wrote a few versions of this post, but I hated them all. Then I realized that jodliterate PDF documents mostly do what I want. So, instead of rewriting MirrorXref.pdf, I will make a few comments about jodliterate group documents in general. If you're interested in using SQLite with J, download the self-contained GitHub files MirrorXref.ijs and MirrorXref.pdf and have a look. - Source: dev.to / almost 2 years ago

SQLite, by many estimates, is the most widely deployed SQL database system on Earth. It's everywhere. It's in your phone, your laptop, your cameras, your car, your cloud, and your breakfast cereal. SQLite's global triumph is a gratifying testament to the virtues of technical excellence and the philosophy of "less is more.". - Source: dev.to / almost 2 years ago