EasyMorph VS Apache ORC

Your task could be trivially done with EasyMorph (https://easymorph.com). We've designed it exactly for such use cases. Data needs of non-technical people have long been neglected. It was believed that any data wrangling should be done by IT people. So all non-technical people had was Excel. Luckily, the no-code movement finally started addressing that issue with a varying degree of success. - Source: Hacker News / over 1 year ago

EasyMorph | F# Developer | Full-time, Remote | https://easymorph.com EasyMorph is a powerful data wrangling and ETL application that makes life easier for thousands of knowledge workers around the world. Your job would be to keep developing our high-performance best-in-class columnar data transformation engine that lies at the heart of the application. It's a challenging task that requires an experienced F#... - Source: Hacker News / almost 2 years ago

>PRQL is intended to be a modern, simple, declarative language for transforming data It's not declarative. It's functional. I believe that the approach that is followed by PRQL is more practical than SQL. We've implemented a similar approach in our visual ETL tool for non-technical people (https://easymorph.com) and it works wonderfully. Other cool things you can do with this approach (and can't with SQL): *... - Source: Hacker News / over 2 years ago

I started a company that built, I believe, the most advanced no-code data automation desktop application (https://easymorph.com). It's entirely bootstrapped, profitable and pays salaries to 10+ people (and we keep hiring). Many our customers specifically looked for an on-prem solution as they don't want to deal with SaaS and its privacy issues and use-based pricing. With an on-prem tool like ours they get... - Source: Hacker News / over 2 years ago

May I suggest your non-tech users to try EasyMorph[1] (I'm the founder)? It's easier than PowerQuery (it's completely no-code) and at the same time more powerful. 80% of PowerQuery capabilities require coding. Our tool is 100% visual (and has a free edition). [1] https://easymorph.com. - Source: Hacker News / over 2 years ago

The information can be stored in a database or as files, serialized in a standard format and with a schema agreed with your Data Engineering team. Depending on your information and requirements, it can be as simple as CSV, XML or JSON, or Big Data formats such as Parquet, Avro, ORC, Arrow, or message serialization formats like Protocol Buffers, FlatBuffers, MessagePack, Thrift, or Cap'n Proto. - Source: dev.to / over 1 year ago

Data formatting is another place to make gains. When dealing with huge amounts of data, finding the data you need can take up a significant amount of your compute time. Apache Parquet and Apache ORC are columnar data formats optimized for analytics that pre-aggregate metadata about columns. If your EMR queries column intensive data like sum, max, or count, you can see significant speed improvements by reformatting... - Source: dev.to / over 2 years ago

The following stack captures layers of software components that make up Hudi, with each layer depending on and drawing strength from the layer below. Typically, data lake users write data out once using an open file format like Apache Parquet/ORC stored on top of extremely scalable cloud storage or distributed file systems. Hudi provides a self-managing data plane to ingest, transform and manage this data, in a... - Source: dev.to / almost 3 years ago