-
The Nim programming language is a concise, fast programming language that compiles to C, C++ and JavaScript.Pricing:
- Open Source
#Programming Language #Generic Programming Language #OOP 141 social mentions
-
Scrapy | A Fast and Powerful Scraping and Web Crawling FrameworkPricing:
- Open Source
I use it regularly for things like web scraping (Scrapy is a joy) and data manipulation. For instance just wrote some fairly complicated scripts for doing address matching to pair up a couple of UK datasets without a common identity field. Human-entered addresses are decidedly fuzzy so you end up with a lot of arbitrary rules and Python is just fast to develop against. I don't really care if the script takes a couple of hours to run on the full datasets (35 million addresses) as opposed to half that time in something else more of a pain to tweak around with.
#Web Scraping #Data Extraction #Data 93 social mentions
-
Dask natively scales Python Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you lovePricing:
- Open Source
Not everyone has the same "parallelism" needs. I have used mpi4py to distribute scientific computations using numpy over thousands of cores on hundreds of servers with much less effort than doing the same thing in C / C++ and almost no performance penalty (I could batch my data in big enough chunks). Today there are higher level distributed computing packages like dask that are even easier to use.
#Workflows #Databases #Software Development 16 social mentions