This is how I do it. I send the URLs I want scraped to Urlbox[0] it renders the pages saves HTML (and screenshot and metadata) to my S3 bucket[1]. I get a webhook[2] when it's ready for me to process. I prefer to use Ruby so Nokogiri[3] is the tool I use for scraping step. This has been particularly useful when I've want to scrape some pages live from a web app and don't want to manage running Puppeteer or... - Source: Hacker News / 4 months ago

Hi there, I run urlbox.io, which is a screenshot API that allows clicking elements, waiting for elements, injecting custom JS/CSS etc. Source: over 1 year ago

What's the relation between the Gitlab repository and the Github one? https://github.com/iamadamdev/bypass-paywalls-chrome. - Source: Hacker News / about 2 months ago

Sorry guys.. should've realized this was paywalled. I have the Bypass Paywalls plugin on my browser. Source: 6 months ago

Edit: Here's a good paywall blocker I use. Mobile users in shambles. Source: 6 months ago

FWIW I am happy that magnolia1234 forked iamadamdev's repo. I wanted to report that barrons.com is not working with the original repo (https://github.com/iamadamdev/bypass-paywalls-chrome was giving the following notice. An owner of this repository has limited the ability to open an issue to users that have contributed to this repository in the past. barrons.com worked out of the box with the new repo... - Source: Hacker News / 8 months ago

What a weird, overly-confrontational message from the fork's author! No need to call someone publishing work for free lazy, ignorant, not the sharpest tool in the shed etc. Plus, the original repository isn't even licensed https://github.com/iamadamdev/bypass-paywalls-chrome . So magnolia1234 probably forked it illegally after copying the name... - Source: Hacker News / 8 months ago