The State of Web Scraping in 2021

1

Playwright

I moved from selenium to playwright. It has a pleasant API and things just works our of box. I ran into odd problems with selenium before, especially when waiting for a specific element. Selenium didn't register it, but I could see it load. It was uncharacteristic of me, because I tend to use boring, older technologies. But this gamble paid off for me. https://playwright.dev/.

#Development #Tool #Browser Testing 231 social mentions

2

Colly

#Web Scraping #Data Extraction #Data 9 social mentions

3

Hola VPN

The large proxy providers operate in a sort of gray market. You pay for "residential" or "ISP" based IP addresses. In some instances these proxy connections are literally being tunneled through browser extensions running on a real world system somewhere (https://hola.org/ for instance).

#VPN #Security & Privacy #Security 24 social mentions