We tried all Major Web scraping API on the market, Scrapfly offer the best success rate/performance. The monitoring feature is very helpful. Happy to pay for their service.