Simple Scraper
Octoparse
Diggernaut
Scraper API
Agenty
eScraper
Crawlbase
artoo.js
DEV.to
WordPress
Medium
Hashnode
Ghost
Drupal
GitHub
Stack Overflow
Simple scraper is the easiest way to scrape the web โ turn any website into an API in seconds and use ready-made scraping recipes to scrape popular sites with ease.
Simple Scraper DEV.toAs a mini-blog, it is a nice alternative for Medium to publish and share information about programming.
However, the community and the organization are biased toward social justice (and they are open to it). You can read its Code of Conduct, it is so vague and politically leads (I prefer a term of service because it defines fair rules for everybody). So it alienates developers that we don't care about politics in pro of people that want to talk about any other topic such as sexuality, how women are unprivileged, and such. It even mandates to use inclusive language. Good grief.
My main complaint is the quality of the community. It is not StackOverflow (so we don't want to ask for an answer here), and most of the top topics are clickbait, such as "how to become a rockstar developer in ... days", "100 tips to become a better programmer" (and it doesn't even talk about programming).
Technically this "mini blog" site allows us to use markdown, and it is okay. However, the whole experience is really basic. Even the template is ugly.
Based on our record, DEV.to seems to be a lot more popular than Simple Scraper. While we know about 661 links to DEV.to, we've tracked only 22 mentions of Simple Scraper. We are tracking product recommendations and mentions on various public social media platforms and blogs. They can help you identify which product is more popular and what people think of it.
Data extraction: https://simplescraper.io A project that I launched on HN that became a business. Simplescraper rode the no-code wave of a few years back ('instant structured data without parsing html'). Now working on increasing the surface area for AI agents: MCP support, screenshots API, and (experimentally) x402^ ^ https://simplescraper.io/blog/x402-payment-protocol/. - Source: Hacker News / 5 months ago
1. Clicking the box programmatically โ possible but inconsistent 2. Outsourcing the task to one of the many CAPTCHA-solving services (2Captcha etc) โ better 3. Using a pool of reliable IP addresses so you don't encounter checkboxes or turnstiles โ best I run a web scraping startup (https://simplescraper.io) and this is usually the approach. It has become more difficult, and I think a lot of the AI crawlers are... - Source: Hacker News / about 1 year ago
Making my data extraction Saas (https://simplescraper.io) more LLM friendly. Markdown extraction, improved Google search, workflows - search for this terms, visit the first N links, summarize etc. Big demand for (or rather, expectation of) this lately. - Source: Hacker News / almost 2 years ago
Things are much easier for one-person startups these daysโit's a gift. I remember building a todo app as my first SaaS project, and choosing something called Stormpath for authentication. It subsequently shut down, forcing me to do a last-minute migration from a hostel in Japan using Nitrous Cloud IDE (which also shut down). Just pain upon pain.[1] Now, you can just pick a full-stack cloud service and run with it.... - Source: Hacker News / about 2 years ago
Simplescraper โ Trigger your webhook after each operation. The free plan includes 100 cloud scrape credits. - Source: dev.to / over 2 years ago
Def pending(): try: drafted_text = open(DRAFTS, encoding="utf-8").read() except FileNotFoundError: drafted_text = "" drafted_codes = set(re.findall(r"^## (\S+)", drafted_text, re.M)) out = [] for a in api(f"/articles?username={ME}&per_page=100"): if not a["comments_count"]: continue for c in api(f"/comments?a_id={a['id']}"): if not... - Source: dev.to / 1 day ago
To think that all of this happens under the hood when Iโm just trying to access YouTube to listen to my favorite music playlists to go through the day is insane! It all happens so quicklyโฆ to think that all eleven of these steps occur instantly when I was first entering a website like https://dev.to, for instance, is mind-blowing. - Source: dev.to / 2 days ago
And keep an exit. This is the argument I made about small models from a different angle: the same task that costs $14,000/month at frontier scale often costs approximately nothing on a 4B model running on your own hardware. Not for everything - for the long tail of focused, repetitive work that makes up most of what we actually automate. Knowing which of your workflows would survive a 10x price increase is cheap... - Source: dev.to / 7 days ago
P.S originally it was only daily.dev. Now it is (dev.to)[https://dev.to] as well! - Source: dev.to / 7 days ago
That is the gap Arcade.dev is purpose-built to close. Unlike a routing gateway that simply proxies requests, Arcade is a full action runtime. It's a secure execution environment for production agents with per-user authorization, vaulted credentials, structured audit logs, and hosted tool execution. It delivers a hosted catalog of over 8,000 pre-built agent-optimized MCP tools, designed for agent intent rather than... - Source: dev.to / 11 days ago
Octoparse - Octoparse provides easy web scraping for anyone. Our advanced web crawler, allows users to turn web pages into structured spreadsheets within clicks.
WordPress - WordPress is web software you can use to create a beautiful website or blog. We like to say that WordPress is both free and priceless at the same time.
Diggernaut - Web scraping is just became easy. Extract any website content and turn it into datasets. No programming skills required.
Medium - Welcome to Medium, a place to read, write, and interact with the stories that matter most to you.
Scraper API - Scale Data Collection with a Simple API.
Hashnode - A friendly and inclusive Q&A network for coders
