DuckDuckGo: Bang VS CommonCrawl

Check out https://duckduckgo.com/bangs you can dump into pretty much any site, Wikipedia, YouTube, Spotify, Google Translate from the DDG search bar if you have a few favorite bangs memorized. You can put the ! anywhere in the query for it to be picked up as well. - Source: Hacker News / 6 days ago

Seriously, I'm surprised at how many devs haven't seen DuckDuckGo's bangs: https://duckduckgo.com/bangs I use them all the time for work. !mdn for MDN, !dnab for .NET, !npm for NPM, !py3 for Python3 docs, !debman for Debian Manpages, !w for Wikipedia, !a for Amazon, !g for Google when you really need it. I'm not affiliated with DDG; I just really, really love it. - Source: Hacker News / 4 months ago

I think everyone in the thread knows about "search engines" in Chrome and bookmark keywords in Firefox. The crux of the issue is that there are more than 10,000 bang commands in DDG. Setting up even a popular subset in any given browser is a significant investment. It's fine if it's the browser you use 99% of the time, but for those spanning multiple computers, phones, and other devices, simply using bang... - Source: Hacker News / 4 months ago

Where is the place to add personal bangs to DuckDuckGo? I can't find it in https://duckduckgo.com/bangs or https://duckduckgo.com/settings. - Source: Hacker News / 4 months ago

Kagi uses the same bangs as duckduckgo (and more), so there should already be 4 different bangs for it https://duckduckgo.com/bangs?q=protondb. - Source: Hacker News / 5 months ago

Common Crawl Foundation | REMOTE | Full and part-time | https://commoncrawl.org/ | web datasets I'm the CTO at the Common Crawl Foundation, which has a 17 year old, 8. - Source: Hacker News / 6 days ago

Https://commoncrawl.org/ is a non-profit which offers a pre-crawled dataset. The specifics of individual tools probably vary. I imagine most tools would be based on academic datasets. - Source: Hacker News / 4 months ago

Should the NYT not sue https://commoncrawl.org/ ? OpenAI just used the data from commoncrawl for training. - Source: Hacker News / 4 months ago

What you’re likely referring to is Common Crawl: https://commoncrawl.org. - Source: Hacker News / 5 months ago

> ... a project called "Nutch" would allow web users to crawl the web themselves. Perhaps that promise is similar to the promises being made about "AI" today. The project did not turn out to be used in the way it was predicted (marketed), or even used by web users at all. Actually Nutch is used to produce the Common Crawl[0] and 60% of GPT-3's training data was Common Crawl[1], so in a way it is being used... - Source: Hacker News / 5 months ago