Downloading from Internet Archive with reasonable speed

1

aria2

#Torrents #Download Manager #BitTorrent 43 social mentions

2

Archive.org

To get any kind of speed out of archive.org you will need to use proxies. I also do this with aria2c but what I did is I wrote a script that I host across several different networks that allows me to grab arbitrary ranges of data from archive.org at each proxy point, and I give aria2c a list of all of the "URLs", but these URLs are really to the proxy script itself with the actual URL to download given as an encoded query string e.g. "foo.com/proxy?url=archive.org/abc". The reason for this is that archive.org limits the speed of connections *per IP*, so I simply use several IPs to download from at the same time, and aria splits up the data in chunks in order to use all the proxies simultaneously for increased speed.

#Ebooks #Productivity #Bookmark Manager 8505 social mentions