Skip to content
TreeProxiesENPTESFRSign in

Guides

Using the address with Scrapy

The address goes into the middleware, not the spider.

In Scrapy, the proxy goes in the request meta: request.meta["proxy"] = "http://IP:20001". Authentication goes in a Proxy-Authorization header with the username and password in base64.

Running the list

To spread across several addresses, pick one from the list at random on each request. Pinning the same address to everything concentrates the blocking on it.

A 407 in the log is the password, not a ban. Handle that code apart from the destination site codes.