Proxies that keep web scrapers running
Fetch thousands of pages from one IP and that IP gets throttled or banned. Spread the work across many addresses, in the right countries, and the crawler keeps going.
Which proxy is best for web scraping?
For most scraping jobs, rotating residential proxies work best: each request goes out from a different household IP, so no single address hits a rate limit. Datacenter proxies are cheaper and fine for sites that let anyone in, and mobile proxies are worth it only when a site challenges residential addresses too.
Match the pool to the site
Begin with the cheapest pool that the site tolerates. APIs and open data usually accept datacenter IPs. Sites with bot protection usually want household addresses. The rare ones that block those as well tend to let carrier IPs through.
Rotate pages, hold journeys
Pages that stand alone, like listings or search results, should use a rotating username so each request gets a new IP. Anything spread over several steps, like pagination or a basket, needs a session label so the IP stays the same throughout.
Spend less on traffic
You pay per gigabyte, so trim it: skip images, fonts and video, ask for compressed responses, and cache pages that rarely change.
Play fair
Take public data only, follow the law where you and the website are based, and never hammer small sites. Every plan is covered by our acceptable use policy.
Recommended proxy types
- Residential Lite$0.96 per GB
- Residential$3.52 per GB
- Unlimited Datacenterfrom $0.56 per hour
- IPv6 Datacenter$0.42 per GB
Setting it up in four steps
- Order the smallest Residential Lite plan (one gigabyte) as a trial. Pay with crypto and it is delivered automatically.
- Take the host
lite.proxies.boo, port6969, and your username and password from the plan page. - Set the exit country with a code in the username, like
USERNAME-country-us. - Point the job at your real target. Working? Top up the same plan. Blocked? Move to the next proxy type listed above.
Docs and code samples · Tool-by-tool guides
Web scraping: common questions
How many proxies does a scraper need?
On a rotating plan you stop counting proxies. One login hands out a new IP on every connection, so what you really budget for is traffic, not addresses.
Must every site be scraped with residential proxies?
No. Where a site is open, start on datacenter, which costs far less. Step up to residential when blocks or captchas appear, and to mobile only if those fail as well.