Deposit $50.00 or more and get 10% extra balance, until Nov 14, 2026.

Create an account
BooProxies
Create account

Web scraping

How to use proxies with Screaming Frog

Screaming Frog SEO Spider crawls a site the way a search engine would. It can send the crawl through one proxy, with a username and password.

Updated · 3 min read

Why does Screaming Frog need a proxy?

A proxy lets you crawl a site as a visitor from another country would see it, or keep a large audit from being rate-limited on your office IP address.

Which proxy works best with Screaming Frog?

Residential proxies in the country you want to audit from. Add a sticky session to the username, so the whole crawl comes from one visitor.

Proxy settings for Screaming Frog: host, port and login

A BooProxies plan gives you four things: a host, a port, a username and a password. These are the values for Residential; the username and password are on your plan page.

Hostresi.proxies.boo
HTTP port8080
SOCKS5 port1080
UsernameUSERNAME
Passwordfrom your plan page

The country and the session are chosen in the username, not in a separate setting. The generator on your plan page writes the username for you, and Targeting and sessions explains each part.

Screaming Frog proxy setup, step by step

  1. Open File > Settings > Proxy on Windows or Linux. On a Mac it is under the Screaming Frog SEO Spider menu, in Settings.
  2. Tick Use Proxy Server and enter the host resi.proxies.boo and the port 8080.
  3. Tick Use Proxy Credentials and enter your username and password.
  4. Click OK and restart the program. The proxy is only used after a restart.

Menu names can differ a little between versions of Screaming Frog. The four values to enter are always the same.

Rotating or sticky sessions for Screaming Frog?

For crawling many pages with Screaming Frog, leave the username without a session so every request leaves from a new IP address. For anything with steps that belong together, such as a login, a cart or a paginated search that sets cookies, add a session to the username so those requests share one address.

A sticky username looks like this:

USERNAME-country-us-time-600-session-a1b2c3d4

Screaming Frog proxy not working?

  • The program takes one proxy at a time. For a new IP address on each request, use a rotating username; the rotation then happens on our side.
  • A full crawl of a large site downloads every page and resource. Estimate the size before you start on a gigabyte plan.
  • A 407 error means the username or password was not accepted. Copy both again from your plan page, and check that nothing added a space or cut the username short.
  • If the IP address is in the wrong country, look at the username: the country code is part of it. More fixes.

Questions about Screaming Frog and proxies

Which BooProxies proxy should I use with Screaming Frog?

Residential proxies in the country you want to audit from. Add a sticky session to the username, so the whole crawl comes from one visitor.

Terms explained: web scraping, rotating proxy, sticky session, rate limiting, headless browser.

Screaming Frog is a product of its own makers. BooProxies is not affiliated with it; the name is used here only to say what this guide is about. Use it within the rules of the sites you work with and our acceptable use policy.