Why does Octoparse need a proxy?
Because Octoparse has nowhere to type a login, a proxy that needs one cannot be entered directly. A small local forwarder bridges the gap.
Which proxy works best with Octoparse?
For most scraping, Residential Lite is the cheapest way to get home IP addresses in the right country. Use Residential when you need a particular state or city, and Unlimited Datacenter for targets in the United States that do not block data centre addresses.
- Residentialfrom $3.52 per GB
- Residential Litefrom $0.96 per GB
- Unlimited Datacenterfrom $0.56 per hour
Proxy settings for Octoparse: host, port and login
A BooProxies plan gives you four things: a host, a port, a username and a password. These are the values for Residential; the username and password are on your plan page.
| Host | resi.proxies.boo |
|---|---|
| HTTP port | 8080 |
| SOCKS5 port | 1080 |
| Username | USERNAME |
| Password | from your plan page |
The country and the session are chosen in the username, not in a separate setting. The generator on your plan page writes the username for you, and Targeting and sessions explains each part.
Octoparse proxy setup, step by step
- Start a local forwarder that listens without a login and passes traffic to your proxy with the login added. With gost:
gost -L=:8899 -F=http://USERNAME:PASSWORD@resi.proxies.boo:8080. - In Octoparse, open the task's settings and find the anti-blocking section.
- Choose
Use my own proxies, clickConfigureand add127.0.0.1:8899. - Run the task locally.
Menu names can differ a little between versions of Octoparse. The four values to enter are always the same.
Rotating or sticky sessions for Octoparse?
For crawling many pages with Octoparse, leave the username without a session so every request leaves from a new IP address. For anything with steps that belong together, such as a login, a cart or a paginated search that sets cookies, add a session to the username so those requests share one address.
A sticky username looks like this:
USERNAME-country-us-time-600-session-a1b2c3d4Octoparse proxy not working?
- A small local forwarder solves this: it listens on your computer without a login and adds the username and password on the way out. See proxies with a login in tools that only take IP and port.
- Your own proxies apply to local runs. Tasks run in Octoparse's cloud use Octoparse's proxies instead.
- A
407error means the username or password was not accepted. Copy both again from your plan page, and check that nothing added a space or cut the username short. - If the IP address is in the wrong country, look at the username: the country code is part of it. More fixes.
Questions about Octoparse and proxies
Can I enter a proxy username and password in Octoparse?
No. Its own-proxy setting takes an IP address and a port only. Run a local forwarder that adds the login, and give Octoparse the forwarder's address.
Which BooProxies proxy should I use with Octoparse?
For most scraping, Residential Lite is the cheapest way to get home IP addresses in the right country. Use Residential when you need a particular state or city, and Unlimited Datacenter for targets in the United States that do not block data centre addresses.
Terms explained: web scraping, rotating proxy, sticky session, rate limiting, headless browser.
Octoparse is a product of its own makers. BooProxies is not affiliated with it; the name is used here only to say what this guide is about. Use it within the rules of the sites you work with and our acceptable use policy.