Is web crawling and data scraping part of your daily routine? If so, you already know that most of the websites implement firewalls that automatically block repeated requests coming from the same IP address. The solution? Regularly change your public IP address. Learn how to change IP every X minutes and increase your productivity automatically.
Table of Contents
Why automatically change IP?
There are two main reasons for changing your IP automatically every several minutes: to increase the level of anonymity or to automate tasks like web crawling or data scraping.
If you are looking for increasing your online anonymity, the automatic IP rotation virtually changes your online identity and hides your public IP address. Thus, tracking your browsing or downloading activities becomes more difficult (unlikely, but not impossible).
For data scraping and web crawling the frequent change/hide IP address is the way to go. Can it be done manually? Yes. Is it efficient? No. The auto IP rotation is an excellent solution to the problem.
Automatically change IP prerequisites
The auto-rotate IP is not a trivial task, and the implementation depends on your needs. Before you proceed, you need to ask your self the following: how many IPs you need, on what locations (countries, states) and what is the desired rotating period.
Number of IPs
As mentioned above, if you are in the data scraping business than you need enough different IPs such that you don’t get blocked. How many exactly? It depends on the site you are scraping, on the site firewall, on the scraping frequency, etc. I guess that you need between 20 to 100 IPs for simple tasks, and more for complex data requests.
Number of locations
You may need to change your virtual position with IPs from different parts of the World. Yes, it is possible, but you have to find the proper solution.
Rotating period
How often do you need to have your IP automatically changed? Once every minute, every 5 minutes, once an hour? Some of the solutions to be presented have specific restrictions regarding the rotating frequency, and others are quite flexible. Keep reading.
Methods for rotating IP – Automatic IP changer solutions
I am going to present several solutions that are based on the following approaches:
- Proxies – free solution, requires some manual work, premium proxies available.
- Tor – free solution, depends on the quality of the Tor nodes, more challenging to set up.
- VPN services – the most reliable and safe solution, but the most expensive.
Take a look at the table below that summarizes the main features of the proposed approaches:
| Proxies | Tor | VPNs | |
| Price | Free/Paid | Free | Paid |
| IP switch speed | Fast | Medium | Slow |
| Control over the IPs and locations | Yes | No | It depends on the provider |
| Works with | Browsers and certain apps | All apps | All apps |
| Provides security/encryption | No | Yes | Yes |
1. Elite Proxy Switcher
The first application that I recommend uses proxies: Elite Proxy Switcher. It tests proxies and rotates them. For the free version, you need to get your proxies from proxy list sites. Take a look below at the video tutorial for more details:
After you get the working proxies you may start rotating them: go to Switch > Use the automatic switch and set the switching interval (in seconds). Yes, you may change the IP every second if you need to.
That’s it! You get the auto change IP feature. Elite Proxy Switcher also offers premium proxies that are already tested daily.
The main downside of the free proxies is that they have a short life and are most of the time unreliable. Thus, you need to check them regularly and add new ones all the time. Another issue (valid for all proxy-based solutions) is that they work only for browsers (Chrome, Firefox, Internet Explorer/Edge), not for other applications. Thus, if you use a particular app that performs the data scraping the Elite Proxy Switcher will not work.
2. TOR + Privoxy
This solution is not easy to be implemented as it requests some technical knowledge. If you are not familiar with Tor and you don’t have any Linux skills than this is not for you.
You need to set up Tor and Privoxy locally. Then, use Stem (Python controller library for Tor) to manage Tor. Here is a good tutorial on how to perform these tasks.
The main advantage of the Tor + Privoxy approach is that it is that you get the auto change IP for free. The downsides include the fact that you can not control the IPs and the locations you get (sometimes, even if you change the node you may get the same Tor IP address). Also, the Tor exists may be slow.
3. ibVPN
Next, a VPN-based solution to auto change IP: ibVPN, an excellent VPN service that may be used to rotate IP addresses.
Start with a free trial from the ibVPN site (now migrated to StrongVPN) and install the All-In-One VPN client for Windows (the macOS app does not include the rotating server’s option, at least not now). After login, go to Servers, select the desired locations/servers as Favorites and click on Rotating Favorites. Eventually, click Connect. The app will change your IP every X minutes (the value of X can be set up from the Settings tab from predefined values: 5, 10, 15, 30, 60 minutes). Thus, you may schedule IP address change as often as you need.
To make sure that the web crawler or data scraper app does not use your real IP I recommend you to switch ON the KillSwitch, a function that cuts the Internet connection while the IP is changed. It worth mentioning that the IP switch is slower when using VPNs than when using proxies, as it requires the setup of a VPN connection, a time-consuming operation (5-30 seconds).
4. IPVanish
IPVanish, another top VPN provider, is also a solution to be considered for the periodic IP address change. The Windows and macOS apps offer access to the Periodic IP address Change feature that you may use to rotate IPs. It comes with two drawbacks: the minimum rotating period is 45 minutes, and you may not select the locations/servers to be rotated.
Important! Your VPN connection will be temporarily interrupted while our app changes your IPVanish IP address.
5. HideMyAss
HideMyAss is (or it was) the preferred tool for those performing data scraping. Unfortunately, the latest versions of the HideMyAss apps (3.x) have removed the rotating IPs feature. Most probably it will be added in future releases.
The good news is that you may still install version 2 of the app. The rotating feature can be found in the IP address settings tab, on the left of the application. If you are using one device the minimum time of IP change can be 2.5 minutes, and if you are connecting from 2 machines, you can set 5 minutes or more for the automatic IP change.
6. Backconnect proxies
Another proxy solution for rotating IPs is the one implemented by backconnect proxies.
A backconnect proxy is a proxy server that assigns a different IP address every time a proxy connection is established. Backconnect proxies can be configured to rotate certain types of IPs and given intervals.
Rotating IPs with backconnect proxies has the advantage that no additional software needs to be installed, but they are more expensive than regular proxies.
Conclusion
To sum up, the automatically change IP is possible. Firstly, you should know what your needs are and, secondly, select the proper solution. My advice is to give a try to proxies and ibVPN as it offers a free trial. Enjoy!
Do you know other methods for automatically change the IP address? Let us know your thoughts in the comments below.
Excellent article! I use Elite Proxy Switcher for years with their premium proxies, but I will give ibvpn a try. I really need a backup solution.
HideMyAss was my choice for years. Unfortunately, the latest updates and the changes to the server network made HMA an average VPN providers. I switched to ibvpn.
I use an older version of IPVanish that allows me to setup the switch interval to less than a minute. The new release with the 45 minutes minimum interval is unusable. I will also give a try to ibvpn. Thank you for the tip!
Don’t upgrade to the latest version of hma! Keep version 2 and you will be happy!
I use ibvpn on Windows. Hopefully, they will implement the feature on Mac as well.
Do you know some Chrome extensions capable of changing the IPs every minute or so?
I tried Tor and Privox … too difficult for me. I will go for EliteProxySwitcher.
Am asking what is the best among all… not about price.
Thank you
There is no such thing as “the best among all”. It depends on what you need. I personally prefer HMA and ibVPN.
Hello there,
Very useful information that you provide in your article. It has saved me time. Thank you.
The question is that I’m thinking of using ibvpn to rotate the ips and scrape with Seopowersuite and Scrapebox. But on the ibvpn website there are several plans and prices, and I am hesitating between standar VPN plan and the ultimate VPN plan. Do you think that what I want to use it for is enough standard VPN plan? or better ultimate VPN plan?
Thank you
Standard VPN plan provides VPN + Proxy. Ultimate VPN plan provides VPN + SmartDNS + Proxy, with which the level of encryption and speed is higher. But for what I want, the standard is enough, according to your experience?
I think that the Standard plan should be enough for you. If you are going to only use the proxies, the additional VPN protocols options are not needed.