Under normal circumstances, you’d want search engines like Google, Bing, and Yahoo to list your site on their search results. In fact, most individual website owners and businesses devote so much time and resources to making sure that their website is optimized for SEO (Search Engine Optimization).
However, there are a few cases where you’d want to keep search engine crawlers away from your website. For example, you usually wouldn’t want search engines under the following circumstances:
- When a site is still in the development phase. At this point, the site is not ready to be used by the general public
- When the site is meant to be private and only accessible to selected people—such as your family or work colleagues.
In this article, I’ll walk you through the processes for stopping search engines from finding and indexing your WordPress website.
Turn Off Search Engine Visibility in Admin Panel
The easiest method for stopping search engines from indexing your site is by disabling “search engine visibility” in your WordPress admin dashboard. To do this, follow the instructions below:
- Log into your WordPress account.
- In the sidebar, navigate to Settings > Reading
- Scroll down to Search Engine Visibility.
- Once there, tick the option which says, Discourage search engines from indexing this site.
- Click on Save Changes when done.
You should note that this method is not reliable—it’s up to the search engines to honor your request.
To forcefully stop search engines from finding your website, you’ll need to password-protect it. We’ll discuss what that means and how to do it in the following section.
Protect Your Entire Website With a Password
When you password-protect a WordPress website, anyone looking to access the site will first be required to provide its correct password. As a consequence, search engine bots will be blocked from accessing the site to crawl it.
There are different methods for password-protecting your site. If the WordPress site in question is on a hosting provider, like Bluehost, for example, you can activate a password inside the cPanel of your hosting account.
Though the interface is going to be different for different providers, the process should be similar across them. Basically, you’ll need to search for the “password protect” directory, choose the website that you want to protect (if you have multiple websites) and then create the password for the whole site.
Reach out to your hosting provider if you need more information about the process.
Use a Free Plugin
You can use a plugin to password-protect your entire website, and as a result, stop search engine crawlers from finding it. While there are many plugins for doing this, I’ll guide you on how to use a free plugin called Password Protected.
In the sidebar:
- Navigate to Plugins > Add New
- Use the search bar on the top right corner of the page to search for “Password Protected“.
- Click on Install Now, then click on Activate.
At this point, the plugin is active.
To enable password protection and create the password, navigate to Settings > Password Protected and fill out the form accordingly.
Tick the Enabled option (next to Password Protected Status) to enable password protection, and the Allow Administrators option (next to Protected Permissions) to allow administrators access to the site. (If you want, you can also allow logged-in users, RSS feeds and REST API access as well.)
Next, set the password for the site, and confirm it. If you want to allow specific users to access the site, add their IP addresses in the Allow IP Address box.
Finally, tick the Allow Remember Me option (this ensures that subsequent access doesn’t require the password) and add the number of days to want to remember an authenticated user for.
When you’re done, click Save Changes to activate password protection on the site. From then on, anyone visiting your site will be required to provide its password and search engine crawlers won’t be allowed to access the website.
Use a Paid Plugin to Include Advanced Features
Free plugins like the one we discussed in the last section allow you to password protect your site, no doubt. But if you’re looking for more advanced features, which I’ll list out soon, you can go with a paid plugin.
For example, there’s a plugin called SeedProd. In addition to password protection, this plugin provides you with several nice, descriptive splash screens that you can display on your site to tell the user that maintenance is ongoing.
The plugin also includes “Coming Soon” themes that, when used on the site, will prompt the user to subscribe and be notified when the site goes live (or some other update occurs) through email.
Conclusion
In this article, we saw how to stop search engines from showing our site on search engine results. This is helpful if you only want certain people to access the site as opposed to the general public.
Now that you know how to stop search engines from indexing your site when the need arises, check out some of our other WordPress articles.