How Create Robots.Txt For WordPress Tutorial

Here is a easy-to-follow guide on how to create robots.txt to your WordPress blog.

---

Step One:

Before you do anything, you need to create a robot.txt  file which list out what you want to prevent Googlebot from crawling and also what you want it to crawl.

Just copy the list below on the Notepad and name it 'robots' (without quotes).

Save it anywhere you prefer. It is only for temporary use. Just remember where you've saved it.

Sitemap: http://yourblog.com/sitemap.xml
Sitemap: http://yourblog.com/sitemap-video.xml
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /feed/
Disallow: /trackback/
Disallow: /page/
Disallow: /comments/feed/
Disallow: /search?
Disallow: /category
Disallow: /xmlrpc.php
Disallow: /tag
Disallow: /index.php
Disallow: /wp-content/plugins/
Disallow: /?=*
Disallow: ?comments=*
Disallow: /?p=*

User-agent: Mediapartners-Google*
Allow: /

User-agent: Googlebot-Image
Allow: /wp-content/uploads/

User-agent: Adsbot-Google
Allow: /

 User-agent: Googlebot-Mobile
Allow: /

Note: the yourblog.com needs to be replaced by your own blog URL.

---

Step Two:

1. Logging to your WordPress Admin.

2. Click "CPanel".

3. Inside the "Files" box, click on "File Manager".

4. A menu box will pop up. If you have several blogs, make sure you select the correct one. Then click on the "Go" button.

5. It will take you to a new page called file Manager. At the left hand side of the page and look for the folder public_html.

6. Another box will pop up called "New File". Under "New Name File:", type robots.txt

7. Click "Create New File".

8. Now you should find "robots.txt" under "Name".

9.Click on the plus sign (+) beside the folder icon and it will drop down to show you the name of your blog folder.

10. Click on the blog name and then click Upload icon which is on top of the page.

11. It takes you to another page. Click "Browse..." button. Look for robots.txt file which you have created earlier have it saved it somewhere (mentioned in Step One).

12. Now click "Open".  It will start uploading.

13. That's all.

---

Test Your Robots.txt

Your robots.txt is: http://yourblog.com/robots.txt

Type your robots.txt onto the browser.  It is done correctly you should see the list which you have created.