Friday, 22 March 2013

Tagged under: , , ,

How To Add Robots.txt file into Blogger Blog

How to add robots.txt file in blogger
Robots.txt file in blogger blog play a vital role in SEO. Robots.txt file will surely increase your SEO rank if your put it in your blogger dashboard in proper way. Robots.txt file comes with new blogger interface. We always talking about blogger search preferences but we forget about the basic tools which can make our blog SEO friendly in a seconds. Now a days robot.txt file used by many pro bloggers that is why because of increasing theirs blog traffics and it also helps to eliminate some unpopular parts of your blog from search indexing. So the today article is all about robot.txt file.

Robots.txt file instruct the search engine bot and tells them which parts of the blog should be access and which parts of the blog should be blocked from indexing forever. Whenever bots search your blog it first check the robot.txt file and follow all the instructions which are given by the robot.txt file.


Custom Robots Header Tags

How to Add robots.txt file to blogger blog

  • Go To Blogger >> Settings >> Search Preferences
  • In The Crawlers and Indexing Sections, Look For The Custom Robots.txt
  • Click On Yes and a Box Will Appear Like The Image Below

Robots.txt file in blogger
  • Now Copy Below Following Code And Paste it in the box:
User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap: http://www.probloggertricks.com/feeds/posts/default?orderby=UPDATED
  • Replace the red color my blog name your own blog address
  • Finaly save click on save and you have done almost.

  Study different part of Robots.txt file


User-agent: *
Disallow: /

These robots.txt lines tells the search robots to allow all the majors Search engines to crawl the home page. Means The User-agent: means the google search search robots and the sign (*) is used for all robots that is Google, Yahoo, Bing etc. While the Disallow:/ means if you don't want to allow search robots to access any particular page or directory existing on your sites/blog. See the another example for disallowing a page from my site:

User-agent: *
Disallow: /images.html

In above robots.txt file we disallowed the robots to not access the page images.html because we don't want robots to even access this page.

  • This was just only study about the various parts and lines of robots.txt file you just only need to copy the given code in the first section and paste in that box..thats is.
  • Remember you must change probloggertricks with your own blog name.   

Google SEO Book
Dynamic Meta Description

23 comments:

  1. Unable to understand it. I have some confusion by using this may be google stop indexing my site. Please let me know bit detail about robots.txt if it is useful then why blogger disable it? This warning create feat in my mind

    Warning! Use with caution. Incorrect use of these features can result in your blog being ignored by search engines.

    ReplyDelete
  2. Unable to understand it. I have some confusion by using this may be google stop indexing my site. Please let me know bit detail about robots.txt if it is useful then why blogger disable it by default? This warning scared me.

    Warning! Use with caution. Incorrect use of these features can result in your blog being ignored by search engines.

    ReplyDelete
  3. @Shahzad
    Exactly its true if you are using it incorrect google will must not index your site.

    Use only that part which i have given in the above section only change my blog address with your own blog addresss.

    I am also using this...ADVANTAGE. BY using this file google will not index the labels on your site. if you are unable to use this file google robots will index your label and it may cause duplicate issues for robots. thats in results your blog will be never rank..

    I hope this answer will cover your question.

    ReplyDelete
  4. @Asghar
    Oh Man thanks for your Appreciations well good to see you every time...keep in touch with us..

    ReplyDelete
    Replies
    1. i add this code but mt pageshow the same problem.the problem is not solve what i do email id ksnadeem1@gmail.com

      Delete
  5. @Shahzd
    Bro you are welcome always.

    ReplyDelete
  6. Thanks Buddy keep in touch for newly updates

    ReplyDelete
  7. you said that remove Disallow: /search this code so am i remove this code from my blog....just keep allow option...

    and disallow particular post when it is disallowed that means web crawler cant index any post is it.....
    lastly am i used ur code which ur are given nor there is any change please confirm. if there is change needed please give me code...

    ReplyDelete
  8. It is really effective as I had applied this method about a month ago and I am really getting good response. Mine visitors and indexing of posts have become quite fast by this method.

    Really effective and great post for blogger's. :)
    Keep it up :)

    ReplyDelete
  9. What will happen if i don't use these robot does it effect my blog badly or its just a formality.
    http://freedsoftwares.blogspot.com/

    ReplyDelete
    Replies
    1. Yes it will beat your rank in search result. By adding robot.txt file it will block un necessary parts of your blog, like archive and labels

      Delete
  10. Bro :)
    Will this help Blog getting more traffic ?

    ReplyDelete
  11. Bro the code you provided guides the bots not to index "/ search " ,can you tell me what really this part is and why should i disallow it for the robots to index????

    ReplyDelete
  12. So using labels will be waste of time with this robot file

    ReplyDelete
  13. great article.....keep posting
    http://www.askdanyal.com/

    ReplyDelete
  14. This comment has been removed by a blog administrator.

    ReplyDelete
  15. Assalamuwalaikum Hafeez Khan bro .
    i am suffering with this problem >Warning! Use with caution. Incorrect use of these features can result in your blog being ignored by search engines.
    I am newbie at all.. recently i have make a blog > www.allfreeebooks.net for all ebooks lover.
    i tired to use meta tag . then getting those warning !!
    i have flowed your advice > User-agent: Mediapartners-Google
    Disallow:

    User-agent: *
    Disallow: /search
    Allow: /

    Sitemap: http://www.allfreeebooks.net/feeds/posts/default?orderby=UPDATED
    and putted this code in my custom robots.txt box . but still showing same massage >Warning! Use with caution. Incorrect use of these features can result in your blog being ignored by search engines. :( plz bro can you tell me now actually where is the main prob . plz share your valuable advice ... thanks in advance

    ReplyDelete
    Replies
    1. Buddy this message show on every blogger account dashboard>>you have to not worry about this..it means...and give this message :>> If you are using anything wrong inside these option then google may not crawl your blog. thats is.

      Warning message show on every dashboard, you can check another blogger account. thanks.

      Delete
  16. hey hefeez bro will it rank up my site on google ?

    ReplyDelete

About ProBloggerTricks

Founded in 2013, Pro Blogger Tricks is the ultimate resource for bloggers and marketers. Looking for ways to make money online? Need blogging tips, SEO tips and other business and marketing related content? Probably, the best source of information about monetizing a blog ....Read More

Copyrights @ 2014, Pro Blogger Tricks - Designed By Templateism