What is robots.txt?

Discussion in 'SEO and Marketing' started by evargro, May 15, 2019.

Share This Page

  1. evargro

    evargro Active Member

    Joined:
    Jan 21, 2019
    Messages:
    62
    Likes Received:
    0
    What is robots.txt?
     
  2. Anand Saama

    Anand Saama New Member

    Joined:
    Friday
    Messages:
    2
    Likes Received:
    0
    when you use robot.TXT file in you site ..then it means that you are giving a command to google crawler that which page you want to crawl or which is not...by using allow and disallow action
     
  3. Robin Willson

    Robin Willson Member

    Joined:
    Feb 2, 2019
    Messages:
    31
    Likes Received:
    0
    Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website.