#What to put into the robots.txt file
#The "/robots.txt" file usually contains a record looking like this:

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /user/1
    Disallow: /user/*