Old 15th April 2005, 06:44   #1
andrewwkyz
Senior Member
 
Join Date: Jun 2004
Location: Miami, Memphis, LA, All over
Posts: 225
Send a message via AIM to andrewwkyz
Robot.txt

I been getting this on my lo-band server invalid request robot.txt What is that?
andrewwkyz is offline   Reply With Quote
Old 15th April 2005, 07:09   #2
djSpinnerCee
Forum King
 
djSpinnerCee's Avatar
 
Join Date: Aug 2004
Location: Hollis, Queens/The Bronx, NYC
Posts: 3,502
The googlebot and other search engine bots [webcrawlers] must read and respect a robots.txt if they find one [it will fail for your DNAS] -- It means you may get "listed" in some search engine -- notice the U:agent string -- it should tell you what bot it is.

There's nothing you can do about it, it's a good thing.
djSpinnerCee is offline   Reply With Quote
Old 15th April 2005, 17:03   #3
andrewwkyz
Senior Member
 
Join Date: Jun 2004
Location: Miami, Memphis, LA, All over
Posts: 225
Send a message via AIM to andrewwkyz
ok I was just wondering what it was. It always had the same UP and it does not list an agent. just robot.txt
andrewwkyz is offline   Reply With Quote
Reply
Go Back   Winamp & Shoutcast Forums > Shoutcast > Shoutcast Technical Support

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump