"resource request"

[Milten]

Hello, today i've found in my log this thing:

<09/07/06@10:34:09> [dest: 65.54.188.144] Invalid resource request(/robots.txt)

Is taht mean that someone wants to open file named "robots.txt" from my server?
Is it possible to create some files and then someone can look to them?
And can I change look of page that i see when i type "http://localhostort"??

[tuckerm]

It's a robot looking at your stream. More than likely its a bot for google, etc.

OrgName: Microsoft Corp
OrgID: MSFT
Address: One Microsoft Way
City: Redmond
StateProv: WA
PostalCode: 98052
Country: US

^^ Thats what i get when i do a who is ^^

[fc*uk]

yep. Microsoft server out of silicon valley I would guess as the traceroute dies in San Jose, Ca. ....

[Milten]

Thanks, and can I use this fact in my radio? (can I make file named robots.txt and save there for ex. info about my station?)

[djSpinnerCee]

robots.txt is used to prohibit access by searchengine bots -- the DNAS does not have one, so that tells the bot it's ok to crawl the entire DNAS site -- you should see a subsequent request for / or /index.html, or whatever page the bot was following the link to.

Proper bot behavior dictates that a bot look for /robots.txt before following any link to an HTTPd.

You cannot make the DNAS have a /robots.txt, but you can make sure that your directory entry in the YP has a proper Station URL, even if it is the link to your DNAS /index.html (the station page) -- leaving it at the shoutcast directory will obviously not help. TIP: don't use the "/" link because all bots dont have "mozilla" in their useragent string, and that would cause some bots to try to get the MP3 stream instead of the info pages -- it is will fail because the response (the stream) is the ICY protocol, not HTTP. and the DNAS will return another "invalid resource request" to the bot.