Smalltalk › Squeak › Squeak - Dev

[Q] How deal with unwanted looking to private stuff?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

Edgar J. De Cleene

[Q] How deal with unwanted looking to private stuff?

[Q] How deal with unwanted looking to private stuff? Folks:

This days I working on HVNaughtieWiki , send info for feedback to squeakros and to aida list.

Besides some squeakers looking this work in progress, this morning I have the attention of Google

This is the print string of request.

HttpRequest (URL=/file/squeak.sts; protocol=HTTP/1.1; header=a Dictionary('accept'->'*/*' 'accept-encoding'->'gzip,deflate' 'connection'->'Keep-alive' 'from'->'googlebot(at)googlebot.com' 'host'->'190.193.89.80:8085' 'user-agent'->'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' ); getFields=a Dictionary(); postFields=a Dictionary())

All is Comanche + HV2 and some on top.

How deal with this?

Refuse the connect ?
Send a error page?

Any hints is welcomed

Edgar

Igor Stasenko

Re: [Q] How deal with unwanted looking to private stuff?

I think, you should define rules for bots in robots.txt file, and put
there anything which you want robots to index, and what's not.
See http://www.robotstxt.org/robotstxt.html

On 29 January 2010 11:14, Edgar J. De Cleene <[hidden email]> wrote:

> Folks:
>
> This days I working on HVNaughtieWiki , send info for feedback to squeakros
> and to aida list.
>
> Besides some squeakers looking this work in progress, this morning I have
> the attention of Google
>
> This is the print string of request.
>
> HttpRequest (URL=/file/squeak.sts; protocol=HTTP/1.1; header=a
> Dictionary('accept'->'*/*' 'accept-encoding'->'gzip,deflate'
> 'connection'->'Keep-alive' 'from'->'googlebot(at)googlebot.com'
> 'host'->'190.193.89.80:8085' 'user-agent'->'Mozilla/5.0 (compatible;
> Googlebot/2.1; +http://www.google.com/bot.html)' ); getFields=a
> Dictionary(); postFields=a Dictionary())
>
> All is Comanche + HV2 and some on top.
>
> How deal with this?
>
> Refuse the connect ?
> Send a error page?
>
> Any hints is welcomed
>
> Edgar
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

Edgar J. De Cleene

Re: [Q] How deal with unwanted looking to private stuff?

On 1/29/10 8:17 AM, "Igor Stasenko" <[hidden email]> wrote:

> I think, you should define rules for bots in robots.txt file, and put
> there anything which you want robots to index, and what's not.
> See http://www.robotstxt.org/robotstxt.html

Very thanks!

Edgar J. De Cleene

Re: [Q] How deal with unwanted looking to private stuff?

In reply to this post by Igor Stasenko

HVNaughtieWiki-dispatchOn.st (1K) Download Attachment