[ANN] LatAm Squeaksource Mirror

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] LatAm Squeaksource Mirror

jfabry
Hi all,

I'm happy to announce that we have a public, read-only Squeaksource mirror up and running at the Universidad de Chile. The URL is: http://www.dsal.cl/squeaksource/ TECHNICAL DETAILS BELOW. Feel free to broadcast this announcement!

MOTIVATION:
For us, Switzerland is a long way and the connection to Squeaksource has always been painfully slow. We've set up this mirror to help to address this, and I imagine that it will be of use not only for the Chileans but also other countries in latin america, which is why I am announcing it here.

ITS TRANSPARENT:
I've worked out a way to make the use of this mirror transparent when using Pharo 1.3: you just use monticello as always, and have a setting to redirect loads from Squeaksource to wherever you want. I have successfully loaded Moose and Seaside from the mirror using this package, so it should work for you. And I can tell you, it was blazingly fast: Seaside loaded in 5m45, and most of the time was spent compiling ... ;-)

So try the mirror and the package out, send me your comments, we want to work on this to make Squeaksource a better experience for everybody.

HOW TO USE from Pharo 1.3:
- Get the redirection package from http://www.squeaksource.com/MonticelloRedirect
- To load from the mirror instead of from Squeaksource, do-it of the following:
MRManager redirectFrom: 'http://www.squeaksource.com/' to: 'http://www.dsal.cl/squeaksource/'.
- To remove this redirection: MRManager removeFrom: 'http://www.squeaksource.com/'.
- See comments of MRManager for more info. MRManager list can be useful at some times.
- Only 2 classes, < 10 methods, not counting accessors. The trick is the use of MethodWrappers.
- I did not test this in Squeak or other versions of Pharo. Feel free to try it out in your environment and send me patches if needed.
- There is no UI, contributions are welcome!!

DETAILS ON THE MIRROR
- It's a full copy of Squeaksource, updated daily.
- The mirror is read-only: if you want to make a commit you still have to do it in Squeaksource. This is to avoid all the complications of distributed writes.
- Updates happen once a day, before midnight Chilean time. Updates take typically (up till now) 200 minutes (!), average download speed here is 200KB/s :-(
- The Squeaksource team has been so kind as to provide a list of the packages that are in Squeaksource, refreshed daily and we pass this to wget (with some massaging of the data inbetween).
- The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
- The software that does the mirror is available here: https://github.com/vramiro/easy-wget-mirror

MANY THANKS TO:
- Daniel Galdames for doing the original mirroring tests
- Victor Ramiro for taking Daniels' work and putting it in production
- The Squeaksource team for providing the daily list of packages, especially Fabrizio Perin and Mircea Lungu
- You for trying it out and telling us about it!

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry  
[hidden email] - http://dcc.uchile.cl/~jfabry
PLEIAD Lab - Computer Science Department (DCC) - University of Chile







_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

Philippe Marschall
2011/8/10 Johan Fabry <[hidden email]>:

> Hi all,
>
> I'm happy to announce that we have a public, read-only Squeaksource mirror up and running at the Universidad de Chile. The URL is: http://www.dsal.cl/squeaksource/ TECHNICAL DETAILS BELOW. Feel free to broadcast this announcement!
>
> MOTIVATION:
> For us, Switzerland is a long way and the connection to Squeaksource has always been painfully slow. We've set up this mirror to help to address this, and I imagine that it will be of use not only for the Chileans but also other countries in latin america, which is why I am announcing it here.
>
> ITS TRANSPARENT:
> I've worked out a way to make the use of this mirror transparent when using Pharo 1.3: you just use monticello as always, and have a setting to redirect loads from Squeaksource to wherever you want. I have successfully loaded Moose and Seaside from the mirror using this package, so it should work for you. And I can tell you, it was blazingly fast: Seaside loaded in 5m45, and most of the time was spent compiling ... ;-)
>
> So try the mirror and the package out, send me your comments, we want to work on this to make Squeaksource a better experience for everybody.
>
> HOW TO USE from Pharo 1.3:
> - Get the redirection package from http://www.squeaksource.com/MonticelloRedirect
> - To load from the mirror instead of from Squeaksource, do-it of the following:
> MRManager redirectFrom: 'http://www.squeaksource.com/' to: 'http://www.dsal.cl/squeaksource/'.
> - To remove this redirection: MRManager removeFrom: 'http://www.squeaksource.com/'.
> - See comments of MRManager for more info. MRManager list can be useful at some times.
> - Only 2 classes, < 10 methods, not counting accessors. The trick is the use of MethodWrappers.
> - I did not test this in Squeak or other versions of Pharo. Feel free to try it out in your environment and send me patches if needed.
> - There is no UI, contributions are welcome!!
>
> DETAILS ON THE MIRROR
> - It's a full copy of Squeaksource, updated daily.
> - The mirror is read-only: if you want to make a commit you still have to do it in Squeaksource. This is to avoid all the complications of distributed writes.
> - Updates happen once a day, before midnight Chilean time. Updates take typically (up till now) 200 minutes (!), average download speed here is 200KB/s :-(
> - The Squeaksource team has been so kind as to provide a list of the packages that are in Squeaksource, refreshed daily and we pass this to wget (with some massaging of the data inbetween).
> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
> - The software that does the mirror is available here: https://github.com/vramiro/easy-wget-mirror
>
> MANY THANKS TO:
> - Daniel Galdames for doing the original mirroring tests
> - Victor Ramiro for taking Daniels' work and putting it in production
> - The Squeaksource team for providing the daily list of packages, especially Fabrizio Perin and Mircea Lungu
> - You for trying it out and telling us about it!

Does this respect private projects?

Cheers
Philippe

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

jfabry

Yes, we respect private projects: we only mirror files that are downloadable without having to log in. Private projects should not be downloadable without having to log in.

On 10 Aug 2011, at 11:15, Philippe Marschall wrote:

> Does this respect private projects?
>
> Cheers
> Philippe



---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry  
[hidden email] - http://dcc.uchile.cl/~jfabry
PLEIAD Lab - Computer Science Department (DCC) - University of Chile







_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

Lukas Renggli
In reply to this post by jfabry
> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt

Please remove/fix this listing IMMEDIATELY, this exposes private projects!

Lukas

--
Lukas Renggli
www.lukas-renggli.ch

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] [ANN] LatAm Squeaksource Mirror

Stéphane Ducasse
In reply to this post by jfabry
Thanks!

Great initiative.

Stef

On Aug 10, 2011, at 5:09 PM, Johan Fabry wrote:

> Hi all,
>
> I'm happy to announce that we have a public, read-only Squeaksource mirror up and running at the Universidad de Chile. The URL is: http://www.dsal.cl/squeaksource/ TECHNICAL DETAILS BELOW. Feel free to broadcast this announcement!
>
> MOTIVATION:
> For us, Switzerland is a long way and the connection to Squeaksource has always been painfully slow. We've set up this mirror to help to address this, and I imagine that it will be of use not only for the Chileans but also other countries in latin america, which is why I am announcing it here.
>
> ITS TRANSPARENT:
> I've worked out a way to make the use of this mirror transparent when using Pharo 1.3: you just use monticello as always, and have a setting to redirect loads from Squeaksource to wherever you want. I have successfully loaded Moose and Seaside from the mirror using this package, so it should work for you. And I can tell you, it was blazingly fast: Seaside loaded in 5m45, and most of the time was spent compiling ... ;-)
>
> So try the mirror and the package out, send me your comments, we want to work on this to make Squeaksource a better experience for everybody.
>
> HOW TO USE from Pharo 1.3:
> - Get the redirection package from http://www.squeaksource.com/MonticelloRedirect
> - To load from the mirror instead of from Squeaksource, do-it of the following:
> MRManager redirectFrom: 'http://www.squeaksource.com/' to: 'http://www.dsal.cl/squeaksource/'.
> - To remove this redirection: MRManager removeFrom: 'http://www.squeaksource.com/'.
> - See comments of MRManager for more info. MRManager list can be useful at some times.
> - Only 2 classes, < 10 methods, not counting accessors. The trick is the use of MethodWrappers.
> - I did not test this in Squeak or other versions of Pharo. Feel free to try it out in your environment and send me patches if needed.
> - There is no UI, contributions are welcome!!
>
> DETAILS ON THE MIRROR
> - It's a full copy of Squeaksource, updated daily.
> - The mirror is read-only: if you want to make a commit you still have to do it in Squeaksource. This is to avoid all the complications of distributed writes.
> - Updates happen once a day, before midnight Chilean time. Updates take typically (up till now) 200 minutes (!), average download speed here is 200KB/s :-(
> - The Squeaksource team has been so kind as to provide a list of the packages that are in Squeaksource, refreshed daily and we pass this to wget (with some massaging of the data inbetween).
> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
> - The software that does the mirror is available here: https://github.com/vramiro/easy-wget-mirror
>
> MANY THANKS TO:
> - Daniel Galdames for doing the original mirroring tests
> - Victor Ramiro for taking Daniels' work and putting it in production
> - The Squeaksource team for providing the daily list of packages, especially Fabrizio Perin and Mircea Lungu
> - You for trying it out and telling us about it!
>
> ---> Save our in-boxes! http://emailcharter.org <---
>
> Johan Fabry  
> [hidden email] - http://dcc.uchile.cl/~jfabry
> PLEIAD Lab - Computer Science Department (DCC) - University of Chile
>
>
>
>
>
>
>


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

jfabry
In reply to this post by Lukas Renggli

On 10 Aug 2011, at 12:44, Lukas Renggli wrote:

>> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
>
> Please remove/fix this listing IMMEDIATELY, this exposes private projects!


Exposing private data was not supposed to happen, I'm very sorry for this! I am taking the mirror offline until this is fixed.

Again, my apologies.

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry  
[hidden email] - http://dcc.uchile.cl/~jfabry
PLEIAD Lab - Computer Science Department (DCC) - University of Chile







_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

Lukas Renggli
http://scg.unibe.ch/download/SSProjectList/projectList.txt is still there.

It exposes the name, and sub sequentially the title and front-page of
private projects. Nothing that you couldn't access if you guessed the
name, but still ...

Lukas

On 10 August 2011 19:11, Johan Fabry <[hidden email]> wrote:

>
> On 10 Aug 2011, at 12:44, Lukas Renggli wrote:
>
>>> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
>>
>> Please remove/fix this listing IMMEDIATELY, this exposes private projects!
>
>
> Exposing private data was not supposed to happen, I'm very sorry for this! I am taking the mirror offline until this is fixed.
>
> Again, my apologies.
>
> ---> Save our in-boxes! http://emailcharter.org <---
>
> Johan Fabry
> [hidden email] - http://dcc.uchile.cl/~jfabry
> PLEIAD Lab - Computer Science Department (DCC) - University of Chile
>
>
>
>
>
>
>



--
Lukas Renggli
www.lukas-renggli.ch

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

jfabry

To clarify:
- the mirror is offline. I have no control over scg.unibe.ch so I cannot take the list offline. :-(
- source code of private projects was never mirrored.

The last in more detail: I manually ran over the list looking for projects that are not visible in the Squeaksource project list page.  If the 'Global' config setting for the project was set to 'No Access' the mirror does *NOT* have the .mcf for these packages. So the code of private projects remained private.

On 10 Aug 2011, at 13:21, Lukas Renggli wrote:

> http://scg.unibe.ch/download/SSProjectList/projectList.txt is still there.
>
> It exposes the name, and sub sequentially the title and front-page of
> private projects. Nothing that you couldn't access if you guessed the
> name, but still ...
>
> Lukas
>
> On 10 August 2011 19:11, Johan Fabry <[hidden email]> wrote:
>>
>> On 10 Aug 2011, at 12:44, Lukas Renggli wrote:
>>
>>>> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
>>>
>>> Please remove/fix this listing IMMEDIATELY, this exposes private projects!
>>
>>
>> Exposing private data was not supposed to happen, I'm very sorry for this! I am taking the mirror offline until this is fixed.
>>
>> Again, my apologies.
>>
>> ---> Save our in-boxes! http://emailcharter.org <---
>>
>> Johan Fabry
>> [hidden email] - http://dcc.uchile.cl/~jfabry
>> PLEIAD Lab - Computer Science Department (DCC) - University of Chile
>>
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Lukas Renggli
> www.lukas-renggli.ch



---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry  
[hidden email] - http://dcc.uchile.cl/~jfabry
PLEIAD Lab - Computer Science Department (DCC) - University of Chile







_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

Ricardo Moran
In reply to this post by Lukas Renggli

On Wed, Aug 10, 2011 at 2:21 PM, Lukas Renggli <[hidden email]> wrote:
http://scg.unibe.ch/download/SSProjectList/projectList.txt is still there.

It exposes the name, and sub sequentially the title and front-page of
private projects. Nothing that you couldn't access if you guessed the
name, but still ...

Hi Lukas,

You don't need to guess their names because Squeaksource already shows them in the project list, even though they are private.

Cheers,
Richo


 

Lukas

On 10 August 2011 19:11, Johan Fabry <[hidden email]> wrote:
>
> On 10 Aug 2011, at 12:44, Lukas Renggli wrote:
>
>>> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
>>
>> Please remove/fix this listing IMMEDIATELY, this exposes private projects!
>
>
> Exposing private data was not supposed to happen, I'm very sorry for this! I am taking the mirror offline until this is fixed.
>
> Again, my apologies.
>
> ---> Save our in-boxes! http://emailcharter.org <---
>
> Johan Fabry
> [hidden email] - http://dcc.uchile.cl/~jfabry
> PLEIAD Lab - Computer Science Department (DCC) - University of Chile
>
>
>
>
>
>
>



--
Lukas Renggli
www.lukas-renggli.ch

_______________________________________________


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

Lukas Renggli
Hmm, this is true :-(

Maybe somebody changed that at some point for performance reasons? Private projects are definitely not visible in the original implementation (source.lukas-renggli.ch has many private projects that don't show up).

Lukas

On Thursday, 11 August 2011, Ricardo Moran <[hidden email]> wrote:
>
> On Wed, Aug 10, 2011 at 2:21 PM, Lukas Renggli <[hidden email]> wrote:
>>
>> http://scg.unibe.ch/download/SSProjectList/projectList.txt <http://scg.unibe.ch/download/SSProjectList/projectList.txt> is still there.
>>
>> It exposes the name, and sub sequentially the title and front-page of
>> private projects. Nothing that you couldn't access if you guessed the
>> name, but still ...
>
> Hi Lukas,
> You don't need to guess their names because Squeaksource already shows them in the project list, even though they are private.
> Cheers,
> Richo
>
>  
>>
>> Lukas
>>
>> On 10 August 2011 19:11, Johan Fabry <[hidden email]> wrote:
>> >
>> > On 10 Aug 2011, at 12:44, Lukas Renggli wrote:
>> >
>> >>> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
>> >>
>> >> Please remove/fix this listing IMMEDIATELY, this exposes private projects!
>> >
>> >
>> > Exposing private data was not supposed to happen, I'm very sorry for this! I am taking the mirror offline until this is fixed.
>> >
>> > Again, my apologies.
>> >
>> > ---> Save our in-boxes! http://emailcharter.org <---
>> >
>> > Johan Fabry
>> > [hidden email] - http://dcc.uchile.cl/~jfabry
>> > PLEIAD Lab - Computer Science Department (DCC) - University of Chile
>> >

>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Lukas Renggli
>> www.lukas-renggli.ch <http://www.lukas-renggli.ch>
>>
>> _______________________________________________
>> Esug-list mailing list
>> [hidden email] <[hidden email]>
>> http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org <http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org>
>
>

--
Lukas Renggli
www.lukas-renggli.ch

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] LatAm Squeaksource Mirror

Lukas Renggli
In reply to this post by Ricardo Moran
Hmm, this is true :-(

Maybe somebody changed that at some point for performance reasons? Private projects are definitely not visible in the original implementation (source.lukas-renggli.ch has many private projects that don't show up).

Lukas

On Thursday, 11 August 2011, Ricardo Moran <[hidden email]> wrote:
>
> On Wed, Aug 10, 2011 at 2:21 PM, Lukas Renggli <[hidden email]> wrote:
>>
>> http://scg.unibe.ch/download/SSProjectList/projectList.txt <http://scg.unibe.ch/download/SSProjectList/projectList.txt> is still there.
>>
>> It exposes the name, and sub sequentially the title and front-page of
>> private projects. Nothing that you couldn't access if you guessed the
>> name, but still ...
>
> Hi Lukas,
> You don't need to guess their names because Squeaksource already shows them in the project list, even though they are private.
> Cheers,
> Richo
>
>  
>>
>> Lukas
>>
>> On 10 August 2011 19:11, Johan Fabry <[hidden email]> wrote:
>> >
>> > On 10 Aug 2011, at 12:44, Lukas Renggli wrote:
>> >
>> >>> - The Squeaksource list of packages is available here: http://scg.unibe.ch/download/SSProjectList/projectList.txt
>> >>
>> >> Please remove/fix this listing IMMEDIATELY, this exposes private projects!
>> >
>> >
>> > Exposing private data was not supposed to happen, I'm very sorry for this! I am taking the mirror offline until this is fixed.
>> >
>> > Again, my apologies.
>> >
>> > ---> Save our in-boxes! http://emailcharter.org <---
>> >
>> > Johan Fabry
>> > [hidden email] - http://dcc.uchile.cl/~jfabry
>> > PLEIAD Lab - Computer Science Department (DCC) - University of Chile
>> >

>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Lukas Renggli
>> www.lukas-renggli.ch <http://www.lukas-renggli.ch>
>>
>> _______________________________________________
>> Esug-list mailing list
>> [hidden email] <[hidden email]>
>> http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org <http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org>
>
>

--
Lukas Renggli
www.lukas-renggli.ch

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org