slow Pharo downloading and Travis

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

slow Pharo downloading and Travis

Peter Uhnák
Hi,

over the past week+ downloading the latest Pharo 6 version has been extremely slow -- 5+ minutes to download the zip.

a) is there a known reason for this?

b) is there a way to know what is the current latest image of particular Pharo without downloading it?

Could we include e.g. "latest.txt" file into http://files.pharo.org/image/60/ (and /70/ etc) that will include the latest image number? Because then we can cache on travis side which could significantly reduce the demands on pharo infrastructure.

Peter
Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Tim Mackinnon
I was wondering thinking something similar - but should this be sitting behind something like cloudflare  if it oss? Then it wouldn't really matter and would be very fast.

Tim



Sent from my iPhone



Sent from my iPhone
On 20 Aug 2017, at 19:09, Peter Uhnák <[hidden email]> wrote:

Hi,

over the past week+ downloading the latest Pharo 6 version has been extremely slow -- 5+ minutes to download the zip.

a) is there a known reason for this?

b) is there a way to know what is the current latest image of particular Pharo without downloading it?

Could we include e.g. "latest.txt" file into http://files.pharo.org/image/60/ (and /70/ etc) that will include the latest image number? Because then we can cache on travis side which could significantly reduce the demands on pharo infrastructure.

Peter
Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Marcus Denker-4
On 20 Aug 2017, at 19:09, Peter Uhnák <[hidden email]> wrote:
a) is there a known reason for this?


No. I will start to check.

b) is there a way to know what is the current latest image of particular Pharo without downloading it?

No, you would need to get a list of files and take the highest number. We should add a files with that info.

On 20 Aug 2017, at 21:06, Tim Mackinnon <[hidden email]> wrote:

I was wondering thinking something similar - but should this be sitting behind something like cloudflare  if it oss? Then it wouldn't really matter and would be very fast.


Cloudflare sadly does not cache any archives (.zip and things), at least in the free version. It is not very clear how it works in the payed plan.

We used in the past  https://www.cdn77.com but performance was not much faster then Inria and the problem is that it needs
some API call to tell the CDN when you upload anything new. Was a bit of a hassle…

Marcus 
Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Sean P. DeNigris
Administrator
In reply to this post by Peter Uhnák
Peter Uhnák wrote
> over the past week+ downloading the latest Pharo 6 version has been
> extremely slow -- 5+ minutes to download the zip.

I don't know if this is related, but Launcher just took several minutes to
"Fetch source files for Pharo 6.0" when I tried to launch an image…



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Marcus Denker-4

> On 31 Aug 2017, at 16:56, Sean P. DeNigris <[hidden email]> wrote:
>
> Peter Uhnák wrote
>> over the past week+ downloading the latest Pharo 6 version has been
>> extremely slow -- 5+ minutes to download the zip.
>
> I don't know if this is related, but Launcher just took several minutes to
> "Fetch source files for Pharo 6.0" when I tried to launch an image…
>
>

I know have opened a ticket to see if the admins know of any Inria related networking
issues.

        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Peter Uhnák
Has there been any progress on this?

Thanks,
Peter

On Thu, Aug 31, 2017 at 5:45 PM, Marcus Denker <[hidden email]> wrote:

> On 31 Aug 2017, at 16:56, Sean P. DeNigris <[hidden email]> wrote:
>
> Peter Uhnák wrote
>> over the past week+ downloading the latest Pharo 6 version has been
>> extremely slow -- 5+ minutes to download the zip.
>
> I don't know if this is related, but Launcher just took several minutes to
> "Fetch source files for Pharo 6.0" when I tried to launch an image…
>
>

I know have opened a ticket to see if the admins know of any Inria related networking
issues.

        Marcus

Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Marcus Denker-4
Sadly not.

The admins can not find a reason for why it is slow.

I think we should evaluate to use S3 instead again.

On 15 Sep 2017, at 11:31, Peter Uhnák <[hidden email]> wrote:

Has there been any progress on this?

Thanks,
Peter

On Thu, Aug 31, 2017 at 5:45 PM, Marcus Denker <[hidden email]> wrote:

> On 31 Aug 2017, at 16:56, Sean P. DeNigris <[hidden email]> wrote:
>
> Peter Uhnák wrote
>> over the past week+ downloading the latest Pharo 6 version has been
>> extremely slow -- 5+ minutes to download the zip.
>
> I don't know if this is related, but Launcher just took several minutes to
> "Fetch source files for Pharo 6.0" when I tried to launch an image…
>
>

I know have opened a ticket to see if the admins know of any Inria related networking
issues.

        Marcus


Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Peter Uhnák
Also this applies not just to files.pharo.org, but also to Inria Jenkins (I just had a build timeout after 10 minutes by trying to download Moose image ).

So in the meantime there's no workaround for this?

Peter

On Fri, Sep 15, 2017 at 11:38 AM, Marcus Denker <[hidden email]> wrote:
Sadly not.

The admins can not find a reason for why it is slow.

I think we should evaluate to use S3 instead again.

On 15 Sep 2017, at 11:31, Peter Uhnák <[hidden email]> wrote:

Has there been any progress on this?

Thanks,
Peter

On Thu, Aug 31, 2017 at 5:45 PM, Marcus Denker <[hidden email]> wrote:

> On 31 Aug 2017, at 16:56, Sean P. DeNigris <[hidden email]> wrote:
>
> Peter Uhnák wrote
>> over the past week+ downloading the latest Pharo 6 version has been
>> extremely slow -- 5+ minutes to download the zip.
>
> I don't know if this is related, but Launcher just took several minutes to
> "Fetch source files for Pharo 6.0" when I tried to launch an image…
>
>

I know have opened a ticket to see if the admins know of any Inria related networking
issues.

        Marcus



Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Tim Mackinnon
Me too - I have started heavily caching things on Gitlab and Dropbox.

I’m surprised they can’t find an issue - something seems wrong.

Tim

On 15 Sep 2017, at 10:50, Peter Uhnák <[hidden email]> wrote:

Also this applies not just to files.pharo.org, but also to Inria Jenkins (I just had a build timeout after 10 minutes by trying to download Moose image ).

So in the meantime there's no workaround for this?

Peter

On Fri, Sep 15, 2017 at 11:38 AM, Marcus Denker <[hidden email]> wrote:
Sadly not.

The admins can not find a reason for why it is slow.

I think we should evaluate to use S3 instead again.

On 15 Sep 2017, at 11:31, Peter Uhnák <[hidden email]> wrote:

Has there been any progress on this?

Thanks,
Peter

On Thu, Aug 31, 2017 at 5:45 PM, Marcus Denker <[hidden email]> wrote:

> On 31 Aug 2017, at 16:56, Sean P. DeNigris <[hidden email]> wrote:
>
> Peter Uhnák wrote
>> over the past week+ downloading the latest Pharo 6 version has been
>> extremely slow -- 5+ minutes to download the zip.
>
> I don't know if this is related, but Launcher just took several minutes to
> "Fetch source files for Pharo 6.0" when I tried to launch an image…
>
>

I know have opened a ticket to see if the admins know of any Inria related networking
issues.

        Marcus




Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Marcus Denker-4
In reply to this post by Peter Uhnák
For files.pharo.org I could enable again the https://www.cdn77.com setup.

I will do that next week.

Marcus

On 15 Sep 2017, at 11:50, Peter Uhnák <[hidden email]> wrote:

Also this applies not just to files.pharo.org, but also to Inria Jenkins (I just had a build timeout after 10 minutes by trying to download Moose image ).

So in the meantime there's no workaround for this?

Peter

On Fri, Sep 15, 2017 at 11:38 AM, Marcus Denker <[hidden email]> wrote:
Sadly not.

The admins can not find a reason for why it is slow.

I think we should evaluate to use S3 instead again.

On 15 Sep 2017, at 11:31, Peter Uhnák <[hidden email]> wrote:

Has there been any progress on this?

Thanks,
Peter

On Thu, Aug 31, 2017 at 5:45 PM, Marcus Denker <[hidden email]> wrote:

> On 31 Aug 2017, at 16:56, Sean P. DeNigris <[hidden email]> wrote:
>
> Peter Uhnák wrote
>> over the past week+ downloading the latest Pharo 6 version has been
>> extremely slow -- 5+ minutes to download the zip.
>
> I don't know if this is related, but Launcher just took several minutes to
> "Fetch source files for Pharo 6.0" when I tried to launch an image…
>
>

I know have opened a ticket to see if the admins know of any Inria related networking
issues.

        Marcus




Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Stephane Ducasse-3
In reply to this post by Marcus Denker-4
Hi marcus

I do not see why they could not find it.
Can you tell me how you ask because we should reask?


On Fri, Sep 15, 2017 at 11:38 AM, Marcus Denker <[hidden email]> wrote:

> Sadly not.
>
> The admins can not find a reason for why it is slow.
>
> I think we should evaluate to use S3 instead again.
>
> On 15 Sep 2017, at 11:31, Peter Uhnák <[hidden email]> wrote:
>
> Has there been any progress on this?
>
> Thanks,
> Peter
>
> On Thu, Aug 31, 2017 at 5:45 PM, Marcus Denker <[hidden email]>
> wrote:
>>
>>
>> > On 31 Aug 2017, at 16:56, Sean P. DeNigris <[hidden email]>
>> > wrote:
>> >
>> > Peter Uhnák wrote
>> >> over the past week+ downloading the latest Pharo 6 version has been
>> >> extremely slow -- 5+ minutes to download the zip.
>> >
>> > I don't know if this is related, but Launcher just took several minutes
>> > to
>> > "Fetch source files for Pharo 6.0" when I tried to launch an image…
>> >
>> >
>>
>> I know have opened a ticket to see if the admins know of any Inria related
>> networking
>> issues.
>>
>>         Marcus
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Marcus Denker-4

> On 16 Sep 2017, at 09:40, Stephane Ducasse <[hidden email]> wrote:
>
> Hi marcus
>
> I do not see why they could not find it.
> Can you tell me how you ask because we should reask?
>
>

I spend a ticket and they wanted to close it last week. I have told them to not close it

I will forward the mails to you.


        Marcus


Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Marcus Denker-4

> On 18 Sep 2017, at 08:37, Marcus Denker <[hidden email]> wrote:
>
>
>> On 16 Sep 2017, at 09:40, Stephane Ducasse <[hidden email]> wrote:
>>
>> Hi marcus
>>
>> I do not see why they could not find it.
>> Can you tell me how you ask because we should reask?
>>
>>
>
> I spend a ticket and they wanted to close it last week. I have told them to not close it
>
The problems seems to be fixed now.

If someone sees any slowness -> send me a mail.

        Marcus


Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

gcotelli
Great, I will test it and let you know how it goes.

Thanks!

On Thu, Sep 21, 2017 at 6:43 AM, Marcus Denker <[hidden email]> wrote:

> On 18 Sep 2017, at 08:37, Marcus Denker <[hidden email]> wrote:
>
>
>> On 16 Sep 2017, at 09:40, Stephane Ducasse <[hidden email]> wrote:
>>
>> Hi marcus
>>
>> I do not see why they could not find it.
>> Can you tell me how you ask because we should reask?
>>
>>
>
> I spend a ticket and they wanted to close it last week. I have told them to not close it
>
The problems seems to be fixed now.

If someone sees any slowness -> send me a mail.

        Marcus



Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Ben Coman
In reply to this post by Marcus Denker-4
Still seems a bit slow...
http://files.pharo.org/get-files/70/pharo64.zip
16MB ==> 10 minutes (and actually it stalled after 3.4MB downloaded)
Local speed test: 40Mb/s down, 20Mb/s up

but then I'm a long way across the internet, the cause could be anywhere...
$ traceroute files.pharo.org
traceroute to files.pharo.org (128.93.162.72), 30 hops max, 60 byte packets
 1  192.168.43.1 (192.168.43.1)  2.040 ms  2.403 ms  4.222 ms
 2  * * *
 3  * * *
 4  Bundle-Ether14.wel-edge901.perth.telstra.net (139.130.16.49)  67.741 ms  67.739 ms  67.730 ms
 5  bundle-ether6.wel-core3.perth.telstra.net (203.50.6.214)  67.696 ms  67.697 ms  67.697 ms
 6  bundle-ether7.fli-core1.adelaide.telstra.net (203.50.11.18)  95.723 ms  91.753 ms  100.574 ms
 7  bundle-ether9.win-core10.melbourne.telstra.net (203.50.11.91)  106.550 ms  108.849 ms  108.851 ms
 8  bundle-ether12.ken-core10.sydney.telstra.net (203.50.11.122)  117.770 ms  117.767 ms  117.751 ms
 9  bundle-ether1.pad-gw11.sydney.telstra.net (203.50.6.61)  117.720 ms  103.603 ms  102.171 ms
10  bundle-ether1.sydp-core04.sydney.reach.com (203.50.13.90)  93.276 ms  95.822 ms  102.577 ms
11  i-52.tlot-core02.bx.telstraglobal.net (202.84.137.101)  330.128 ms  321.231 ms  322.522 ms
12  i-0-4-0-10.tlot-core01.bi.telstraglobal.net (202.84.253.21)  321.187 ms i-0-2-0-10.tlot-core01.bi.telstraglobal.net (202.40.149.181)  321.181 ms i-0-2-0-9.tlot-core01.bi.telstraglobal.net (202.40.149.177)  308.786 ms
13  gtt-peer.tlot02.pr.telstraglobal.net (134.159.63.182)  273.784 ms  283.530 ms  281.079 ms
14  xe-1-2-0.cr0-par7.ip4.gtt.net (89.149.185.53)  466.648 ms  479.910 ms  481.254 ms
15  renater-gw-ix1.gtt.net (77.67.123.206)  479.900 ms  479.882 ms  478.641 ms
16  * * *
17  inria-rocquencourt-te1-4-inria-rtr-021.noc.renater.fr (193.51.184.177)  436.776 ms  479.819 ms  478.907 ms
18  * * *
19  * * *

$ httping -h files.pharo.org
connected to 128.93.162.72:80 (169 bytes), seq=0 time=970.27 ms 
connected to 128.93.162.72:80 (169 bytes), seq=1 time=928.26 ms 
connected to 128.93.162.72:80 (169 bytes), seq=2 time=860.06 ms 
connected to 128.93.162.72:80 (169 bytes), seq=3 time=976.04 ms 
connected to 128.93.162.72:80 (169 bytes), seq=4 time=1238.72 ms 
connected to 128.93.162.72:80 (169 bytes), seq=5 time=1239.65 ms 
connected to 128.93.162.72:80 (169 bytes), seq=6 time=918.06 ms 


cheers -ben

On Thu, Sep 21, 2017 at 5:43 PM, Marcus Denker <[hidden email]> wrote:

> On 18 Sep 2017, at 08:37, Marcus Denker <[hidden email]> wrote:
>
>
>> On 16 Sep 2017, at 09:40, Stephane Ducasse <[hidden email]> wrote:
>>
>> Hi marcus
>>
>> I do not see why they could not find it.
>> Can you tell me how you ask because we should reask?
>>
>>
>
> I spend a ticket and they wanted to close it last week. I have told them to not close it
>
The problems seems to be fixed now.

If someone sees any slowness -> send me a mail.

        Marcus



Reply | Threaded
Open this post in threaded view
|

Re: slow Pharo downloading and Travis

Ben Coman
btw, I got 17MB in good time (~25s) this link...
https://ci.inria.fr/pharo/view/Launcher/job/Launcher/149/PHARO=61,VERSION=bleedingEdge,VM=vm,label=linux/artifact/PharoLauncher-user-bleedingEdge-2017.09.20.zip

So here is my entirely unsubstantiated exotic random troubleshooting tip... 
check for a mismatch in auto-negotiate setting between server ethernet port and etherswitch 
I've seen more than once strange slow downs due to one side being configured for 100Mb, and the other side configured to auto-negotiate and it craps out.  Both sides need to be set autonegotiate, or both sides need to be set 100Mb, not mixed. 

cheers -ben

On Thu, Sep 21, 2017 at 11:13 PM, Ben Coman <[hidden email]> wrote:
Still seems a bit slow...
http://files.pharo.org/get-files/70/pharo64.zip
16MB ==> 10 minutes (and actually it stalled after 3.4MB downloaded)
Local speed test: 40Mb/s down, 20Mb/s up

but then I'm a long way across the internet, the cause could be anywhere...
$ traceroute files.pharo.org
traceroute to files.pharo.org (128.93.162.72), 30 hops max, 60 byte packets
 1  192.168.43.1 (192.168.43.1)  2.040 ms  2.403 ms  4.222 ms
 2  * * *
 3  * * *
 4  Bundle-Ether14.wel-edge901.perth.telstra.net (139.130.16.49)  67.741 ms  67.739 ms  67.730 ms
 5  bundle-ether6.wel-core3.perth.telstra.net (203.50.6.214)  67.696 ms  67.697 ms  67.697 ms
 6  bundle-ether7.fli-core1.adelaide.telstra.net (203.50.11.18)  95.723 ms  91.753 ms  100.574 ms
 7  bundle-ether9.win-core10.melbourne.telstra.net (203.50.11.91)  106.550 ms  108.849 ms  108.851 ms
 8  bundle-ether12.ken-core10.sydney.telstra.net (203.50.11.122)  117.770 ms  117.767 ms  117.751 ms
 9  bundle-ether1.pad-gw11.sydney.telstra.net (203.50.6.61)  117.720 ms  103.603 ms  102.171 ms
10  bundle-ether1.sydp-core04.sydney.reach.com (203.50.13.90)  93.276 ms  95.822 ms  102.577 ms
11  i-52.tlot-core02.bx.telstraglobal.net (202.84.137.101)  330.128 ms  321.231 ms  322.522 ms
12  i-0-4-0-10.tlot-core01.bi.telstraglobal.net (202.84.253.21)  321.187 ms i-0-2-0-10.tlot-core01.bi.telstraglobal.net (202.40.149.181)  321.181 ms i-0-2-0-9.tlot-core01.bi.telstraglobal.net (202.40.149.177)  308.786 ms
13  gtt-peer.tlot02.pr.telstraglobal.net (134.159.63.182)  273.784 ms  283.530 ms  281.079 ms
14  xe-1-2-0.cr0-par7.ip4.gtt.net (89.149.185.53)  466.648 ms  479.910 ms  481.254 ms
15  renater-gw-ix1.gtt.net (77.67.123.206)  479.900 ms  479.882 ms  478.641 ms
16  * * *
17  inria-rocquencourt-te1-4-inria-rtr-021.noc.renater.fr (193.51.184.177)  436.776 ms  479.819 ms  478.907 ms
18  * * *
19  * * *

$ httping -h files.pharo.org
connected to 128.93.162.72:80 (169 bytes), seq=0 time=970.27 ms 
connected to 128.93.162.72:80 (169 bytes), seq=1 time=928.26 ms 
connected to 128.93.162.72:80 (169 bytes), seq=2 time=860.06 ms 
connected to 128.93.162.72:80 (169 bytes), seq=3 time=976.04 ms 
connected to 128.93.162.72:80 (169 bytes), seq=4 time=1238.72 ms 
connected to 128.93.162.72:80 (169 bytes), seq=5 time=1239.65 ms 
connected to 128.93.162.72:80 (169 bytes), seq=6 time=918.06 ms 


cheers -ben

On Thu, Sep 21, 2017 at 5:43 PM, Marcus Denker <[hidden email]> wrote:

> On 18 Sep 2017, at 08:37, Marcus Denker <[hidden email]> wrote:
>
>
>> On 16 Sep 2017, at 09:40, Stephane Ducasse <[hidden email]> wrote:
>>
>> Hi marcus
>>
>> I do not see why they could not find it.
>> Can you tell me how you ask because we should reask?
>>
>>
>
> I spend a ticket and they wanted to close it last week. I have told them to not close it
>
The problems seems to be fixed now.

If someone sees any slowness -> send me a mail.

        Marcus