Port numbers in network tests

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Port numbers in network tests

Frank Shearar-3
So this isn't exactly a common use case, except on build.squeak.org:

You have a bunch of network tests, so in your #setUp you start up a
server on some port and in #tearDown you shut down the server. What
port do you use? Well, it doesn't really matter too much, as long as
the port's available. So you choose some probably-not-used port, like
7799. (WebClient's test suite uses this.) All is almost always well,
because tests only fail when something's using 7799, which is almost
never.

build.squeak.org has two executors, which means it can run two builds
concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
Which means you will often execute the same test suite in different
images at the same time. Suddenly those tests using a hard-coded port
all fail, causing noise. (Xtreams' test suite is also vulnerable to
this problem: look at the  XTSocketReadingWritingTest failures in
http://build.squeak.org/job/ExternalPackages/43/testReport/)

So what do we do?

One approach that I can think of is, like WebClient's test suite, to
use a method #port returning the port to use in a test. However, #port
should not return a constant but rather attempt to open a port within
some range, and return _some_ open port. The problem here is a
time-of-check-time-of-use race.

Now that might nearly always work. Can anyone think of a better method?

frank

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Frank Shearar-3
On 4 April 2013 11:16, Frank Shearar <[hidden email]> wrote:

> So this isn't exactly a common use case, except on build.squeak.org:
>
> You have a bunch of network tests, so in your #setUp you start up a
> server on some port and in #tearDown you shut down the server. What
> port do you use? Well, it doesn't really matter too much, as long as
> the port's available. So you choose some probably-not-used port, like
> 7799. (WebClient's test suite uses this.) All is almost always well,
> because tests only fail when something's using 7799, which is almost
> never.
>
> build.squeak.org has two executors, which means it can run two builds
> concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
> ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
> Which means you will often execute the same test suite in different
> images at the same time. Suddenly those tests using a hard-coded port
> all fail, causing noise. (Xtreams' test suite is also vulnerable to
> this problem: look at the  XTSocketReadingWritingTest failures in
> http://build.squeak.org/job/ExternalPackages/43/testReport/)
>
> So what do we do?
>
> One approach that I can think of is, like WebClient's test suite, to
> use a method #port returning the port to use in a test. However, #port
> should not return a constant but rather attempt to open a port within
> some range, and return _some_ open port. The problem here is a
> time-of-check-time-of-use race.

In particular, this doesn't break any tests (but haven't yet verified
that it works for the above use case!):

port
    | freePort socket |
    port ifNotNil: [^ port].
    freePort := 7766.
    socket := Socket newTCP.
    [[socket listenOn: freePort.
    freePort := freePort + 1.
    socket isWaitingForConnection] whileFalse]
        ensure: [socket destroy].
    ^ port := freePort.

where port is a new instvar that allows us to remember the free port
we just found. Note that this only works because of TestCase's (too)
broad remit; one TestCase instance runs one test.

frank

> Now that might nearly always work. Can anyone think of a better method?
>
> frank

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Colin Putney-3



On Thu, Apr 4, 2013 at 5:30 AM, Frank Shearar <[hidden email]> wrote:
 
In particular, this doesn't break any tests (but haven't yet verified
that it works for the above use case!):

port
    | freePort socket |
    port ifNotNil: [^ port].
    freePort := 7766.
    socket := Socket newTCP.
    [[socket listenOn: freePort.
    freePort := freePort + 1.
    socket isWaitingForConnection] whileFalse]
        ensure: [socket destroy].
    ^ port := freePort.

where port is a new instvar that allows us to remember the free port
we just found. Note that this only works because of TestCase's (too)
broad remit; one TestCase instance runs one test.

I suggest trying ports randomly, rather than sequentially, as there's lower chance of collision.

Also, TestCase's broad remit is designed for exactly this sort of situation. It's a feature! :-)

Colin


Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Frank Shearar-3
On 4 April 2013 17:34, Colin Putney <[hidden email]> wrote:

>
>
>
> On Thu, Apr 4, 2013 at 5:30 AM, Frank Shearar <[hidden email]>
> wrote:
>
>>
>> In particular, this doesn't break any tests (but haven't yet verified
>> that it works for the above use case!):
>>
>> port
>>     | freePort socket |
>>     port ifNotNil: [^ port].
>>     freePort := 7766.
>>     socket := Socket newTCP.
>>     [[socket listenOn: freePort.
>>     freePort := freePort + 1.
>>     socket isWaitingForConnection] whileFalse]
>>         ensure: [socket destroy].
>>     ^ port := freePort.
>>
>> where port is a new instvar that allows us to remember the free port
>> we just found. Note that this only works because of TestCase's (too)
>> broad remit; one TestCase instance runs one test.
>
>
> I suggest trying ports randomly, rather than sequentially, as there's lower
> chance of collision.

Sure, so we'd say

    freePort := (10000 to: 40000) atRandom.

instead.

> Also, TestCase's broad remit is designed for exactly this sort of situation.
> It's a feature! :-)

That's true today, certainly :)

OK, so it's not a crazy implementation? It's been a long time since I
worked with sockets, and pretty much the first time I've worked with
sockets in Squeak.

frank

> Colin

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Colin Putney-3



On Thu, Apr 4, 2013 at 10:18 AM, Frank Shearar <[hidden email]> wrote:
 
OK, so it's not a crazy implementation? It's been a long time since I
worked with sockets, and pretty much the first time I've worked with
sockets in Squeak.

I think that in practice, I'd just pick a random port (1025 to 65535) and answer that. Between the 2^15 possible ports and the short time that each test occupies a port, chances of collision are low, and if one does occur, well, it's not a catastrophe.

Your "crazy" implementation might end up causing *more* failures, because the OS sometimes takes a short while to clean up after a socket is closed, and attempts to listen on the same port will fail until the clean up is done.

Colin


Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Frank Shearar-3
On 4 April 2013 18:32, Colin Putney <[hidden email]> wrote:

>
>
>
> On Thu, Apr 4, 2013 at 10:18 AM, Frank Shearar <[hidden email]>
> wrote:
>
>>
>> OK, so it's not a crazy implementation? It's been a long time since I
>> worked with sockets, and pretty much the first time I've worked with
>> sockets in Squeak.
>
>
> I think that in practice, I'd just pick a random port (1025 to 65535) and
> answer that. Between the 2^15 possible ports and the short time that each
> test occupies a port, chances of collision are low, and if one does occur,
> well, it's not a catastrophe.

Hm, I guess...

> Your "crazy" implementation might end up causing *more* failures, because
> the OS sometimes takes a short while to clean up after a socket is closed,
> and attempts to listen on the same port will fail until the clean up is
> done.

OK. I haven't looked at the Xtreams socket test setUp yet, but I'm
betting it's also simply a hardcoded value. At any rate, I'll submit a
fix for WebClient soon.

frank

> Colin
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Yanni Chiu
In reply to this post by Frank Shearar-3
On 04/04/13 6:16 AM, Frank Shearar wrote:
>
> build.squeak.org has two executors, which means it can run two builds
> concurrently.

How about using the EXECUTOR_NUMBER from the environment variables that
are set by Jenkins. You could just add the value to a base port number,
and default to zero for non-Jenkins environments.


Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Levente Uzonyi-2
In reply to this post by Frank Shearar-3
On Thu, 4 Apr 2013, Frank Shearar wrote:

> So this isn't exactly a common use case, except on build.squeak.org:
>
> You have a bunch of network tests, so in your #setUp you start up a
> server on some port and in #tearDown you shut down the server. What
> port do you use? Well, it doesn't really matter too much, as long as
> the port's available. So you choose some probably-not-used port, like
> 7799. (WebClient's test suite uses this.) All is almost always well,
> because tests only fail when something's using 7799, which is almost
> never.
>
> build.squeak.org has two executors, which means it can run two builds
> concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
> ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
> Which means you will often execute the same test suite in different
> images at the same time. Suddenly those tests using a hard-coded port
> all fail, causing noise. (Xtreams' test suite is also vulnerable to
> this problem: look at the  XTSocketReadingWritingTest failures in
> http://build.squeak.org/job/ExternalPackages/43/testReport/)
>
> So what do we do?
>
> One approach that I can think of is, like WebClient's test suite, to
> use a method #port returning the port to use in a test. However, #port
> should not return a constant but rather attempt to open a port within
> some range, and return _some_ open port. The problem here is a
> time-of-check-time-of-use race.
>
> Now that might nearly always work. Can anyone think of a better method?

The current socket implementation does exactly what you need. If a port
is being used, the socket will listen on a random free port. So the
solution is to ask the created Socket instance for which port it's
listening on.

Code showing the behavior:

s := Socket newTCP.
s listenOn: 1234 backlogSize: 10.
s port.
s2 := Socket newTCP.
s2 listenOn: 1234 backlogSize: 10.
{ s port. s2 port } "==> #(1234 50542)"


Levente

>
> frank
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Frank Shearar-3
On 5 April 2013 04:16, Levente Uzonyi <[hidden email]> wrote:

> On Thu, 4 Apr 2013, Frank Shearar wrote:
>
>> So this isn't exactly a common use case, except on build.squeak.org:
>>
>> You have a bunch of network tests, so in your #setUp you start up a
>> server on some port and in #tearDown you shut down the server. What
>> port do you use? Well, it doesn't really matter too much, as long as
>> the port's available. So you choose some probably-not-used port, like
>> 7799. (WebClient's test suite uses this.) All is almost always well,
>> because tests only fail when something's using 7799, which is almost
>> never.
>>
>> build.squeak.org has two executors, which means it can run two builds
>> concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
>> ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
>> Which means you will often execute the same test suite in different
>> images at the same time. Suddenly those tests using a hard-coded port
>> all fail, causing noise. (Xtreams' test suite is also vulnerable to
>> this problem: look at the  XTSocketReadingWritingTest failures in
>> http://build.squeak.org/job/ExternalPackages/43/testReport/)
>>
>> So what do we do?
>>
>> One approach that I can think of is, like WebClient's test suite, to
>> use a method #port returning the port to use in a test. However, #port
>> should not return a constant but rather attempt to open a port within
>> some range, and return _some_ open port. The problem here is a
>> time-of-check-time-of-use race.
>>
>> Now that might nearly always work. Can anyone think of a better method?
>
>
> The current socket implementation does exactly what you need. If a port is
> being used, the socket will listen on a random free port. So the solution is
> to ask the created Socket instance for which port it's listening on.
>
> Code showing the behavior:
>
> s := Socket newTCP.
> s listenOn: 1234 backlogSize: 10.
> s port.
> s2 := Socket newTCP.
> s2 listenOn: 1234 backlogSize: 10.
> { s port. s2 port } "==> #(1234 50542)"

That's.... surprising. Is that something the SocketPlugin does? No EADDRINUSE?

OK, so leaving aside that I think that's crazy behaviour, that means
Xtreams could be easily fixed. In its #setUp there's a block

[output := Socket newTCP.
    output connectTo: #[127 0 0 1] port: 9999 waitForConnectionFor: 2.
    sync signal] fork.

which should become

[output := Socket newTCP.
    output connectTo: #[127 0 0 1] port: listene port waitForConnectionFor: 2.
    sync signal] fork.

frank

> Levente
>
>>
>> frank
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Frank Shearar-3
On 5 April 2013 10:04, Frank Shearar <[hidden email]> wrote:

> On 5 April 2013 04:16, Levente Uzonyi <[hidden email]> wrote:
>> On Thu, 4 Apr 2013, Frank Shearar wrote:
>>
>>> So this isn't exactly a common use case, except on build.squeak.org:
>>>
>>> You have a bunch of network tests, so in your #setUp you start up a
>>> server on some port and in #tearDown you shut down the server. What
>>> port do you use? Well, it doesn't really matter too much, as long as
>>> the port's available. So you choose some probably-not-used port, like
>>> 7799. (WebClient's test suite uses this.) All is almost always well,
>>> because tests only fail when something's using 7799, which is almost
>>> never.
>>>
>>> build.squeak.org has two executors, which means it can run two builds
>>> concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
>>> ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
>>> Which means you will often execute the same test suite in different
>>> images at the same time. Suddenly those tests using a hard-coded port
>>> all fail, causing noise. (Xtreams' test suite is also vulnerable to
>>> this problem: look at the  XTSocketReadingWritingTest failures in
>>> http://build.squeak.org/job/ExternalPackages/43/testReport/)
>>>
>>> So what do we do?
>>>
>>> One approach that I can think of is, like WebClient's test suite, to
>>> use a method #port returning the port to use in a test. However, #port
>>> should not return a constant but rather attempt to open a port within
>>> some range, and return _some_ open port. The problem here is a
>>> time-of-check-time-of-use race.
>>>
>>> Now that might nearly always work. Can anyone think of a better method?
>>
>>
>> The current socket implementation does exactly what you need. If a port is
>> being used, the socket will listen on a random free port. So the solution is
>> to ask the created Socket instance for which port it's listening on.
>>
>> Code showing the behavior:
>>
>> s := Socket newTCP.
>> s listenOn: 1234 backlogSize: 10.
>> s port.
>> s2 := Socket newTCP.
>> s2 listenOn: 1234 backlogSize: 10.
>> { s port. s2 port } "==> #(1234 50542)"
>
> That's.... surprising. Is that something the SocketPlugin does? No EADDRINUSE?
>
> OK, so leaving aside that I think that's crazy behaviour,

Upon reflection, it's not as crazy as I thought. listen() then
actually means "try open this specific port. If you can't, open on
some random port, and let me know I only got a listening socket on
some random port, not the particular port I asked for, by returning
EADDRINUSE."

That's handy for protocols that dynamically instantiate servers (FTP, say).

That _also_ means I've just been on a wild goose chase, thinking that
concurrent builds on separate executors were interfering. Only those
tests that expected a certain port number (and in WebClient that's
just one test) would fail if another job had opened that port.

frank

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Chris Muller-3
Of course, the other half are the connecting clients.  Not sure how
your code works but in my network tests I've been bitten by ports in
use because the clients didn't know about the random port having been
selected.

On Fri, Apr 5, 2013 at 6:54 AM, Frank Shearar <[hidden email]> wrote:

> On 5 April 2013 10:04, Frank Shearar <[hidden email]> wrote:
>> On 5 April 2013 04:16, Levente Uzonyi <[hidden email]> wrote:
>>> On Thu, 4 Apr 2013, Frank Shearar wrote:
>>>
>>>> So this isn't exactly a common use case, except on build.squeak.org:
>>>>
>>>> You have a bunch of network tests, so in your #setUp you start up a
>>>> server on some port and in #tearDown you shut down the server. What
>>>> port do you use? Well, it doesn't really matter too much, as long as
>>>> the port's available. So you choose some probably-not-used port, like
>>>> 7799. (WebClient's test suite uses this.) All is almost always well,
>>>> because tests only fail when something's using 7799, which is almost
>>>> never.
>>>>
>>>> build.squeak.org has two executors, which means it can run two builds
>>>> concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
>>>> ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
>>>> Which means you will often execute the same test suite in different
>>>> images at the same time. Suddenly those tests using a hard-coded port
>>>> all fail, causing noise. (Xtreams' test suite is also vulnerable to
>>>> this problem: look at the  XTSocketReadingWritingTest failures in
>>>> http://build.squeak.org/job/ExternalPackages/43/testReport/)
>>>>
>>>> So what do we do?
>>>>
>>>> One approach that I can think of is, like WebClient's test suite, to
>>>> use a method #port returning the port to use in a test. However, #port
>>>> should not return a constant but rather attempt to open a port within
>>>> some range, and return _some_ open port. The problem here is a
>>>> time-of-check-time-of-use race.
>>>>
>>>> Now that might nearly always work. Can anyone think of a better method?
>>>
>>>
>>> The current socket implementation does exactly what you need. If a port is
>>> being used, the socket will listen on a random free port. So the solution is
>>> to ask the created Socket instance for which port it's listening on.
>>>
>>> Code showing the behavior:
>>>
>>> s := Socket newTCP.
>>> s listenOn: 1234 backlogSize: 10.
>>> s port.
>>> s2 := Socket newTCP.
>>> s2 listenOn: 1234 backlogSize: 10.
>>> { s port. s2 port } "==> #(1234 50542)"
>>
>> That's.... surprising. Is that something the SocketPlugin does? No EADDRINUSE?
>>
>> OK, so leaving aside that I think that's crazy behaviour,
>
> Upon reflection, it's not as crazy as I thought. listen() then
> actually means "try open this specific port. If you can't, open on
> some random port, and let me know I only got a listening socket on
> some random port, not the particular port I asked for, by returning
> EADDRINUSE."
>
> That's handy for protocols that dynamically instantiate servers (FTP, say).
>
> That _also_ means I've just been on a wild goose chase, thinking that
> concurrent builds on separate executors were interfering. Only those
> tests that expected a certain port number (and in WebClient that's
> just one test) would fail if another job had opened that port.
>
> frank
>

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Frank Shearar-3
On 5 April 2013 17:09, Chris Muller <[hidden email]> wrote:
> Of course, the other half are the connecting clients.  Not sure how
> your code works but in my network tests I've been bitten by ports in
> use because the clients didn't know about the random port having been
> selected.

Yes, but that's why Levente suggests asking the server for its port,
instead of relying on a common magic number.

frank

> On Fri, Apr 5, 2013 at 6:54 AM, Frank Shearar <[hidden email]> wrote:
>> On 5 April 2013 10:04, Frank Shearar <[hidden email]> wrote:
>>> On 5 April 2013 04:16, Levente Uzonyi <[hidden email]> wrote:
>>>> On Thu, 4 Apr 2013, Frank Shearar wrote:
>>>>
>>>>> So this isn't exactly a common use case, except on build.squeak.org:
>>>>>
>>>>> You have a bunch of network tests, so in your #setUp you start up a
>>>>> server on some port and in #tearDown you shut down the server. What
>>>>> port do you use? Well, it doesn't really matter too much, as long as
>>>>> the port's available. So you choose some probably-not-used port, like
>>>>> 7799. (WebClient's test suite uses this.) All is almost always well,
>>>>> because tests only fail when something's using 7799, which is almost
>>>>> never.
>>>>>
>>>>> build.squeak.org has two executors, which means it can run two builds
>>>>> concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
>>>>> ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
>>>>> Which means you will often execute the same test suite in different
>>>>> images at the same time. Suddenly those tests using a hard-coded port
>>>>> all fail, causing noise. (Xtreams' test suite is also vulnerable to
>>>>> this problem: look at the  XTSocketReadingWritingTest failures in
>>>>> http://build.squeak.org/job/ExternalPackages/43/testReport/)
>>>>>
>>>>> So what do we do?
>>>>>
>>>>> One approach that I can think of is, like WebClient's test suite, to
>>>>> use a method #port returning the port to use in a test. However, #port
>>>>> should not return a constant but rather attempt to open a port within
>>>>> some range, and return _some_ open port. The problem here is a
>>>>> time-of-check-time-of-use race.
>>>>>
>>>>> Now that might nearly always work. Can anyone think of a better method?
>>>>
>>>>
>>>> The current socket implementation does exactly what you need. If a port is
>>>> being used, the socket will listen on a random free port. So the solution is
>>>> to ask the created Socket instance for which port it's listening on.
>>>>
>>>> Code showing the behavior:
>>>>
>>>> s := Socket newTCP.
>>>> s listenOn: 1234 backlogSize: 10.
>>>> s port.
>>>> s2 := Socket newTCP.
>>>> s2 listenOn: 1234 backlogSize: 10.
>>>> { s port. s2 port } "==> #(1234 50542)"
>>>
>>> That's.... surprising. Is that something the SocketPlugin does? No EADDRINUSE?
>>>
>>> OK, so leaving aside that I think that's crazy behaviour,
>>
>> Upon reflection, it's not as crazy as I thought. listen() then
>> actually means "try open this specific port. If you can't, open on
>> some random port, and let me know I only got a listening socket on
>> some random port, not the particular port I asked for, by returning
>> EADDRINUSE."
>>
>> That's handy for protocols that dynamically instantiate servers (FTP, say).
>>
>> That _also_ means I've just been on a wild goose chase, thinking that
>> concurrent builds on separate executors were interfering. Only those
>> tests that expected a certain port number (and in WebClient that's
>> just one test) would fail if another job had opened that port.
>>
>> frank
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Port numbers in network tests

Chris Muller-4
(Oops, I did mean for my note to go to the list).

For unit-test the "assume" approach has been more than practical
enough for me over the years.  For the random port numbers I chose,
the only time it ever didn't work is when I had forgotten to shutdown
the server image from a previously-failed test.

But wait!  Now I remember something!  There was some sort of "port
discovery" package available  for Squeak in years past -- can't
remember the name of it does anyone else?

On Fri, Apr 5, 2013 at 11:48 AM, Frank Shearar <[hidden email]> wrote:

> (Not sure if you meant to drop off-list, but OK.)
>
> Sure. But that's well beyond my implicit assumption of a unit test.
> You then still have a problem, with the cross-image test: either you
> assume that your listenOn: really did start listening on that port, or
> you check the port and communicate the actual real port in a side
> channel (for instance by passing in the port during the client image
> spawn directly, or via an environment variable. But I'd probably just
> fail the test, in your situation: the server starts up, and #assert:s
> that the proper port's being used.
>
> frank
>
> On 5 April 2013 17:36, Chris Muller <[hidden email]> wrote:
>> That means server and client must run in the same image, yes?  My
>> network tests actually run in separate images (launched by OSProcess),
>> so if a port number was taken and the server got assigned a random
>> port (instead of its hard coded port), the client in a separate image
>> has no way to know about that.
>>
>> On Fri, Apr 5, 2013 at 11:25 AM, Frank Shearar <[hidden email]> wrote:
>>> On 5 April 2013 17:09, Chris Muller <[hidden email]> wrote:
>>>> Of course, the other half are the connecting clients.  Not sure how
>>>> your code works but in my network tests I've been bitten by ports in
>>>> use because the clients didn't know about the random port having been
>>>> selected.
>>>
>>> Yes, but that's why Levente suggests asking the server for its port,
>>> instead of relying on a common magic number.
>>>
>>> frank
>>>
>>>> On Fri, Apr 5, 2013 at 6:54 AM, Frank Shearar <[hidden email]> wrote:
>>>>> On 5 April 2013 10:04, Frank Shearar <[hidden email]> wrote:
>>>>>> On 5 April 2013 04:16, Levente Uzonyi <[hidden email]> wrote:
>>>>>>> On Thu, 4 Apr 2013, Frank Shearar wrote:
>>>>>>>
>>>>>>>> So this isn't exactly a common use case, except on build.squeak.org:
>>>>>>>>
>>>>>>>> You have a bunch of network tests, so in your #setUp you start up a
>>>>>>>> server on some port and in #tearDown you shut down the server. What
>>>>>>>> port do you use? Well, it doesn't really matter too much, as long as
>>>>>>>> the port's available. So you choose some probably-not-used port, like
>>>>>>>> 7799. (WebClient's test suite uses this.) All is almost always well,
>>>>>>>> because tests only fail when something's using 7799, which is almost
>>>>>>>> never.
>>>>>>>>
>>>>>>>> build.squeak.org has two executors, which means it can run two builds
>>>>>>>> concurrently. Suddenly, if you build SqueakTrunk, you'll trigger
>>>>>>>> ExternalPackages, ExternalPackages-Squeak4.3 and External-Squeak4.4.
>>>>>>>> Which means you will often execute the same test suite in different
>>>>>>>> images at the same time. Suddenly those tests using a hard-coded port
>>>>>>>> all fail, causing noise. (Xtreams' test suite is also vulnerable to
>>>>>>>> this problem: look at the  XTSocketReadingWritingTest failures in
>>>>>>>> http://build.squeak.org/job/ExternalPackages/43/testReport/)
>>>>>>>>
>>>>>>>> So what do we do?
>>>>>>>>
>>>>>>>> One approach that I can think of is, like WebClient's test suite, to
>>>>>>>> use a method #port returning the port to use in a test. However, #port
>>>>>>>> should not return a constant but rather attempt to open a port within
>>>>>>>> some range, and return _some_ open port. The problem here is a
>>>>>>>> time-of-check-time-of-use race.
>>>>>>>>
>>>>>>>> Now that might nearly always work. Can anyone think of a better method?
>>>>>>>
>>>>>>>
>>>>>>> The current socket implementation does exactly what you need. If a port is
>>>>>>> being used, the socket will listen on a random free port. So the solution is
>>>>>>> to ask the created Socket instance for which port it's listening on.
>>>>>>>
>>>>>>> Code showing the behavior:
>>>>>>>
>>>>>>> s := Socket newTCP.
>>>>>>> s listenOn: 1234 backlogSize: 10.
>>>>>>> s port.
>>>>>>> s2 := Socket newTCP.
>>>>>>> s2 listenOn: 1234 backlogSize: 10.
>>>>>>> { s port. s2 port } "==> #(1234 50542)"
>>>>>>
>>>>>> That's.... surprising. Is that something the SocketPlugin does? No EADDRINUSE?
>>>>>>
>>>>>> OK, so leaving aside that I think that's crazy behaviour,
>>>>>
>>>>> Upon reflection, it's not as crazy as I thought. listen() then
>>>>> actually means "try open this specific port. If you can't, open on
>>>>> some random port, and let me know I only got a listening socket on
>>>>> some random port, not the particular port I asked for, by returning
>>>>> EADDRINUSE."
>>>>>
>>>>> That's handy for protocols that dynamically instantiate servers (FTP, say).
>>>>>
>>>>> That _also_ means I've just been on a wild goose chase, thinking that
>>>>> concurrent builds on separate executors were interfering. Only those
>>>>> tests that expected a certain port number (and in WebClient that's
>>>>> just one test) would fail if another job had opened that port.
>>>>>
>>>>> frank
>>>>>
>>>>

Reply | Threaded
Open this post in threaded view
|

re: Port numbers in network tests

ccrraaiigg

Hi Chris--

> But wait!  Now I remember something!  There was some sort of "port
> discovery" package available  for Squeak in years past -- can't
> remember the name of it does anyone else?

     Was it part of the NAT-traversal stuff Cees did?


-C

--
Craig Latta
www.netjam.org/resume
+1 510 396 5727
(Skype rings this until 16 April 2013)


Reply | Threaded
Open this post in threaded view
|

re: Port numbers in network tests

Karl Ramberg
http://swikis.ddo.jp:9091/Discovery.html

Does not work with IPv6

Karl



On Sun, Apr 7, 2013 at 3:59 AM, Craig Latta <[hidden email]> wrote:

Hi Chris--

> But wait!  Now I remember something!  There was some sort of "port
> discovery" package available  for Squeak in years past -- can't
> remember the name of it does anyone else?

     Was it part of the NAT-traversal stuff Cees did?


-C

--
Craig Latta
www.netjam.org/resume
<a href="tel:%2B1%20510%20396%205727" value="+15103965727">+1 510 396 5727
(Skype rings this until 16 April 2013)





Reply | Threaded
Open this post in threaded view
|

re: Port numbers in network tests

Chris Muller-3
In reply to this post by ccrraaiigg
Hm, not sure.  I looked around in my mail archives but only found.

    http://wiki.squeak.org/squeak/5629

That might be what I was thinking of, not sure.

On Sat, Apr 6, 2013 at 8:59 PM, Craig Latta <[hidden email]> wrote:

>
> Hi Chris--
>
>> But wait!  Now I remember something!  There was some sort of "port
>> discovery" package available  for Squeak in years past -- can't
>> remember the name of it does anyone else?
>
>      Was it part of the NAT-traversal stuff Cees did?
>
>
> -C
>
> --
> Craig Latta
> www.netjam.org/resume
> +1 510 396 5727
> (Skype rings this until 16 April 2013)
>
>