The Trunk: Network-dtl.240.mcz
Locked
 
|
David T. Lewis uploaded a new version of Network to project The Trunk:
http://source.squeak.org/trunk/Network-dtl.240.mcz ==================== Summary ==================== Name: Network-dtl.240 Author: dtl Time: 6 April 2020, 10:33:07.751386 pm UUID: a8f00e08-7398-4d4a-8cd2-09cedcada729 Ancestors: Network-eem.239 This update enables the Squeak new networking and IPv6 support by default. The update is in the package postscript. IPv6 network support for the VM (SocketPlugin) and image were implemented by Ian Piumarta circa 2007. This included IPv6 support and a more general image-side representation of network addressing. In 2012, this work was merged into Squeak trunk, with a preference to switch between the old networking support or the new networking support with IPv6. At that time, there were concerns with performance of NetNameResolver on some Windows platforms, and for this reason the "Enable IPv6 and new network support" preference was left disabled for official Squeak distributions. It now 2020, and operating systems have improved over the years, so it is time to enable the new networking preference by default. =============== Diff against Network-eem.239 =============== Item was changed: + (PackageInfo named: 'Network') postscript: '"Enable IPv6 and new network support by default" + NetNameResolver enableIPv6: true + '! - (PackageInfo named: 'Network') postscript: '"Rebuild CSNonAtom." - MailAddressTokenizer cleanUp: false'! |
Re: The Trunk: Network-dtl.240.mcz
|
Hi Dave! :-) Now we have to fix those tests that suddenly got nervous and fail or time out -- at least on my machine: YELLOW - FAIL WebClientServerTest>>#testBasicAuth WebClientServerTest>>#testBerarerAuth WebClientServerTest>>#testContentTypeCharsetEncoding WebClientServerTest>>#testDigestAuth WebClientServerTest>>#testHtmlSubmit WebClientServerTest>>#testHttpDelete WebClientServerTest>>#testInvalidCookies WebClientServerTest>>#testNestedAction RED - ERROR ArbitraryObjectSocketTestCase>>#testBasics SocketTest>>#testClientConnect SocketTest>>#testDataReceive SocketTest>>#testDataSending SocketTest>>#testLocalAddress SocketTest>>#testLocalPort SocketTest>>#testPeerName SocketTest>>#testReceiveTimeout SocketTest>>#testRemoteAddress SocketTest>>#testRemotePort SocketTest>>#testSendTimeout SocketTest>>#testServerAccept SocketTest>>#testStringFromAddress SocketTest>>#testUDP SqueakSSLTest>>#testGooglePopStream StringSocketTestCase>>#testBasics StringSocketTestCase>>#testBogusInput1 StringSocketTestCase>>#testBogusInput2 WebClientServerTest>>#testListenOnInterface ;o) Best, Marcel
|
Re: The Trunk: Network-dtl.240.mcz
|
|
Oh my, that's going to take some tidying up.
And I expect that in order to get full coverage, we will need to arrange for tests that switch the preference to cover both the old and the new network settings when we run the full test suite. I can't look at it right now but will do so soon-ish. Dave On Wed, Apr 22, 2020 at 03:56:37PM +0200, Marcel Taeumel wrote: > Hi Dave! :-) > > Now we have to fix those tests that suddenly got nervous and fail or time out -- at least on my machine: > > YELLOW - FAIL > > WebClientServerTest>>#testBasicAuth > WebClientServerTest>>#testBerarerAuth > WebClientServerTest>>#testContentTypeCharsetEncoding > WebClientServerTest>>#testDigestAuth > WebClientServerTest>>#testHtmlSubmit > WebClientServerTest>>#testHttpDelete > WebClientServerTest>>#testInvalidCookies > WebClientServerTest>>#testNestedAction > > RED - ERROR > > ArbitraryObjectSocketTestCase>>#testBasics > SocketTest>>#testClientConnect > SocketTest>>#testDataReceive > SocketTest>>#testDataSending > SocketTest>>#testLocalAddress > SocketTest>>#testLocalPort > SocketTest>>#testPeerName > SocketTest>>#testReceiveTimeout > SocketTest>>#testRemoteAddress > SocketTest>>#testRemotePort > SocketTest>>#testSendTimeout > SocketTest>>#testServerAccept > SocketTest>>#testStringFromAddress > SocketTest>>#testUDP > SqueakSSLTest>>#testGooglePopStream > StringSocketTestCase>>#testBasics > StringSocketTestCase>>#testBogusInput1 > StringSocketTestCase>>#testBogusInput2 > WebClientServerTest>>#testListenOnInterface > > ;o) > > Best, > Marcel > Am 07.04.2020 17:12:43 schrieb [hidden email] <[hidden email]>: > David T. Lewis uploaded a new version of Network to project The Trunk: > http://source.squeak.org/trunk/Network-dtl.240.mcz > > ==================== Summary ==================== > > Name: Network-dtl.240 > Author: dtl > Time: 6 April 2020, 10:33:07.751386 pm > UUID: a8f00e08-7398-4d4a-8cd2-09cedcada729 > Ancestors: Network-eem.239 > > This update enables the Squeak new networking and IPv6 support by default. The update is in the package postscript. > > IPv6 network support for the VM (SocketPlugin) and image were implemented by Ian Piumarta circa 2007. This included IPv6 support and a more general image-side representation of network addressing. In 2012, this work was merged into Squeak trunk, with a preference to switch between the old networking support or the new networking support with IPv6. At that time, there were concerns with performance of NetNameResolver on some Windows platforms, and for this reason the "Enable IPv6 and new network support" preference was left disabled for official Squeak distributions. > > It now 2020, and operating systems have improved over the years, so it is time to enable the new networking preference by default. > > =============== Diff against Network-eem.239 =============== > > Item was changed: > + (PackageInfo named: 'Network') postscript: '"Enable IPv6 and new network support by default" > + NetNameResolver enableIPv6: true > + '! > - (PackageInfo named: 'Network') postscript: '"Rebuild CSNonAtom." > - MailAddressTokenizer cleanUp: false'! > > > |
Re: The Trunk: Network-dtl.240.mcz
|
|
Hi Marcel,
Actually, I'm not seeing the problem on my system (Linux). The tests in NetworkTests amd the WebClient tests are green, regardless of whether the "Enable IPv6" preference is true or false. I do see intermittent error failures in WebClientServerTest. Usually all tests pass, but I have seen failures in #testStreaming and #testPersistentAutRedirectSession so far. It is very intermittent, but I know that the new resolver primitives introduce some new possibilities for concurrency issue, so there may be some problems in that regard. But the failures you see are presumably consistent, and are much different from what I am seeing on my system. I have to suspect platform differences at this point. I think that you are using OS X (is that right?) and I expect that the network resolver may behave quite differently on different platforms, so we'll need to collect a few more data points to see where the issues are. Can others please try running WebClientServerTest both with "Enable IPv6" true and with it false, and report back on whether the tests pass on your Unix/Linux/Windows/OS X system? Thanks, Dave On Wed, Apr 22, 2020 at 11:31:30AM -0400, David T. Lewis wrote: > Oh my, that's going to take some tidying up. > > And I expect that in order to get full coverage, we will need to arrange for > tests that switch the preference to cover both the old and the new network > settings when we run the full test suite. > > I can't look at it right now but will do so soon-ish. > > Dave > > On Wed, Apr 22, 2020 at 03:56:37PM +0200, Marcel Taeumel wrote: > > Hi Dave! :-) > > > > Now we have to fix those tests that suddenly got nervous and fail or time out -- at least on my machine: > > > > YELLOW - FAIL > > > > WebClientServerTest>>#testBasicAuth > > WebClientServerTest>>#testBerarerAuth > > WebClientServerTest>>#testContentTypeCharsetEncoding > > WebClientServerTest>>#testDigestAuth > > WebClientServerTest>>#testHtmlSubmit > > WebClientServerTest>>#testHttpDelete > > WebClientServerTest>>#testInvalidCookies > > WebClientServerTest>>#testNestedAction > > > > RED - ERROR > > > > ArbitraryObjectSocketTestCase>>#testBasics > > SocketTest>>#testClientConnect > > SocketTest>>#testDataReceive > > SocketTest>>#testDataSending > > SocketTest>>#testLocalAddress > > SocketTest>>#testLocalPort > > SocketTest>>#testPeerName > > SocketTest>>#testReceiveTimeout > > SocketTest>>#testRemoteAddress > > SocketTest>>#testRemotePort > > SocketTest>>#testSendTimeout > > SocketTest>>#testServerAccept > > SocketTest>>#testStringFromAddress > > SocketTest>>#testUDP > > SqueakSSLTest>>#testGooglePopStream > > StringSocketTestCase>>#testBasics > > StringSocketTestCase>>#testBogusInput1 > > StringSocketTestCase>>#testBogusInput2 > > WebClientServerTest>>#testListenOnInterface > > > > ;o) > > > > Best, > > Marcel > > Am 07.04.2020 17:12:43 schrieb [hidden email] <[hidden email]>: > > David T. Lewis uploaded a new version of Network to project The Trunk: > > http://source.squeak.org/trunk/Network-dtl.240.mcz > > > > ==================== Summary ==================== > > > > Name: Network-dtl.240 > > Author: dtl > > Time: 6 April 2020, 10:33:07.751386 pm > > UUID: a8f00e08-7398-4d4a-8cd2-09cedcada729 > > Ancestors: Network-eem.239 > > > > This update enables the Squeak new networking and IPv6 support by default. The update is in the package postscript. > > > > IPv6 network support for the VM (SocketPlugin) and image were implemented by Ian Piumarta circa 2007. This included IPv6 support and a more general image-side representation of network addressing. In 2012, this work was merged into Squeak trunk, with a preference to switch between the old networking support or the new networking support with IPv6. At that time, there were concerns with performance of NetNameResolver on some Windows platforms, and for this reason the "Enable IPv6 and new network support" preference was left disabled for official Squeak distributions. > > > > It now 2020, and operating systems have improved over the years, so it is time to enable the new networking preference by default. > > > > =============== Diff against Network-eem.239 =============== > > > > Item was changed: > > + (PackageInfo named: 'Network') postscript: '"Enable IPv6 and new network support by default" > > + NetNameResolver enableIPv6: true > > + '! > > - (PackageInfo named: 'Network') postscript: '"Rebuild CSNonAtom." > > - MailAddressTokenizer cleanUp: false'! > > > > > > > > > |
Re: The Trunk: Network-dtl.240.mcz
|
|
PI 4, Raspbian, vm 20200324, image 5.3#19335 all tests pass either way
> On 2020-04-22, at 6:36 PM, David T. Lewis <[hidden email]> wrote: > > > Can others please try running WebClientServerTest both with "Enable IPv6" > true and with it false, and report back on whether the tests pass on > your Unix/Linux/Windows/OS X system? tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Strange OpCodes: SFA: Seek Financial Assistance |
Re: The Trunk: Network-dtl.240.mcz
|
|
In reply to this post by David T. Lewis
The network resolvers are definitely different in Linux and MacOS, the one in MacOS is less forgiving, or maybe it’s less robust.
/—————————————————————/ For encrypted mail use [hidden email] - Free account at ProtonMail.com Web: https://objectnets.net and https://objectnets.org https://datascilv.com https://datascilv.org On Apr 22, 2020, at 18:36, David T. Lewis <[hidden email]> wrote:
|
Re: The Trunk: Network-dtl.240.mcz
|
|
Hi all! I am on Windows 10. This is my output for the usual calls. Maybe it helps: SocketAddressInformation forHost: 'localhost' service: '12345' flags: 0 addressFamily: 0 socketType: SocketAddressInformation socketTypeStream protocol: SocketAddressInformation protocolTCP. " an OrderedCollection(::1(MOBILUS-TAB-NEW),12345(12345)-inet6-stream-tcp 127.0.0.1(MOBILUS-TAB-NEW),12345(12345)-inet4-stream-tcp)" NetNameResolver addressForName: 'localhost'. " ::1(MOBILUS-TAB-NEW),0(0)" *** Socket >> #connectTo: times out. Basically "WebClient httpGet: 'http://localhost:12345/test/auth'" does not work. ConnectionRefused. Best, Marcel
|
Re: The Trunk: Network-dtl.240.mcz
|
|
On Thu, 23 Apr 2020, Marcel Taeumel wrote:
> Hi all! > I am on Windows 10. This is my output for the usual calls. Maybe it helps: > > SocketAddressInformation > forHost: 'localhost' > service: '12345' > flags: 0 > addressFamily: 0 > socketType: SocketAddressInformation socketTypeStream > protocol: SocketAddressInformation protocolTCP. > > " an OrderedCollection(::1(MOBILUS-TAB-NEW),12345(12345)-inet6-stream-tcp 127.0.0.1(MOBILUS-TAB-NEW),12345(12345)-inet4-stream-tcp)" > > NetNameResolver addressForName: 'localhost'. > > " ::1(MOBILUS-TAB-NEW),0(0)" > > *** > > Socket >> #connectTo: times out. Basically "WebClient httpGet: 'http://localhost:12345/test/auth'" does not work. ConnectionRefused. its undeterministic nature (it returns the first address no matter how many are there, and the order of the addresses is not defined), #addressForName: gives you the IPv6 address, so the client can't connect to the server. Another issue is that listening on IPv6 addresses is not possible (at least on my machine): IPv4 works: s := Socket newTCP. s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver addressForName: '127.0.0.1'). s "==> a Socket[waitingForConnection]" IPv6 does not work. It just fails silently: s := Socket newTCP. s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver addressForName: '::1'). s "==> a Socket[destroyed]" Levente > > Best, > Marcel > > Am 23.04.2020 09:42:00 schrieb John Pfersich via Squeak-dev <[hidden email]>: > > The network resolvers are definitely different in Linux and MacOS, the one in MacOS is less forgiving, or maybe it’s less robust. > > /—————————————————————/For encrypted mail use [hidden email] - Free account at ProtonMail.comWeb: https://objectnets.net and https://objectnets.org > https://datascilv.com https://datascilv.org > > > On Apr 22, 2020, at 18:36, David T. Lewis <[hidden email]> wrote: > > I have to suspect platform differences at this point. I think that you > are using OS X (is that right?) and I expect that the network resolver > may behave quite differently on different platforms, so we'll need to > collect a few more data points to see where the issues are. > > > |
Re: The Trunk: Network-dtl.240.mcz
|
|
Looking at "Socket newTCP", IPv6 is not even considered. One has to call "Socket newTCP: SocketAddressInformation addressFamilyINET6". Maybe out IPv6 preferences needs to be considered more often? Best, Marcel
|
Re: The Trunk: Network-dtl.240.mcz
|
|
In reply to this post by Levente Uzonyi
> On 23.04.2020, at 16:57, Levente Uzonyi <[hidden email]> wrote: > > On Thu, 23 Apr 2020, Marcel Taeumel wrote: > >> Hi all! >> I am on Windows 10. This is my output for the usual calls. Maybe it helps: >> SocketAddressInformation >> forHost: 'localhost' >> service: '12345' >> flags: 0 >> addressFamily: 0 >> socketType: SocketAddressInformation socketTypeStream >> protocol: SocketAddressInformation protocolTCP. >> " an OrderedCollection(::1(MOBILUS-TAB-NEW),12345(12345)-inet6-stream-tcp 127.0.0.1(MOBILUS-TAB-NEW),12345(12345)-inet4-stream-tcp)" >> NetNameResolver addressForName: 'localhost'. >> " ::1(MOBILUS-TAB-NEW),0(0)" >> *** >> Socket >> #connectTo: times out. Basically "WebClient httpGet: 'http://localhost:12345/test/auth'" does not work. ConnectionRefused. > > I suspect that the server only listens on the IPv4 address while, due to its undeterministic nature (it returns the first address no matter how many are there, and the order of the addresses is not defined), #addressForName: gives you the IPv6 address, so the client can't connect to the server. > > Another issue is that listening on IPv6 addresses is not possible (at least on my machine): > > IPv4 works: > > s := Socket newTCP. > s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver addressForName: '127.0.0.1'). > s "==> a Socket[waitingForConnection]" > > IPv6 does not work. It just fails silently: > > s := Socket newTCP. > s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver addressForName: '::1'). > s "==> a Socket[destroyed]" Shouldn't that rather be infos := SocketAddressInformation forHost: '::1' service: '60000' flags: SocketAddressInformation passiveFlag addressFamily: SocketAddressInformation addressFamilyINET6 socketType: SocketAddressInformation socketTypeStream protocol: SocketAddressInformation protocolTCP. listeningSocket := infos first listenWithBacklog: 5. as in NetNameResolver|testIPv6? To be frank, yes we need a few more convenience methods :) best regards -Tobias > > > Levente > >> Best, >> Marcel >> >> Am 23.04.2020 09:42:00 schrieb John Pfersich via Squeak-dev <[hidden email]>: >> >> The network resolvers are definitely different in Linux and MacOS, the one in MacOS is less forgiving, or maybe it’s less robust. >> >> /—————————————————————/For encrypted mail use [hidden email] - Free account at ProtonMail.comWeb: https://objectnets.net and https://objectnets.org >> https://datascilv.com https://datascilv.org >> >> On Apr 22, 2020, at 18:36, David T. Lewis <[hidden email]> wrote: >> >> I have to suspect platform differences at this point. I think that you >> are using OS X (is that right?) and I expect that the network resolver >> may behave quite differently on different platforms, so we'll need to >> collect a few more data points to see where the issues are. |
Re: The Trunk: Network-dtl.240.mcz
|
|
In reply to this post by marcel.taeumel
On Thu, 23 Apr 2020, Marcel Taeumel wrote:
> Looking at "Socket newTCP", IPv6 is not even considered. One has to call "Socket newTCP: SocketAddressInformation addressFamilyINET6". Maybe out IPv6 preferences needs to be considered more often? Right, so #newTCP defines the family prematurely. In my opinion, it should happen when the interface (which is actually an address not an interface) is specified because the two address families can contradict, as the example showed. Let's do it the "right" way: s := Socket newTCP: SocketAddressInformation addressFamilyINET6. s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver addressForName: '::1'). s "==> a Socket[destroyed]". Still destroyed. Why? Perhaps because #listenOn:backlogSize:interface: tries to convert the address with #asByteArray and (NetNameResolver addressForName: '::1') asByteArray returns #[1] (what?). So, let's remove that #asByteArray send hoping that the VM can handle the new opaque addresses. s := Socket newTCP: SocketAddressInformation addressFamilyINET6. s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver addressForName: '::1'). s "==> a Socket[destroyed]". Still destroyed, so that didn't help. Why? Let's have a look at the plugin code (unix this time)[1]: void sqSocketListenOnPortBacklogSizeInterface(SocketPtr s, sqInt port, sqInt backlogSize, sqInt addr) Okay, addr must be a pointer to one of those opaque blobs. saddr.sin_port= htons((short)port); That short was meant to be unsigned short, right? saddr.sin_addr.s_addr= htonl(addr); The VM code expects addr to be a 4-byte unsigned integer (where's the cast?) representing an IPv4 address in host byte order. So, too bad, #listenOn:backlogSize:interface: (primitiveSocketListenOnPortBacklogInterface) does not support IPv6. It never did. Levente [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SocketPlugin/sqUnixSocket.c#L708 > Best, > Marcel > > Am 23.04.2020 16:57:44 schrieb Levente Uzonyi <[hidden email]>: > > On Thu, 23 Apr 2020, Marcel Taeumel wrote: > > > Hi all! > > I am on Windows 10. This is my output for the usual calls. Maybe it helps: > > > > SocketAddressInformation > > forHost: 'localhost' > > service: '12345' > > flags: 0 > > addressFamily: 0 > > socketType: SocketAddressInformation socketTypeStream > > protocol: SocketAddressInformation protocolTCP. > > > > " an OrderedCollection(::1(MOBILUS-TAB-NEW),12345(12345)-inet6-stream-tcp 127.0.0.1(MOBILUS-TAB-NEW),12345(12345)-inet4-stream-tcp)" > > > > NetNameResolver addressForName: 'localhost'. > > > > " ::1(MOBILUS-TAB-NEW),0(0)" > > > > *** > > > > Socket >> #connectTo: times out. Basically "WebClient httpGet: 'http://localhost:12345/test/auth'" does not work. ConnectionRefused. > > I suspect that the server only listens on the IPv4 address while, due to > its undeterministic nature (it returns the first address no matter how > many are there, and the order of the addresses is not defined), > #addressForName: gives you the IPv6 address, so the client can't > connect to the server. > > Another issue is that listening on IPv6 addresses is not possible (at > least on my machine): > > IPv4 works: > > s := Socket newTCP. > s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver > addressForName: '127.0.0.1'). > s "==> a Socket[waitingForConnection]" > > IPv6 does not work. It just fails silently: > > s := Socket newTCP. > s listenOn: 60000 backlogSize: 8 interface: (NetNameResolver > addressForName: '::1'). > s "==> a Socket[destroyed]" > > > Levente > > > > > Best, > > Marcel > > > > Am 23.04.2020 09:42:00 schrieb John Pfersich via Squeak-dev : > > > > The network resolvers are definitely different in Linux and MacOS, the one in MacOS is less forgiving, or maybe it’s less robust. > > > > /—————————————————————/For encrypted mail use [hidden email] - Free account at ProtonMail.comWeb: https://objectnets.net and https://objectnets.org > > https://datascilv.com https://datascilv.org > > > > > > On Apr 22, 2020, at 18:36, David T. Lewis wrote: > > > > I have to suspect platform differences at this point. I think that you > > are using OS X (is that right?) and I expect that the network resolver > > may behave quite differently on different platforms, so we'll need to > > collect a few more data points to see where the issues are. > > > > > > > > > |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |