Wrong behavior of receiveDataInto:fromHost:port:

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Wrong behavior of receiveDataInto:fromHost:port:

HilaireFernandes
Hello,

I would like to discuss one point regarding this method. In my opinion its
parameter hostAddress is expected to be a SocketAdress, but the method
interprets it as a ByteArray. So in the code bellow, the returned result is
0, although the host responds a datagram:

server := NetNameResolver addressForName: 'localhost'.
socket := Socket newUDP.
socket sendUDPData: packet toHost: server port: 30000.
socket waitForData.
packet := ByteArray new: 200.
result := socket receiveDataInto: packet fromHost: server port: 30000.

The method receiveUDPData:toHost:port: should be amended as follow to make
it symmetric with sendUDPData... :

        [ | datagram |
                datagram := self receiveUDPDataInto: aStringOrByteArray.
                ((datagram at: 2) *asSocketAddress* = hostAddress and: [ (datagram at: 3)
= portNumber ])
                        ifTrue: [ ^ datagram at: 1 ]
                        ifFalse: [ ^0 ] ] repeat

What do you think?



-----
http://drgeo.eu
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

Sven Van Caekenberghe-2
Hi Hilaire,

Given the following comment:

Socket>>#primSocket: socketID receiveUDPDataInto: aStringOrByteArray startingAt: startIndex count: count
        "Receive data from the given socket into the given array starting at the given index.
        Return an Array containing the amount read, the host address byte array, the host port, and the more flag"

        <primitive: 'primitiveSocketReceiveUDPDataBufCount' module: 'SocketPlugin'>
        self primitiveFailed

I would says that the primitive result uses a ByteArray to represent the (other party's) host address in its second slot.

I find the method Socket>>#receiveDataInto:fromHost:port: quite weird: why would you restrict from whom you receive datagrams, especially since non-matching datagrams are simply thrown away.

But I do agree that SocketAddress is a weird class, maybe its #species should be set to ByteArray so that #= would work between them, but I am not sure.

Sven

> On 19 Jan 2020, at 20:24, HilaireFernandes <[hidden email]> wrote:
>
> Hello,
>
> I would like to discuss one point regarding this method. In my opinion its
> parameter hostAddress is expected to be a SocketAdress, but the method
> interprets it as a ByteArray. So in the code bellow, the returned result is
> 0, although the host responds a datagram:
>
> server := NetNameResolver addressForName: 'localhost'.
> socket := Socket newUDP.
> socket sendUDPData: packet toHost: server port: 30000.
> socket waitForData.
> packet := ByteArray new: 200.
> result := socket receiveDataInto: packet fromHost: server port: 30000.
>
> The method receiveUDPData:toHost:port: should be amended as follow to make
> it symmetric with sendUDPData... :
>
> [ | datagram |
> datagram := self receiveUDPDataInto: aStringOrByteArray.
> ((datagram at: 2) *asSocketAddress* = hostAddress and: [ (datagram at: 3)
> = portNumber ])
> ifTrue: [ ^ datagram at: 1 ]
> ifFalse: [ ^0 ] ] repeat
>
> What do you think?
>
>
>
> -----
> http://drgeo.eu
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>


Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

HilaireFernandes
IMHO, this method is usefull when you are connected to a networked service
(IP and PORT). It helps to filter the datagrams you are interested by.

Turning SocketAdress as ByteArray could help without the need to convert
each IP ByteArray instance, as I proposed. Not sure about the implication,
though.

Hilaire



-----
http://drgeo.eu
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

Sven Van Caekenberghe-2
I am confused, you talk about three different methods,

receiveDataInto:fromHost:port:
receiveUDPData:toHost:port:
sendUDPData... (unspecified)

which one do you want to change and how ?

In any case, as answered by the primitives, the address seems to be a byte array.

So for filtering incoming packets, I would not convert each packet's address, but the target address.

> On 20 Jan 2020, at 10:10, HilaireFernandes <[hidden email]> wrote:
>
> IMHO, this method is usefull when you are connected to a networked service
> (IP and PORT). It helps to filter the datagrams you are interested by.

How would you have a setup where you need to do that ?

Are you building a firewall ?

Can you elaborate on what you are trying to do ?

You do not have to answer, but I am interested in learning where this comes from.

> Turning SocketAdress as ByteArray could help without the need to convert
> each IP ByteArray instance, as I proposed. Not sure about the implication,
> though.
>
> Hilaire
>
>
>
> -----
> http://drgeo.eu
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>


Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

HilaireFernandes
It is indeed about *receiveDataInto:fromHost:port:* I am talking about.

Its *fromHost* parameter is not of the same nature as the *toHost* parameter
in the message *sendUDPData:toHost:port:  *
In the former it is expected a ByteArray to work, in the later a
SocketAdress.

*About the context.
*This school year I am doing basic Pharo programming with Dr.Geo (12 and 13
years old). I wrote a school book[1] for that. But It is a bit tedious to
teach and not enough appealing. It may just be my fault or the format of the
book. For the next semester, I will explore alternative format, in more
independent teaching unit.

For the longer term, I want to explore alternative way of teaching Pharo,
more appealing. Something more in the kids interest area.

Do you know Minetest, it is a free implementation of Minecraft? It comes
with a server software and client instances communicating with the server
with a dedicated UDP protocol. In the past, we use it in an architecture
project in connexion with math.

Written with Pharo, I envision a Minetest client using the UPD protocol to
interact with a server. This Pharo written client will be used as a
programming environment to control the kid avatar in the Minetest 3D world,
like a BotsInc of 3D. The Pharo client will not render the view, it will be
done by the usual Minetest client.

For example, a kid could code the behaviour of his avatar to build a tower
with a Minetest DSL:

    ../..
    100 timesRepeat: [
        avatar goUp
        4 timesRepeat: [
            10 timesRepeat: [avatar dropBrick; moveForward].
            avatar turnLeft] ]


There are some extensions in Minetest client (Mod in Minetest terminology)
to do some basic programming but it is not as interesting as could be Pharo
programming with a dedicated DSL[3].

I speculate it could be pretty cool and it should meet a lot of success
among kids. Plus provide good teaching interest and more Pharo awareness.

Hilaire

[1]
https://launchpad.net/drgeo/trunk/19.09/+download/programmer-avec-drgeo.pdf
[2] http://www.minetest.net
[3]
https://forum.minetest.net/viewtopic.php?f=14&p=365620&sid=5d825763b9cb5579c5e195c0bf4e3b28#p365113



-----
http://drgeo.eu
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

Sven Van Caekenberghe-2


> On 20 Jan 2020, at 17:39, HilaireFernandes <[hidden email]> wrote:
>
> It is indeed about *receiveDataInto:fromHost:port:* I am talking about.
>
> Its *fromHost* parameter is not of the same nature as the *toHost* parameter
> in the message *sendUDPData:toHost:port:  *
> In the former it is expected a ByteArray to work, in the later a
> SocketAdress.

I don't see any indication in the code to prove your statement

"Socket>>#sendUDPData:toHost:port: is assuming hostAddress to be a SocketAddress".

Like I said, all primitives take host addresses to be a byte array (of which SocketAddress is a subclass).

> *About the context.
> *This school year I am doing basic Pharo programming with Dr.Geo (12 and 13
> years old). I wrote a school book[1] for that. But It is a bit tedious to
> teach and not enough appealing. It may just be my fault or the format of the
> book. For the next semester, I will explore alternative format, in more
> independent teaching unit.
>
> For the longer term, I want to explore alternative way of teaching Pharo,
> more appealing. Something more in the kids interest area.
>
> Do you know Minetest, it is a free implementation of Minecraft? It comes
> with a server software and client instances communicating with the server
> with a dedicated UDP protocol. In the past, we use it in an architecture
> project in connexion with math.
>
> Written with Pharo, I envision a Minetest client using the UPD protocol to
> interact with a server. This Pharo written client will be used as a
> programming environment to control the kid avatar in the Minetest 3D world,
> like a BotsInc of 3D. The Pharo client will not render the view, it will be
> done by the usual Minetest client.
>
> For example, a kid could code the behaviour of his avatar to build a tower
> with a Minetest DSL:
>
>    ../..
>    100 timesRepeat: [
>        avatar goUp
>        4 timesRepeat: [
>            10 timesRepeat: [avatar dropBrick; moveForward].
>            avatar turnLeft] ]
>
>
> There are some extensions in Minetest client (Mod in Minetest terminology)
> to do some basic programming but it is not as interesting as could be Pharo
> programming with a dedicated DSL[3].
>
> I speculate it could be pretty cool and it should meet a lot of success
> among kids. Plus provide good teaching interest and more Pharo awareness.
>
> Hilaire
>
> [1]
> https://launchpad.net/drgeo/trunk/19.09/+download/programmer-avec-drgeo.pdf
> [2] http://www.minetest.net
> [3]
> https://forum.minetest.net/viewtopic.php?f=14&p=365620&sid=5d825763b9cb5579c5e195c0bf4e3b28#p365113

Sounds nice, and indeed a (more) fun way to teach programming.

But I still fail to see why you have to filter incoming UDP packets on their origin address.

I do agree that using ByteArray and SocketAddress interchanging fails in some cases, even though one is a subclass of the other, because #= fails, due to the class check.

I can't see an easy way to fix this (as #= is defined in SequenceableCollection). One solution might be to add #species ByteArray to SocketAddress, but it is hard to see the implications of that.

Another option might be to add:

SocketAddress>>#= anObject
  ^ self == anObject or: [ self hasEqualElements: anObject ]

This way, #= would work when the SocketAddress is the receiver (but not the other way around which is ugly).

If you then put hostAddress first in:

Socket>>#receiveDataInto: aStringOrByteArray fromHost: hostAddress port: portNumber
        "Receive a UDP packet from the given hostAddress/portNumber, storing the data in the given buffer, and return the number of bytes received. Note the given buffer may be only partially filled by the received data."

        [ | datagram |
                datagram := self receiveUDPDataInto: aStringOrByteArray.
                (hostAddress = (datagram at: 2) and: [ (datagram at: 3) = portNumber ])
                        ifTrue: [ ^ datagram at: 1 ]
                        ifFalse: [ ^0 ] ] repeat

it should work.

Sven


Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

HilaireFernandes
Hi Sven,


Sven Van Caekenberghe wrote
> Sounds nice, and indeed a (more) fun way to teach programming.
>
> But I still fail to see why you have to filter incoming UDP packets on
> their origin address.

If third party software on the workstation are connected to other network
services, don't you want to filter out? Or am I missing something?


Sven Van Caekenberghe wrote

> I do agree that using ByteArray and SocketAddress interchanging fails in
> some cases, even though one is a subclass of the other, because #= fails,
> due to the class check.
>
> I can't see an easy way to fix this (as #= is defined in
> SequenceableCollection). One solution might be to add #species ByteArray
> to SocketAddress, but it is hard to see the implications of that.
>
> Another option might be to add:
>
> SocketAddress>>#= anObject
>   ^ self == anObject or: [ self hasEqualElements: anObject ]
>
> This way, #= would work when the SocketAddress is the receiver (but not
> the other way around which is ugly).

Yes, not to be symmetric is a bit a problem, likely not a good addition to
Pharo.
I will see how it goes on my exploration of this topic.

Thanks
Hilaire





-----
http://drgeo.eu
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

Sven Van Caekenberghe-2


> On 26 Jan 2020, at 09:42, HilaireFernandes <[hidden email]> wrote:
>
> Sven Van Caekenberghe wrote
>> Sounds nice, and indeed a (more) fun way to teach programming.
>>
>> But I still fail to see why you have to filter incoming UDP packets on
>> their origin address.
>
> If third party software on the workstation are connected to other network
> services, don't you want to filter out? Or am I missing something?

I think so: you listen on an UDP socket on a specific port (and optionally bound to a specific network interface on your machine), then you get only datagrams directed to you - this happens at the OS level.

A (at host:port) sends a datagram to B (at host:port), to reply, B looks at the datagram to figure out the sender's info and replies.

You can study the example UDPSocketEchoTest to see how this works.

HTH,

Sven



Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

Danil Osipchuk-2
As soon as you start listening on udp port every host having connectivity will be able to send you a datagram which will hit a smalltalk handler. It is a concern both from performance and security pov. You may be willing to implement your  own protocol suit on top of udp to have a tcp like endpoint pairing and session managment, but without buffering/windowing and reliable transmission to meet realtime requirements of gaming etc. For that you will need this.
Probably it was used by multiuser collaboration project croquet

Regards 
Danil

вс, 26 янв. 2020 г., 12:08 Sven Van Caekenberghe <[hidden email]>:


> On 26 Jan 2020, at 09:42, HilaireFernandes <[hidden email]> wrote:
>
> Sven Van Caekenberghe wrote
>> Sounds nice, and indeed a (more) fun way to teach programming.
>>
>> But I still fail to see why you have to filter incoming UDP packets on
>> their origin address.
>
> If third party software on the workstation are connected to other network
> services, don't you want to filter out? Or am I missing something?

I think so: you listen on an UDP socket on a specific port (and optionally bound to a specific network interface on your machine), then you get only datagrams directed to you - this happens at the OS level.

A (at host:port) sends a datagram to B (at host:port), to reply, B looks at the datagram to figure out the sender's info and replies.

You can study the example UDPSocketEchoTest to see how this works.

HTH,

Sven



Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

Danil Osipchuk-2
A bit of archeology. I could not find message sends quickly in https://github.com/NikolaySuslov/croquet-squeak and in 5.2 squeak image.
Code by Johh MacIntosh of 2000 in squeak 5.2  is below, for the primitiveOnlySupportsOneSemaphore=true it relies on #setPeer:port which in turn does a linux syscall to connect(2)  restricting socket to send/receive only to/from a single peer.
in 2011 it was changed by 'ul' (I don't know whom the initials belong to) setPeer:port was dropped and this is how it stays since then both in squeak and pharo - purely image side logic to filter incoming datagrams.

= JMM =
receiveDataInto: aStringOrByteArray fromHost: hostAddress port: portNumber
| datagram |
"Receive a UDP packet from the given hostAddress/portNumber, storing the data in the given buffer, and return the number of bytes received. Note the given buffer may be only partially filled by the received data."

primitiveOnlySupportsOneSemaphore ifTrue:
[self setPeer: hostAddress port: portNumber.
^self receiveDataInto: aStringOrByteArray].
[true] whileTrue:
[datagram := self receiveUDPDataInto: aStringOrByteArray.
((datagram at: 2) = hostAddress and: [(datagram at: 3) = portNumber])
ifTrue: [^datagram at: 1]
ifFalse: [^0]]


= ul =
receiveDataInto: aStringOrByteArray fromHost: hostAddress port: portNumber
| datagram |
"Receive a UDP packet from the given hostAddress/portNumber, storing the data in the given buffer, and return the number of bytes received. Note the given buffer may be only partially filled by the received data."

[
datagram := self receiveUDPDataInto: aStringOrByteArray.
((datagram at: 2) = hostAddress and: [(datagram at: 3) = portNumber])
ifTrue: [^datagram at: 1]
ifFalse: [^0]] repeat

вс, 26 янв. 2020 г. в 13:14, Danil Osipchuk <[hidden email]>:
As soon as you start listening on udp port every host having connectivity will be able to send you a datagram which will hit a smalltalk handler. It is a concern both from performance and security pov. You may be willing to implement your  own protocol suit on top of udp to have a tcp like endpoint pairing and session managment, but without buffering/windowing and reliable transmission to meet realtime requirements of gaming etc. For that you will need this.
Probably it was used by multiuser collaboration project croquet

Regards 
Danil

вс, 26 янв. 2020 г., 12:08 Sven Van Caekenberghe <[hidden email]>:


> On 26 Jan 2020, at 09:42, HilaireFernandes <[hidden email]> wrote:
>
> Sven Van Caekenberghe wrote
>> Sounds nice, and indeed a (more) fun way to teach programming.
>>
>> But I still fail to see why you have to filter incoming UDP packets on
>> their origin address.
>
> If third party software on the workstation are connected to other network
> services, don't you want to filter out? Or am I missing something?

I think so: you listen on an UDP socket on a specific port (and optionally bound to a specific network interface on your machine), then you get only datagrams directed to you - this happens at the OS level.

A (at host:port) sends a datagram to B (at host:port), to reply, B looks at the datagram to figure out the sender's info and replies.

You can study the example UDPSocketEchoTest to see how this works.

HTH,

Sven



Reply | Threaded
Open this post in threaded view
|

Re: Wrong behavior of receiveDataInto:fromHost:port:

David T. Lewis
On Sun, Jan 26, 2020 at 04:04:38PM +0300, Danil Osipchuk wrote:

> in 2011 it was changed by 'ul' (I don't know whom the initials belong to)

'ul' is Levente Uzonyi (leves at caesar.elte.hu)