Exchanging information between 2 pharo applications (2 images running on two different computers)
Locked
12
12
 
|
Hi all, I want to connect two applications (1 by image, each one on a different computer) so as as to exchange information (data) between them. So my question is about the best (smalltalk) practices to connect two app/image and exchange data. I imagine either with a direct connection through a network (TCP Socket, Web socket, pure HTTP with Zinc) and/or with a serial connection. At first, without any « security ». But later, information exchanges will be encrypted. I’ve seen some information on how to use SerialPort, or even FileStream. I could do it (or at least simulate it with HTTP). What are the other options ? Socket ? Do we have P2P libs (I couldn’t find) with eventually discovery features ? Any comment / suggestion / pointers are greatly welcome. TIA. Cédrick
|
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
|
Look at Seamless https://github.com/dionisiydk/Seamless. 2017-10-25 14:21 GMT+02:00 Cédrick Béler <[hidden email]>:
|
|
|
Thanks Denis. I will ! I knew I have seen a telephoto component that could help but forgot about it !
Do you know if it’s possible to handle offline mode ?
|
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
|
What is offline mode? 2017-10-25 15:10 GMT+02:00 Cédrick Béler <[hidden email]>:
|
|
|
I had a look and this is not (natively) possible.
The idea of the off-line mode would be to delay messages that are sent to the peer until a connection is established. I think I have to try to do it by myself (like having a list of information exchange that wait until a connexion is established). What I try to do is not as complex as two general image exchanging messages on objects (like on TelePharo). I just want a repository of information (mainly a collection of static information/data versions) on both peers to be synchronized when a connection is established.
|
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
|
In reply to this post by cedreek
Hi Cederik,
you should have a look at http://smalltalkhub.com/#!/~panuw/zeromq ZeroMQ is a networking library. Nice thing about it is that all the networking related workload is dealt with in a second process. That can save you some resources within Smalltalk. There is a lot of documentation around and it should be easy for you to dive into this. http://zguide.zeromq.org/page:all Cheers! Sebastian On 2017-10-25 05:21 AM, Cédrick Béler wrote:
|
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
|
In reply to this post by cedreek
Yes, it is not part of Seamless. 2017-10-25 15:40 GMT+02:00 Cédrick Béler <[hidden email]>:
|
|
|
Ok.
I wonder also if Basys is not enough to experiment. I’ll also try Sebastian suggestion. Cheers, Cédrick
|
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
|
Am 25.10.2017 um 15:52 schrieb Cédrick Béler:
> Ok. > > I wonder also if Basys is not enough to experiment. > https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/182/artifact/book-result/BasysNetwork/ > > I’ll also try Sebastian suggestion. > ZeroMQ is a wonderful library. We used it together with Gemstone/s to build an "Event data bus" based on ZeroMQ ... and with each successful commit our Gemstone/S send events on this "bus". On the "bus" several different programs written in different languages are listening and do some reaction based on the events they were interested in. Marten Feldtmann |
Binary Decision Diagram Package in Smalltalk
|
|
Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk?
Andrew |
Re: Binary Decision Diagram Package in Smalltalk
|
|
Hi andrew
I think that Avi did a package about BDD (but I thought it was special binary trees) so this is probably the same. Did you check on Squeaksource? http://www.squeaksource.com/BTree.html If this is what you are looking for I can help porting it to Pharo. Stef On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <[hidden email]> wrote: > Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk? > > Andrew > > |
Re: Binary Decision Diagram Package in Smalltalk
|
|
Phil presented an Xmpp integration at Pharo days - I understood that was a common comms mechanism too.
Tim Sent from my iPhone > On 25 Oct 2017, at 20:39, Stephane Ducasse <[hidden email]> wrote: > > Hi andrew > > I think that Avi did a package about BDD (but I thought it was special > binary trees) so this is probably the same. > Did you check on Squeaksource? > http://www.squeaksource.com/BTree.html > If this is what you are looking for I can help porting it to Pharo. > > Stef > > >> On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <[hidden email]> wrote: >> Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk? >> >> Andrew >> >> > |
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
In reply to this post by cedreek
Nothing complex about two images exchanging messages, its not even complex to transmit objects via fuel, you even transmit a debugger or any part of the live environment or even make an "internet" of images that join objects together. Sky is the limit. Pharo already provides you will all the tools/libraries to do this.
A streamsocket (not to be confused with regular sockets) will delay the messages (a collection of bytes) until they arrive to the receiver or until they have reached the timeout period. Offline mode is pretty much obligatory even for plain old internet web apps because of drops in connection or the plain fact a connection can become slow. I used streamsockets in my Pharo to Python bridge (Atlas) because the execution was not necessary synchronous , I was sending Python command to a 3d application from Pharo and I had to make sure that the bridge was working even in the case of a command that could take hours to execute like a rendering process. One cool trick I did was to send the Python errors back to Pharo and trigger them as Pharo's regular MessageNotUnderstood , this is a nice way to make sure that you can fix a wrong message after it has been executing without going to the image that is executing it. The limitation with sockets which what every frameworks uses because AFAIK they are the only means to communicate remotely is that they are not top performance so that mean that you can send long loops but executing communication inside long loops will either slow you down considerably or simply timeout your socket. Sockets can timeout from both ends if they feel that for some reason they lost communication with the other side and reached their timeout period. Timeout period is customization. This is a problem I did not tackle with my implementation because I dont think there can be an actual practical solution better than avoiding this scenario. Only shared memory seems to overcome this but it can be used only locally and not remotely (aka on same computer) . On Wed, Oct 25, 2017 at 4:41 PM Cédrick Béler <[hidden email]> wrote:
|
|
|
In reply to this post by Stephane Ducasse-3
On Thu, Oct 26, 2017 at 3:39 AM, Stephane Ducasse <[hidden email]> wrote: Hi andrew BTree is a database indexing implementation. Actually it would be a good feature to have available with Pharo. BDD is diagram[1] . Roassal would be the best bet. If it doesn't have an out of the box BDD, all the pieces are there make one. And the Roassal guys love new use cases. cheers -ben
|
Re: Binary Decision Diagram Package in Smalltalk
|
|
In reply to this post by Stephane Ducasse-3
Thanks for the responses so far. I see that I need to clarify my enquiry.
B-Trees and BDDs are not the same. BDDs are an efficient and compact representations for Boolean functions, sometimes used in SAT-solvers and electronics design. The key idea is that since the output must be 0 or 1, you can represent any Boolean function as a tree whose depth is the same as the number of bits in the input. To make the tree small and efficient, though, you need to eliminate any node whose two children are the same, and to share subtrees, so that you really get a DAG, not a tree. The full name for these efficient compressed trees is “Reduced Order Binary Decision Diagrams”, or ROBDDs. I was hoping that someone else had implemented the algorithms necessary to build this representation. Because sets can be considered to be Booleans functions (true => argument is in the set), you can use ROBDDs to efficiently represent large sets. To be clear, despite the word “diagram” in the name, one is not normally interested in drawing the BDD — except in the documentation for the package ;-). Normally, BDDs they are used to represent sets, or functions, where the drawing would be hopelessly large. The BuDDy package (http://buddy.sourceforge.net/manual/main.html) is an example of what I’m looking for, but unfortunately it’s in C++. Andrew > On 25 Oct 2017, at 21:39 , Stephane Ducasse <[hidden email]> wrote: > > Hi andrew > > I think that Avi did a package about BDD (but I thought it was special > binary trees) so this is probably the same. > Did you check on Squeaksource? > http://www.squeaksource.com/BTree.html > If this is what you are looking for I can help porting it to Pharo. > > Stef > > > On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <[hidden email]> wrote: >> Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk? >> >> Andrew >> >> > |
|
|
In reply to this post by kilon.alios
Yes. That’s why I asked for better practices.
Offline mode is to me of first importance. Let’s say I have a sets of Pharo app (nodes). Each one are independent running on different devices/computers. I want to exchange information between them according to some kind of contracts and also on connection availability. What I find different from usual message exchanges between two images is that I want to have full control on it. Plus I consider images offline by default (meaning two images have different versions of the same info). So conflict is the norm. Also, I don’t want to send message remotely (with behavior), I just want to send/share an information and sync them (when possible). Connection between 2 images can be through a network but eventually through a serial connection (usb cable, Bluetooth,...). So what I try to do as a proto is having connection controlled by my apps (I put the app offline (not by switching off wifi but by switching logically). I can see nodes availability and activate connexion manually then see if conflict resolution is ok, + I log every connecitions changes and exchanges). This is my plans for now but it may change ! ^^
I’ll look at stream sockets and mq stuffs. Cheers, Cédrick
|
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
|
Hi Cedric, a short answer: some of what you're trying to do has been traditionnally handled by object databases - multiple images retrieving and updating objects on a central store ensuring lots of good properties : see Gemstone. Another answer, since you're looking at content-addressable distributed file systems. You can resolve the offline mode with object duplications (duplicate all objects in the images using them), and, when coming back online, have a merge approach to reconcile changes between versions of shared objects. Content-based hashes as identifiers in a distributed store have very nice properties for that purpose, because, if image C has modified object 0xcdaff3, then that object has in fact become object 0xee345d for that image (and the unmodified object is still 0xcdaff3 for images A and B). I wouldn't be against a slightly higher granularity when dealing with object transfers, however. Regards, Thierry 2017-10-27 10:43 GMT+02:00 Cédrick Béler <[hidden email]>:
|
|
|
Yes , I’d like to avoid this centralized approach.
You nailed it. This is what I’d like to reproduce. Existing implementations out there seems to uses whatever nodes on the network to replicate the information. I’d like to control nodes where it is replicated. My nodes (all my app instances + nodes of person I’m exchanging information with + eventually friend of friend). What hash function would you use ? To get something compatible with ipfs, I’d need something like: https://github.com/multiformats/multihash It looks to me quite universal as self describing. But any (existing) hash method compatible with content hashing would do the job.
You mean at the pharo level ? Higher granularity means having more control on the exchange/merge ? Cheers, Cédrick |
Re: Exchanging information between 2 pharo applications (2 images running on two different computers)
|
|
2017-10-27 11:28 GMT+02:00 Cédrick Béler <[hidden email]>:
I mean that they have already some of the mechanisms in place, even if centralized.
Yes, because that makes them decentralized :)
We've done recent work on capability-based content adressing, but, first it's very slow (so you use a two level cryptosystem: the crypted header contains the key to decrypt the content, and the crypted header can only be decrypted if your private key has the right level of capabilities on the data item.
Anything that is fast, bonus points if it uses CPU special instructions (that you can't use from Pharo, of course ;)), and has the right cryptographic properties. Unless you go into a specific cryptosystem, I'd say that it is not important.
Interesting, but it looks like a minor issue in the overall scheme. Makes some of your system robust to evolution in the hash used, but, since first bytes are not well distributed, can you use it safely to build a multi-hash function system? Probably not.
No, just that the content-based address scheme is costly... and that a practical implementation would probably look to provide addresses only to large enough entities (a page containing objects, for example, or a large objects containing smaller ones). So that you donc create an address for each character of the string object describing the name of a person, say). Regards, Thierry
|
|
|
Yes :) But couldn't we limit the decentralized networks to some nodes (my node + nodes of persons/organization I trust) + eventually node they trusts. I have the feeling that limiting the networks would simplify the implementation cost as the underlying cryptographic system (I have a very naive understanding of such systems but I know I need them to sign/crypt information). I have no idea of a minimal node number that is acceptable to do content addressing.
Interesting ! I need to get more knowledge on that. BTW, I found these introduction courses well done on cryptography in information theory (https://khanacademy.org/computing/computer-science) Just thinking loud: I don’t actually need a quick implementation as the default is to have not synced version that get synchronized on opportunistic connection. What if each node has by default private keys and all public key of the external node I have relation with ? Wouldn’t it be simpler ? To prototype stuff, I think I can consider naive crypto and even do symmetric cryptography (and use Cryptography package or NaCL binding). I think I’ll do a very naive approach to sign and encrypt. If you know some existing applications that could help, please, let me know.
:) ok
Yes I agree. One idea was to use only ipfs for public information shared with everybody (even outside of nodes I have relation with). Clearly, not a central point.
Oh ok. I think this is how I was seeing it… says I exchange my phone number with somebody, the content hashed would be the full information (full content + version number + (maybe) its model hash + its geolocalized timestamp). Thanks a lot for the exchange, Cédrick
|
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |