Esteban's ChangeLog week of 16 January 2017

yeah, I’m aware I need to iterate on the text exporter to have some better output :P

Esteban

> On 23 Jan 2017, at 09:00, [hidden email] wrote:
>
> Hello!
>
> This is my weekly ChangeLog, from 16 January 2017 to 22 January 2017.
> You can see it in a better format by going here: http://log.smallworks.eu
>
> ChangeLog
> =========
>
> 20 January 2017:
> ----------------
>
>  * I'm still fixing a couple of final failing tests on iceberg,
>    multi-remotes branch. Problem is that there are still somethings to understand
>    on the "command line
>    backend" (we need to find a better name for this), and the #pushTo: method: if I
>    do it as I
>    think is correct, a couple of failing tests appear... but then, I need to
>    reflect if this is
>    a problem on the functionality or on the tests.
>
> 19 January 2017:
> ----------------
>
>  * I think I finally got the problem with FT2Plugin!
>    Yesterday night talking in slack I realised that there was actually duplicated
>    instances of
>    FT2Handle hierarchy with exactly same handle  (pointer address). This, of
>    course, causes a double free when those instances are released... and
>    that's the VM crash :)
>    So I quickly sketched a workaround, that you can see here:
>    FT2Handle>>pvtDestroyHandle
>    "This should only be sent from the finalizer.
>    It runs inside a mutex because in strange cases it can happen that this is
>    executed twice,
>    causing a primitiveFailed to be raised."
>    self class destroyMutex critical: [ | handleToRelease |
>    self isValid ifFalse: [ ^self ].
>    handleToRelease := self handle copy.
>    self primDestroyHandle.
>    "This is super-ugly, but it will prevent duplicated handles to be released"
>    FT2Handle allSubInstancesDo: [ :each |
>    (handleToRelease = each handle) ifTrue: [ each beNull ] ] ]
>    and so far nobody who is testing it crashed!
>    This can be important, but of course... now we need to realise why the
>    duplicated instances are happening.
>  * I spent some time preparing a new way to annoy people :)
>    I prepared a way to send a digest mail about my activity (this stream) to
>    pharo-dev list.
>    I hope it will not be too much.
>
> 18 January 2017:
> ----------------
>
>  * ... and still some more work on iceberg, this time making some test
>    pass on the command line backend (who is just a fallback, but well... we still
>    need it around).
>  * Still working on UFFI 64bits. Now I introduced a new hierarchy, FFIArchitecture
>    for now with two children:
>    * FFI_i386
>    * FFI_x86_64
>    the purpose is obvious: to allow UFFI to handle differences between different
>    architectures (for example,
>    long has to be parsed to FFIInt32 in i386 and to FFIInt64 in x86_64).
>
> 17 January 2017:
> ----------------
>
>  * I'm working on the UFFI support for 64bits.
>    In general, it works fine, but there is a huge problem with structures. What's
>    this problem? Well, let's take
>    this simple structure as example:
>    struct point {
>    unsigned long x;
>    unsigned long y;
>    }
>    Well, thing is in 32bits sizeof(struct point) = 8 while in 64bits sizeof(struct
>    point) = 16, and of
>    course this breaks our current implementation which would be something like this
>    StructPoint>>x
>    ^ handle unsignedLongAt: 1
>    StructPoint>>y
>    ^ handle unsignedLongAt: 5
>    and of course, this is not correct for 64bits.
>    My solution is quite simple, instead hardcoding the field offsets, I store those
>    values into class variables
>    that can be calculated each session (but just once).
>    So, this structure would end looking like this:
>    StructPoint>>x
>    ^ handle unsignedLongAt: OFFSET_X
>    StructPoint>>y
>    ^ handle unsignedLongAt: OFFSET_Y
>    this would solve the problem for most of the cases, but there is still the issue
>    that emerges when complete
>    implementation of the struct changes between platforms. I'm planning to
>    implement a strategy pattern to solve
>    this, but not yet (probably I will wait until having a concrete case).
>    This seems to be working, but now I have other problems not related to UFFI:
>    many people has implemented
>    void * parameters of funtions as unsigned long (including me, time ago) which
>    was correct in 32bits but
>    it is not in 64bits. So now I need to review this implementations to see where a
>    fix is needed.
>    But this is not directly a problem of UFFI, I think.
>  * I worked with Nico on fixing libgit2 backend tests for iceberg.
>    Now (more thanks to Nico than me, heh), test for multi-remotes branch are
>    passing for linux but
>    still not for macOS (and forget it about Pharo 5.0).
>    I suspect a problem with VM, I need to review if I can configure smalltalkci
>    to work with latest vm instead stable.
>
> 15 January 2017:
> ----------------
>
>  * ... and now I restored the posix permissions to windows platform. Next latest VM
>    will have this change, I
>    hope that will end the problems with FilePlugin, but we'll see.
>  * I just fixed a problem with FilePlugin on macOS: the problem was when
>    determining if a symlink is also a
>    directory:
>    ‘/tmp’ asFileReference isDirectory
>    was answering false, because it is a symlink and when trying to resolve the
>    symlink to obtain the
>    attributes, there was a cyclic condition that was transforming ‘/tmp’ into
>    ‘/private/tmp’ and then
>    back to ‘/tmp’, and because of that the test for directory was failing.
>
> cheers!
> Esteban

On 23 Jan 2017, at 10:02, Stephane Ducasse <[hidden email]> wrote:

Tx a lot esteban!

This is so great. 

thanks :)

Showing what you are doing is important. 

Can you send this mail to the consortium mailing-list too?

of course, this is a Pharo script after all ;)

Esteban

On Mon, Jan 23, 2017 at 9:22 AM, Esteban Lorenzano <[hidden email]> wrote:

yeah, I’m aware I need to iterate on the text exporter to have some better output :P

Esteban

> On 23 Jan 2017, at 09:00, [hidden email] wrote:
>
> Hello!
>
> This is my weekly ChangeLog, from 16 January 2017 to 22 January 2017.
> You can see it in a better format by going here: http://log.smallworks.eu
>
> ChangeLog
> =========
>
> 20 January 2017:
> ----------------
>
>  * I'm still fixing a couple of final failing tests on iceberg,
>    multi-remotes branch. Problem is that there are still somethings to understand
>    on the "command line
>    backend" (we need to find a better name for this), and the #pushTo: method: if I
>    do it as I
>    think is correct, a couple of failing tests appear... but then, I need to
>    reflect if this is
>    a problem on the functionality or on the tests.
>
> 19 January 2017:
> ----------------
>
>  * I think I finally got the problem with FT2Plugin!
>    Yesterday night talking in slack I realised that there was actually duplicated
>    instances of
>    FT2Handle hierarchy with exactly same handle  (pointer address). This, of
>    course, causes a double free when those instances are released... and
>    that's the VM crash :)
>    So I quickly sketched a workaround, that you can see here:
>    FT2Handle&gt;&gt;pvtDestroyHandle
>    "This should only be sent from the finalizer.
>    It runs inside a mutex because in strange cases it can happen that this is
>    executed twice,
>    causing a primitiveFailed to be raised."
>    self class destroyMutex critical: [ | handleToRelease |
>    self isValid ifFalse: [ ^self ].
>    handleToRelease := self handle copy.
>    self primDestroyHandle.
>    "This is super-ugly, but it will prevent duplicated handles to be released"
>    FT2Handle allSubInstancesDo: [ :each |
>    (handleToRelease = each handle) ifTrue: [ each beNull ] ] ]
>    and so far nobody who is testing it crashed!
>    This can be important, but of course... now we need to realise why the
>    duplicated instances are happening.
>  * I spent some time preparing a new way to annoy people :)
>    I prepared a way to send a digest mail about my activity (this stream) to
>    pharo-dev list.
>    I hope it will not be too much.
>
> 18 January 2017:
> ----------------
>
>  * ... and still some more work on iceberg, this time making some test
>    pass on the command line backend (who is just a fallback, but well... we still
>    need it around).
>  * Still working on UFFI 64bits. Now I introduced a new hierarchy, FFIArchitecture
>    for now with two children:
>    * FFI_i386
>    * FFI_x86_64
>    the purpose is obvious: to allow UFFI to handle differences between different
>    architectures (for example,
>    long has to be parsed to FFIInt32 in i386 and to FFIInt64 in x86_64).
>
> 17 January 2017:
> ----------------
>
>  * I'm working on the UFFI support for 64bits.
>    In general, it works fine, but there is a huge problem with structures. What's
>    this problem? Well, let's take
>    this simple structure as example:
>    struct point {
>    unsigned long x;
>    unsigned long y;
>    }
>    Well, thing is in 32bits sizeof(struct point) = 8 while in 64bits sizeof(struct
>    point) = 16, and of
>    course this breaks our current implementation which would be something like this
>    StructPoint&gt;&gt;x
>    ^ handle unsignedLongAt: 1
>    StructPoint&gt;&gt;y
>    ^ handle unsignedLongAt: 5
>    and of course, this is not correct for 64bits.
>    My solution is quite simple, instead hardcoding the field offsets, I store those
>    values into class variables
>    that can be calculated each session (but just once).
>    So, this structure would end looking like this:
>    StructPoint&gt;&gt;x
>    ^ handle unsignedLongAt: OFFSET_X
>    StructPoint&gt;&gt;y
>    ^ handle unsignedLongAt: OFFSET_Y
>    this would solve the problem for most of the cases, but there is still the issue
>    that emerges when complete
>    implementation of the struct changes between platforms. I'm planning to
>    implement a strategy pattern to solve
>    this, but not yet (probably I will wait until having a concrete case).
>    This seems to be working, but now I have other problems not related to UFFI:
>    many people has implemented
>    void * parameters of funtions as unsigned long (including me, time ago) which
>    was correct in 32bits but
>    it is not in 64bits. So now I need to review this implementations to see where a
>    fix is needed.
>    But this is not directly a problem of UFFI, I think.
>  * I worked with Nico on fixing libgit2 backend tests for iceberg.
>    Now (more thanks to Nico than me, heh), test for multi-remotes branch are
>    passing for linux but
>    still not for macOS (and forget it about Pharo 5.0).
>    I suspect a problem with VM, I need to review if I can configure smalltalkci
>    to work with latest vm instead stable.
>
> 15 January 2017:
> ----------------
>
>  * ... and now I restored the posix permissions to windows platform. Next latest VM
>    will have this change, I
>    hope that will end the problems with FilePlugin, but we'll see.
>  * I just fixed a problem with FilePlugin on macOS: the problem was when
>    determining if a symlink is also a
>    directory:
>    ‘/tmp’ asFileReference isDirectory
>    was answering false, because it is a symlink and when trying to resolve the
>    symlink to obtain the
>    attributes, there was a cyclic condition that was transforming ‘/tmp’ into
>    ‘/private/tmp’ and then
>    back to ‘/tmp’, and because of that the test for directory was failing.
>
> cheers!
> Esteban