Reproducible VM Crash using UFFI

On 7 Oct 2017, at 00:11, Hernán Morales Durand <[hidden email]> wrote:

2017-10-06 2:39 GMT-03:00 Sven Van Caekenberghe <[hidden email]>:

On 6 Oct 2017, at 05:34, Hernán Morales Durand <[hidden email]> wrote:

Hi Nicolas

2017-10-05 16:04 GMT-03:00 Nicolas Cellier <[hidden email]>:

2017-10-05 15:42 GMT+02:00 Hernán Morales Durand <[hidden email]>:

I tried, now I get an exception "Use ExternalAddress instead!"

I guess the message means Use ExternalAddres in the "out" parameter.
But replacing byte with ExternalAddress also crashes the VM (crash.dmp
attached).



2017-10-05 10:03 GMT-03:00 Esteban Lorenzano <[hidden email]>:

On 5 Oct 2017, at 14:55, Hernán Morales Durand
<[hidden email]> wrote:

Forgot to comment that Nacl worked in Pharo 5.

yes but that was with NB and there are some minimum differences.
I do not have the library and I lack the time to try more, but seems to
me that here:

apiCryptoHashSha512Output: outByteArray input: inByteArray inputLength:
inByteArrayLength

      ^ self
              ffiCall: #(long crypto_hash_sha512_ref (byte *
outByteArray, byte * inByteArray, ulonglong * inByteArrayLength))
              module: 'libsodium’.


instead "byte * outByteArray”, you want "byte **outByteArray”

can you try?

Esteban

Hmm no, internet reference says:

extern int crypto_hash_sha512_ref(unsigned char *,const unsigned char
*,unsigned long long);

The error is rather that you pass the address of length rather than the
length...
So you pass an erroneous length that may lead to a segfault...

Yes, that extra * was the error indeed.
Thank you Nicolas.

I do not know the VM internals, however I wonder if it is
understandable to expect a VM crash for a typo like this?
Maybe it is too difficult to catch in the VM side?

Just asking, not criticizing anyone's work.

Of course that is normal. This is C code, any error has the potential to crash your program. In C there are no runtime safe guards.

There is a reason why we like/use Pharo.

However there are a couple of lightweight libraries which claim to
provide exceptions for C:
https://github.com/guillermocalvo/exceptions4c or this one
https://github.com/psevon/exceptions-and-raii-in-c

I wonder if something like that could be used in UFFI

Hernán

Cheers,

Hernán

2017-10-05 3:23 GMT-03:00 Esteban Lorenzano <[hidden email]>:

H Hernani,

Most probably is a problem in the library and not UFFI, but I could
not know without a crash report.

Esteban

On 5 Oct 2017, at 06:00, Hernán Morales Durand
<[hidden email]> wrote:

Hi,

I ported Nacl (a libsodium wrapper) from the old FFI apicall: format
to use the UFFI ffiCall:, but there should be something terribly
wrong
because is crashing the VM, in both Windows 8.1 and Linux.

How to reproduce in Pharo 6.1

Metacello new
smalltalkhubUser: 'tonyg' project: 'Crypto-Nacl';
configuration: 'Nacl';
version: #development;
load.

(Nacl hashString: 'The quick brown fox jumps over the lazy dog') hex

This one does deserve a bug entry?

Cheers,

Hernán

On 8 Oct 2017, at 10:00, Hernán Morales Durand <[hidden email]> wrote:

Hi Esteban

2017-10-07 5:51 GMT-03:00 Esteban Lorenzano <[hidden email]>:

On 7 Oct 2017, at 00:11, Hernán Morales Durand <[hidden email]>
wrote:

2017-10-06 2:39 GMT-03:00 Sven Van Caekenberghe <[hidden email]>:



On 6 Oct 2017, at 05:34, Hernán Morales Durand <[hidden email]>
wrote:

Hi Nicolas

2017-10-05 16:04 GMT-03:00 Nicolas Cellier
<[hidden email]>:



2017-10-05 15:42 GMT+02:00 Hernán Morales Durand <[hidden email]>:


I tried, now I get an exception "Use ExternalAddress instead!"

I guess the message means Use ExternalAddres in the "out" parameter.
But replacing byte with ExternalAddress also crashes the VM (crash.dmp
attached).



2017-10-05 10:03 GMT-03:00 Esteban Lorenzano <[hidden email]>:


On 5 Oct 2017, at 14:55, Hernán Morales Durand
<[hidden email]> wrote:

Forgot to comment that Nacl worked in Pharo 5.


yes but that was with NB and there are some minimum differences.
I do not have the library and I lack the time to try more, but seems to
me that here:

apiCryptoHashSha512Output: outByteArray input: inByteArray inputLength:
inByteArrayLength

     ^ self
             ffiCall: #(long crypto_hash_sha512_ref (byte *
outByteArray, byte * inByteArray, ulonglong * inByteArrayLength))
             module: 'libsodium’.


instead "byte * outByteArray”, you want "byte **outByteArray”

can you try?

Esteban



Hmm no, internet reference says:

extern int crypto_hash_sha512_ref(unsigned char *,const unsigned char
*,unsigned long long);

The error is rather that you pass the address of length rather than the
length...
So you pass an erroneous length that may lead to a segfault...


Yes, that extra * was the error indeed.
Thank you Nicolas.

I do not know the VM internals, however I wonder if it is
understandable to expect a VM crash for a typo like this?
Maybe it is too difficult to catch in the VM side?

Just asking, not criticizing anyone's work.


Of course that is normal. This is C code, any error has the potential to
crash your program. In C there are no runtime safe guards.

There is a reason why we like/use Pharo.


However there are a couple of lightweight libraries which claim to
provide exceptions for C:
https://github.com/guillermocalvo/exceptions4c or this one
https://github.com/psevon/exceptions-and-raii-in-c

I wonder if something like that could be used in UFFI


Nothing in C *ever* will save you from accessing bad memory blocks. And yes,
it will always crash your program.

Let me see if I get this right.

If there is nothing to prevent accessing bad memory blocks, assuming
accessing or executing invalid memory, then how the memory debuggers
like mpatrol or valgrind, do their work? Just a rethoric question to
make a point, please keep reading…

Memory debuggers don't prevent segmentation violation of the debuggee,
but they don't crash either. I know they slow down application, but it
wouldn't be preferable to gracefully recover from crashes during
coding time? My (maybe crazy) idea was while coding sensitive parts
(like using UFFI) one can use a "debugger vm", and use the "normal vm"
when safer.

So I think we are talking about different things here. I don't want to
save "bad memory block" errors nor dream about bullet proof VM, but if
we know the bullet then let's use a nice bulletproof vest :)

What do you think about having some self-healing feedback loop to
"hide" the crash? like the described in
https://link.springer.com/article/10.1007/s00607-010-0107-y

In general, how the VM developers view the resilient systems approach?

Are they too far away to get it right given the current VM complexity?

those links you point are exception mechanisms, a set of tools to easy the
catch of programming exceptions.
But, if you try to read a chunk of memory in, say 0x8000 and the
data/function that should be there is now garbage, you’re screw.

as Sven pointed, that’s why we use higher level languages, with GC and all
that.

Thank you for the clarifications BTW

Hernán

Esteban



Hernán




Cheers,

Hernán

2017-10-05 3:23 GMT-03:00 Esteban Lorenzano <[hidden email]>:

H Hernani,

Most probably is a problem in the library and not UFFI, but I could
not know without a crash report.

Esteban

On 5 Oct 2017, at 06:00, Hernán Morales Durand
<[hidden email]> wrote:

Hi,

I ported Nacl (a libsodium wrapper) from the old FFI apicall: format
to use the UFFI ffiCall:, but there should be something terribly
wrong
because is crashing the VM, in both Windows 8.1 and Linux.

How to reproduce in Pharo 6.1

Metacello new
smalltalkhubUser: 'tonyg' project: 'Crypto-Nacl';
configuration: 'Nacl';
version: #development;
load.

(Nacl hashString: 'The quick brown fox jumps over the lazy dog') hex

This one does deserve a bug entry?

Cheers,

Hernán