Re: [Vm-dev] FFI Struct Argument Pass By Value Fails on Mac 64 bit

> i'm getting the idea that we should probably write a test suite/library for
> FFI
>
> On Nov 24, 2017, at 12:54 AM, Ben Coman <[hidden email]> wrote:
>
>
>
> On 24 November 2017 at 13:16, Ben Coman <[hidden email]> wrote:
>>
>>
>>
>> On 22 November 2017 at 21:59, Ben Coman <[hidden email]> wrote:
>>>
>>>
>>>
>>> On 22 November 2017 at 13:38, Todd Blanchard <[hidden email]> wrote:
>>>>
>>>>
>>>> I've been trying to track this down for a couple weeks now.
>>>>
>>>> I have concluded that structs passed by value to functions on the 64 bit
>>>> VM are not properly populated.  The struct's memory is all zero'd.
>>>>
>>>> I found this while trying to work with LibClang and found that functions
>>>> that fetched code locations from code ranges always returned invalid zero'd
>>>> locations.  After spending some time with lldb I have traced the problem
>>>> into the native code and found that the argument is not correct.
>>>>
>>>> I've carved out the wee bit of clang to reproduce this in a tiny
>>>> library.
>>>>
>>>> The gist of it is below and the entire file is included.  Basically the
>>>> struct passed to the function clang_getRangeStart is zero'd memory
>>>> regardless of the data I send from the image side.
>>>>
>>>> The build command I used on sierra is clang -shared -undefined
>>>> dynamic_lookup -o microclang.dylib microclang.c
>>>
>>>
>>> On Ubuntu 16.04 I used...
>>> $ clang -shared -fPIC -o libmicroclang.so microclang.c
>>>
>>> $ clang test.c -L. -l microclang
>>>    test.c:6:53: error: no member named 'begin_int_data' in
>>> 'CXSourceLocation'
>>>    if(clang_getRangeStart(clang_getArbitraryRange()).begin_int_data == 0)
>>>
>>> I presume you meant...
>>>     if(clang_getRangeStart(clang_getArbitraryRange()).int_data == 0)
>>> so correcting and continuing...
>>>
>>> $ clang test.c -L. -l microclang
>>> $ LD_LIBRARY_PATH=. ./a.out
>>> That failed
>>>
>>> So I'm not sure how to proceed.
>>> I was expecting that would work while Pharo failed.
>>>
>>> Now interestingly...
>>> $ clang test.c microlang.c
>>> $ ./a.out
>>> That worked
>>>
>>>
>>> So it seems a similar problem exists outside our FFI.
>>>
>>> cheers -ben
>>>
>>> P.S. I refactored you code to extract a header file (attached)
>>
>>
>> The issue is still beyond my ken, but I've made some progress towards
>> isolating/understanding the issue.
>> Attached zip exploded here for easy reference...
>>
>>
>> ___microlang.h___
>> typedef unsigned  uintptr_t;
>>
>> typedef struct {
>>   const void *ptr_data[2];
>> } CXSourceRange_;
>>
>> CXSourceRange_ clang_getArbitraryRange_();
>> int clang_getRangeEnd_(CXSourceRange_ range);
>>
>>
>>
>> ___microclang.c___
>> #include "microclang.h"
>> const char* libraryString = "library_pointer";
>>
>> CXSourceRange_ clang_getArbitraryRange_()
>> {       CXSourceRange_ range = {0};
>>         range.ptr_data[0] = (void*)libraryString;
>>         return range;
>> }
>>
>> int clang_getRangeEnd_(CXSourceRange_ range)
>> {       // Special decoding for CXSourceLocations for CXLoadedDiagnostics.
>>         if ((uintptr_t)range.ptr_data[0] & 0x1)
>>         {       return 0;       }
>>         else
>>         {       return 1;       }
>> }
>>
>>
>>
>>
>> ___test.c___
>> #include <stdio.h>
>> #include "microclang.h"
>> const char* localString =  "local_pointer";
>>
>> void test( CXSourceRange_ range, char *note )
>> {       int result = clang_getRangeEnd_(range);
>>         if(result == 0)
>>         {       printf("That failed (%s)\n", note); }
>>         else
>>         {       printf("That worked (%s)\n", note); }
>> }
>>
>> int main()
>> {       CXSourceRange_ range1 = clang_getArbitraryRange_();
>>         test(range1, "library string");
>>
>>         CXSourceRange_ range2 = {0};
>>         range2.ptr_data[0] = (void*)localString;
>>         test(range2, "local string");
>> }
>>
>>
>>
>> ___Makefile___
>> default: clean static shared
>>
>> clean:
>>         rm -f *so *App
>>         @echo
>>
>> shared:
>>         clang -g -o libmicroclang.so -shared -fPIC microclang.c
>>         clang -g -o sharedApp test.c -L. -lmicroclang
>>         LD_LIBRARY_PATH=. ./sharedApp
>>         @echo
>>
>> static:
>>         clang -g -o staticApp test.c microclang.c
>>         ./staticApp
>>         @echo
>>
>>
>>
>> Now running...
>> $ make > report
>>
>> gives...
>> ___report___
>> rm -f *so *App
>>
>> clang -g -o staticApp test.c microclang.c
>> ./staticApp
>> That worked (library string)
>> That worked (local string)
>>
>> clang -g -o libmicroclang.so -shared -fPIC microclang.c
>> clang -g -o sharedApp test.c -L. -lmicroclang
>> LD_LIBRARY_PATH=. ./sharedApp
>> That failed (library string)
>> That worked (local string)
>
>
>
> Further simplification dealing *only* with strings (see attached
> sharedLibString.zip)
> (also attached is clang3.zip as a step along the way)
>
> ___microclang.c___
> typedef unsigned  uintptr_t;
> const char* myLibraryString = "library_pointer";
>
> const char * lib_getLibraryString()
> {       return myLibraryString;
> }
>
> int lib_testString( const char *aString )
> {       unsigned test = (uintptr_t)aString & 0x1;
>         printf("\n  test=%d, aString-->%d\n", test, (uintptr_t)aString);
>         if (test)
>         {       return 0;       }
>         else
>         {       return 1;       }
> }
>
>
> ___test.c___
> #include <stdio.h>
> int lib_testString( const char *aString );
> const char *lib_getLibraryString();
> const char *localString =  "local_pointer";
>
> void test( int result, char *note )
> {       if(result == 0)
>         {       printf("That failed (%s)\n", note); }
>         else
>         {       printf("That worked (%s)\n", note); }
> }
>
> int main()
> {       const char * libraryString = lib_getLibraryString();
>         test(lib_testString(libraryString), "library string");
>
>         test(lib_testString(localString), "local string");
> }
>
>
> $ make > report
>
> ___report___
> rm -f *so *App
>
> clang -g -o staticApp test.c microclang.c
> ./staticApp
>
>   test=0, aString-->4196150
>   That worked (library string)
>
>   test=0, aString-->4196068
>   That worked (local string)
>
> clang -g -o libmicroclang.so -shared -fPIC microclang.c
> clang -g -o sharedApp test.c -L. -lmicroclang
> LD_LIBRARY_PATH=. ./sharedApp
>
>   test=1, aString-->-792512599
>   That failed (library string)
>
>   test=0, aString-->4196484
>   That worked (local string)
>
>
> cheers -ben
>
> <sharedLibString.zip><clang3.zip>
>
>