FFI and C subsystems

James,

It seems that is possible to create a new header file and includes 2 subsystems in the same GS Library.

For example sftp_server_all.h:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

Then:
| header wrapperClass wrapper |
header := CHeader path: 'sftp_server_all.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

Do you see any problem with this approach ?

The new library generated all functions now i going to check if they work.
It should work i think ... (but again not a C expert -dealing with C again after 15 years-)



regards,
Bruno

Bruno, 

My understanding is that the header parser follows all the #include directives, so will find everything that could be called from C code, including abs() in libc. But the process of calling a dynamic library at runtime involves doing a name lookup of entry points *for that library,* not for anything visible *to* the library, so libssh does not have an abs() entry point. For that you would need a reference to libc. 

The wrapper generator does not know what is actually in the given library, just what functions are defined by all included header files. So you have to do some selection on your own. One wrapper API has a filter as an argument, and so for a library like GemStone C Interface where all the functions begin with ‘gci’ we can filter to only generate wrappers for those functions (it is unlikely that libc has gci*() functions!).

So, yes, you need to have a different library object for each library so the system can do name lookups properly.

James

On Oct 9, 2020, at 5:40 AM, Bruno Buzzi Brassesco via Glass <[hidden email]> wrote:

Hi,

Maybe i'm wrong in some concepts here (not a C expert) but ...

If a C library (libssh) has a subsystem (sftp) in order to create a GS wrapper i can do:

| header wrapperClass wrapper |
header := CHeader path: '/usr/include/libssh/sftp.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

In this case I got all the functions of Libssh and also SFTP, so far so good.

But what happens when there is more than one C subsystem?

How can I generate only one GS wrapper for the original C library (libssh) and all its subsystems ?

Or I will have to create one wrapper per subsystem ?

regards,

bruno

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

James,

The new header files does not have any function declaration only "#includes" sentences.

Actually the entire new header files is:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

So i do not see how a lookup will fail. But again maybe I'm missing something :)

regards,
bruno

Bruno,

If it works, then it may be fine, but I’ll be somewhat surprised if that happens. My vague idea of the way a program interacts with a shared library is as follows: (1) you ask the OS to load a library (such as '/usr/lib64/libssh.so.4.4.0’ and the OS give you back a ‘handle’ to that library; (2) you ask the OS to look up a function in the library (you pass a string with the function name and you pass in the previously returned handle) and the OS give you back a pointer to a function in that library; (3) you use the pointer to call the function. These are the steps that GemStone handles; you will have previously created an object that encapsulates the function name, parameter types, and return types.

If you use the wrong header to generate a list of expected entry points in a library, then the Smalltalk code will still be generated and you can still load the library (step 1), but when you attempt to lookup a function from library A in library B, the OS will fail at step 2 since the function does not exist. I think that if you want to call functions in two libraries, then you need to load both libraries (step 1) and do the name lookup in the proper library (step 2). 

But my expertise in this area is several years old and was on the Smalltalk side not on the VM side.

James

On Nov 26, 2020, at 4:42 AM, bruno buzzi brassesco via Glass <[hidden email]> wrote:

James,

It seems that is possible to create a new header file and includes 2 subsystems in the same GS Library.

For example sftp_server_all.h:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

Then:
| header wrapperClass wrapper |
header := CHeader path: 'sftp_server_all.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

Do you see any problem with this approach ?

The new library generated all functions now i going to check if they work.
It should work i think ... (but again not a C expert -dealing with C again after 15 years-)



regards,
Bruno

On 9/10/2020 14:53, James Foster via Glass wrote:

Bruno, 

My understanding is that the header parser follows all the #include directives, so will find everything that could be called from C code, including abs() in libc. But the process of calling a dynamic library at runtime involves doing a name lookup of entry points *for that library,* not for anything visible *to* the library, so libssh does not have an abs() entry point. For that you would need a reference to libc. 

The wrapper generator does not know what is actually in the given library, just what functions are defined by all included header files. So you have to do some selection on your own. One wrapper API has a filter as an argument, and so for a library like GemStone C Interface where all the functions begin with ‘gci’ we can filter to only generate wrappers for those functions (it is unlikely that libc has gci*() functions!).

So, yes, you need to have a different library object for each library so the system can do name lookups properly.

James

On Oct 9, 2020, at 5:40 AM, Bruno Buzzi Brassesco via Glass <[hidden email]> wrote:

Hi,

Maybe i'm wrong in some concepts here (not a C expert) but ...

If a C library (libssh) has a subsystem (sftp) in order to create a GS wrapper i can do:

| header wrapperClass wrapper |
header := CHeader path: '/usr/include/libssh/sftp.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

In this case I got all the functions of Libssh and also SFTP, so far so good.

But what happens when there is more than one C subsystem?

How can I generate only one GS wrapper for the original C library (libssh) and all its subsystems ?

Or I will have to create one wrapper per subsystem ?

regards,

bruno

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

James,

But all these C files are compiled into one library '/usr/lib64/libssh.so.4.8.5'.

If i create one GS wrapper per each "subsystem" (maybe this word is confused) all are going to point to:
CLibrary named: '/usr/lib64/libssh.so.4.8.5'.

Also in Windows is only one DLL.

So there is only one shared library binary with all functions. And depending of the header used to generate the wrapper you get a subset of the available functions to call.

regards,
bruno

Bruno,

Header files describe what is in a library. The “libssh.h” header describes what is in the “libssh” library and the “sftp.h” header describes what is in the “sftp” library. While you can create a new header file that (wrongly) claims that function A is in library B, that doesn’t mean that when we go to look for A in B that it will be found.

Just because I put down Casa Rosada as your address on my contact list doesn’t make it so. I might mail a letter, but you won’t get it. Just because you pretend that a function is in a library (making up a new header) doesn’t make it so. You may be able to create the Smalltalk wrapper, but when you call it the function won’t be found because it isn’t in the library.

The header file is simply a text file that describes what is in a binary file. Changing the text file doesn’t change the binary file.

James

On Nov 26, 2020, at 9:01 AM, bruno buzzi brassesco <[hidden email]> wrote:

James,

The new header files does not have any function declaration only "#includes" sentences.

Actually the entire new header files is:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

So i do not see how a lookup will fail. But again maybe I'm missing something :)

regards,
bruno

On 26/11/2020 13:21, [hidden email] wrote:

Bruno,

If it works, then it may be fine, but I’ll be somewhat surprised if that happens. My vague idea of the way a program interacts with a shared library is as follows: (1) you ask the OS to load a library (such as '/usr/lib64/libssh.so.4.4.0’ and the OS give you back a ‘handle’ to that library; (2) you ask the OS to look up a function in the library (you pass a string with the function name and you pass in the previously returned handle) and the OS give you back a pointer to a function in that library; (3) you use the pointer to call the function. These are the steps that GemStone handles; you will have previously created an object that encapsulates the function name, parameter types, and return types.

If you use the wrong header to generate a list of expected entry points in a library, then the Smalltalk code will still be generated and you can still load the library (step 1), but when you attempt to lookup a function from library A in library B, the OS will fail at step 2 since the function does not exist. I think that if you want to call functions in two libraries, then you need to load both libraries (step 1) and do the name lookup in the proper library (step 2). 

But my expertise in this area is several years old and was on the Smalltalk side not on the VM side.

James

On Nov 26, 2020, at 4:42 AM, bruno buzzi brassesco via Glass <[hidden email]> wrote:

James,

It seems that is possible to create a new header file and includes 2 subsystems in the same GS Library.

For example sftp_server_all.h:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

Then:
| header wrapperClass wrapper |
header := CHeader path: 'sftp_server_all.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

Do you see any problem with this approach ?

The new library generated all functions now i going to check if they work.
It should work i think ... (but again not a C expert -dealing with C again after 15 years-)



regards,
Bruno

On 9/10/2020 14:53, James Foster via Glass wrote:

Bruno, 

My understanding is that the header parser follows all the #include directives, so will find everything that could be called from C code, including abs() in libc. But the process of calling a dynamic library at runtime involves doing a name lookup of entry points *for that library,* not for anything visible *to* the library, so libssh does not have an abs() entry point. For that you would need a reference to libc. 

The wrapper generator does not know what is actually in the given library, just what functions are defined by all included header files. So you have to do some selection on your own. One wrapper API has a filter as an argument, and so for a library like GemStone C Interface where all the functions begin with ‘gci’ we can filter to only generate wrappers for those functions (it is unlikely that libc has gci*() functions!).

So, yes, you need to have a different library object for each library so the system can do name lookups properly.

James

On Oct 9, 2020, at 5:40 AM, Bruno Buzzi Brassesco via Glass <[hidden email]> wrote:

Hi,

Maybe i'm wrong in some concepts here (not a C expert) but ...

If a C library (libssh) has a subsystem (sftp) in order to create a GS wrapper i can do:

| header wrapperClass wrapper |
header := CHeader path: '/usr/include/libssh/sftp.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

In this case I got all the functions of Libssh and also SFTP, so far so good.

But what happens when there is more than one C subsystem?

How can I generate only one GS wrapper for the original C library (libssh) and all its subsystems ?

Or I will have to create one wrapper per subsystem ?

regards,

bruno

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

Bruno,

Header files describe what is in a library. The “libssh.h” header describes what is in the “libssh” library and the “sftp.h” header describes what is in the “sftp” library. While you can create a new header file that (wrongly) claims that function A is in library B, that doesn’t mean that when we go to look for A in B that it will be found.

Just because I put down Casa Rosada as your address on my contact list doesn’t make it so. I might mail a letter, but you won’t get it. Just because you pretend that a function is in a library (making up a new header) doesn’t make it so. You may be able to create the Smalltalk wrapper, but when you call it the function won’t be found because it isn’t in the library.

The header file is simply a text file that describes what is in a binary file. Changing the text file doesn’t change the binary file.

James

On Nov 26, 2020, at 9:01 AM, bruno buzzi brassesco <[hidden email]> wrote:

James,

The new header files does not have any function declaration only "#includes" sentences.

Actually the entire new header files is:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

So i do not see how a lookup will fail. But again maybe I'm missing something :)

regards,
bruno

On 26/11/2020 13:21, [hidden email] wrote:

Bruno,

If it works, then it may be fine, but I’ll be somewhat surprised if that happens. My vague idea of the way a program interacts with a shared library is as follows: (1) you ask the OS to load a library (such as '/usr/lib64/libssh.so.4.4.0’ and the OS give you back a ‘handle’ to that library; (2) you ask the OS to look up a function in the library (you pass a string with the function name and you pass in the previously returned handle) and the OS give you back a pointer to a function in that library; (3) you use the pointer to call the function. These are the steps that GemStone handles; you will have previously created an object that encapsulates the function name, parameter types, and return types.

If you use the wrong header to generate a list of expected entry points in a library, then the Smalltalk code will still be generated and you can still load the library (step 1), but when you attempt to lookup a function from library A in library B, the OS will fail at step 2 since the function does not exist. I think that if you want to call functions in two libraries, then you need to load both libraries (step 1) and do the name lookup in the proper library (step 2). 

But my expertise in this area is several years old and was on the Smalltalk side not on the VM side.

James

On Nov 26, 2020, at 4:42 AM, bruno buzzi brassesco via Glass <[hidden email]> wrote:

James,

It seems that is possible to create a new header file and includes 2 subsystems in the same GS Library.

For example sftp_server_all.h:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

Then:
| header wrapperClass wrapper |
header := CHeader path: 'sftp_server_all.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

Do you see any problem with this approach ?

The new library generated all functions now i going to check if they work.
It should work i think ... (but again not a C expert -dealing with C again after 15 years-)



regards,
Bruno

On 9/10/2020 14:53, James Foster via Glass wrote:

Bruno, 

My understanding is that the header parser follows all the #include directives, so will find everything that could be called from C code, including abs() in libc. But the process of calling a dynamic library at runtime involves doing a name lookup of entry points *for that library,* not for anything visible *to* the library, so libssh does not have an abs() entry point. For that you would need a reference to libc. 

The wrapper generator does not know what is actually in the given library, just what functions are defined by all included header files. So you have to do some selection on your own. One wrapper API has a filter as an argument, and so for a library like GemStone C Interface where all the functions begin with ‘gci’ we can filter to only generate wrappers for those functions (it is unlikely that libc has gci*() functions!).

So, yes, you need to have a different library object for each library so the system can do name lookups properly.

James

On Oct 9, 2020, at 5:40 AM, Bruno Buzzi Brassesco via Glass <[hidden email]> wrote:

Hi,

Maybe i'm wrong in some concepts here (not a C expert) but ...

If a C library (libssh) has a subsystem (sftp) in order to create a GS wrapper i can do:

| header wrapperClass wrapper |
header := CHeader path: '/usr/include/libssh/sftp.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

In this case I got all the functions of Libssh and also SFTP, so far so good.

But what happens when there is more than one C subsystem?

How can I generate only one GS wrapper for the original C library (libssh) and all its subsystems ?

Or I will have to create one wrapper per subsystem ?

regards,

bruno

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

Martin/James,

Thank for your answers.

We are implementing a SSH Server (with SFTP also). The sftp.h header implement all related stuff to SFTP but if you load:
CHeader path: '/usr/include/libssh/sftp.h'.

Then you have the SFTP stuff but not the server capabilities. And if you load server.h header:
CHeader path: '/usr/include/libssh/server.h'.

Then you have the Server stuff but not the SFTP.

And there is almost no documentation for the server part.
(the SSH and SFTP client are working ok)

So i did this kind of workaround to have both in the same GS wrapper.

I did this because i was not sure if i can combine calls of two different wrapper in GS.

For example if i generate a sshSession (aCPointer) from LibsshServerWrapper (server.h) then Can i use it (aCPointer) as argument to call a function in LibsshSftpWrapper?

regards,
bruno

Hi Bruno, James,

libssh.so is one of the libraries that defines many functions, and the prototypes for those functions are spread over multiple header files.

When I personally deal with such libraries in GemStone I tend to make one class per header file, since that seems a bit cleaner to me. The documentation for each function says which header to include, and that tells me which one of my classes to use.

However, this is a matter of taste. Certainly it should work to include libssh.h and sftp.h into the same class, since the functions defined in both are for the same shared library. I don't know about server.h -- in my system the only headers with that name are unrelated to libssh.so.

Regards,
-Martin

On 11/26/20 9:13 AM, James Foster via Glass wrote:

Bruno,

Header files describe what is in a library. The “libssh.h” header describes what is in the “libssh” library and the “sftp.h” header describes what is in the “sftp” library. While you can create a new header file that (wrongly) claims that function A is in library B, that doesn’t mean that when we go to look for A in B that it will be found.

Just because I put down Casa Rosada as your address on my contact list doesn’t make it so. I might mail a letter, but you won’t get it. Just because you pretend that a function is in a library (making up a new header) doesn’t make it so. You may be able to create the Smalltalk wrapper, but when you call it the function won’t be found because it isn’t in the library.

The header file is simply a text file that describes what is in a binary file. Changing the text file doesn’t change the binary file.

James

On Nov 26, 2020, at 9:01 AM, bruno buzzi brassesco <[hidden email]> wrote:

James,

The new header files does not have any function declaration only "#includes" sentences.

Actually the entire new header files is:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

So i do not see how a lookup will fail. But again maybe I'm missing something :)

regards,
bruno

On 26/11/2020 13:21, [hidden email] wrote:

Bruno,

If it works, then it may be fine, but I’ll be somewhat surprised if that happens. My vague idea of the way a program interacts with a shared library is as follows: (1) you ask the OS to load a library (such as '/usr/lib64/libssh.so.4.4.0’ and the OS give you back a ‘handle’ to that library; (2) you ask the OS to look up a function in the library (you pass a string with the function name and you pass in the previously returned handle) and the OS give you back a pointer to a function in that library; (3) you use the pointer to call the function. These are the steps that GemStone handles; you will have previously created an object that encapsulates the function name, parameter types, and return types.

If you use the wrong header to generate a list of expected entry points in a library, then the Smalltalk code will still be generated and you can still load the library (step 1), but when you attempt to lookup a function from library A in library B, the OS will fail at step 2 since the function does not exist. I think that if you want to call functions in two libraries, then you need to load both libraries (step 1) and do the name lookup in the proper library (step 2). 

But my expertise in this area is several years old and was on the Smalltalk side not on the VM side.

James

On Nov 26, 2020, at 4:42 AM, bruno buzzi brassesco via Glass <[hidden email]> wrote:

James,

It seems that is possible to create a new header file and includes 2 subsystems in the same GS Library.

For example sftp_server_all.h:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

Then:
| header wrapperClass wrapper |
header := CHeader path: 'sftp_server_all.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

Do you see any problem with this approach ?

The new library generated all functions now i going to check if they work.
It should work i think ... (but again not a C expert -dealing with C again after 15 years-)



regards,
Bruno

On 9/10/2020 14:53, James Foster via Glass wrote:

Bruno, 

My understanding is that the header parser follows all the #include directives, so will find everything that could be called from C code, including abs() in libc. But the process of calling a dynamic library at runtime involves doing a name lookup of entry points *for that library,* not for anything visible *to* the library, so libssh does not have an abs() entry point. For that you would need a reference to libc. 

The wrapper generator does not know what is actually in the given library, just what functions are defined by all included header files. So you have to do some selection on your own. One wrapper API has a filter as an argument, and so for a library like GemStone C Interface where all the functions begin with ‘gci’ we can filter to only generate wrappers for those functions (it is unlikely that libc has gci*() functions!).

So, yes, you need to have a different library object for each library so the system can do name lookups properly.

James

On Oct 9, 2020, at 5:40 AM, Bruno Buzzi Brassesco via Glass <[hidden email]> wrote:

Hi,

Maybe i'm wrong in some concepts here (not a C expert) but ...

If a C library (libssh) has a subsystem (sftp) in order to create a GS wrapper i can do:

| header wrapperClass wrapper |
header := CHeader path: '/usr/include/libssh/sftp.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

In this case I got all the functions of Libssh and also SFTP, so far so good.

But what happens when there is more than one C subsystem?

How can I generate only one GS wrapper for the original C library (libssh) and all its subsystems ?

Or I will have to create one wrapper per subsystem ?

regards,

bruno

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

Martin/James,

Thank for your answers.

We are implementing a SSH Server (with SFTP also). The sftp.h header implement all related stuff to SFTP but if you load:
CHeader path: '/usr/include/libssh/sftp.h'.

Then you have the SFTP stuff but not the server capabilities. And if you load server.h header:
CHeader path: '/usr/include/libssh/server.h'.

Then you have the Server stuff but not the SFTP.

And there is almost no documentation for the server part.
(the SSH and SFTP client are working ok)

So i did this kind of workaround to have both in the same GS wrapper.

I did this because i was not sure if i can combine calls of two different wrapper in GS.

For example if i generate a sshSession (aCPointer) from LibsshServerWrapper (server.h) then Can i use it (aCPointer) as argument to call a function in LibsshSftpWrapper?

regards,
bruno

On 26/11/2020 14:42, Martin McClure wrote:

Hi Bruno, James,

libssh.so is one of the libraries that defines many functions, and the prototypes for those functions are spread over multiple header files.

When I personally deal with such libraries in GemStone I tend to make one class per header file, since that seems a bit cleaner to me. The documentation for each function says which header to include, and that tells me which one of my classes to use.

However, this is a matter of taste. Certainly it should work to include libssh.h and sftp.h into the same class, since the functions defined in both are for the same shared library. I don't know about server.h -- in my system the only headers with that name are unrelated to libssh.so.

Regards,
-Martin

On 11/26/20 9:13 AM, James Foster via Glass wrote:

Bruno,

Header files describe what is in a library. The “libssh.h” header describes what is in the “libssh” library and the “sftp.h” header describes what is in the “sftp” library. While you can create a new header file that (wrongly) claims that function A is in library B, that doesn’t mean that when we go to look for A in B that it will be found.

Just because I put down Casa Rosada as your address on my contact list doesn’t make it so. I might mail a letter, but you won’t get it. Just because you pretend that a function is in a library (making up a new header) doesn’t make it so. You may be able to create the Smalltalk wrapper, but when you call it the function won’t be found because it isn’t in the library.

The header file is simply a text file that describes what is in a binary file. Changing the text file doesn’t change the binary file.

James

On Nov 26, 2020, at 9:01 AM, bruno buzzi brassesco <[hidden email]> wrote:

James,

The new header files does not have any function declaration only "#includes" sentences.

Actually the entire new header files is:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

So i do not see how a lookup will fail. But again maybe I'm missing something :)

regards,
bruno

On 26/11/2020 13:21, [hidden email] wrote:

Bruno,

If it works, then it may be fine, but I’ll be somewhat surprised if that happens. My vague idea of the way a program interacts with a shared library is as follows: (1) you ask the OS to load a library (such as '/usr/lib64/libssh.so.4.4.0’ and the OS give you back a ‘handle’ to that library; (2) you ask the OS to look up a function in the library (you pass a string with the function name and you pass in the previously returned handle) and the OS give you back a pointer to a function in that library; (3) you use the pointer to call the function. These are the steps that GemStone handles; you will have previously created an object that encapsulates the function name, parameter types, and return types.

If you use the wrong header to generate a list of expected entry points in a library, then the Smalltalk code will still be generated and you can still load the library (step 1), but when you attempt to lookup a function from library A in library B, the OS will fail at step 2 since the function does not exist. I think that if you want to call functions in two libraries, then you need to load both libraries (step 1) and do the name lookup in the proper library (step 2). 

But my expertise in this area is several years old and was on the Smalltalk side not on the VM side.

James

On Nov 26, 2020, at 4:42 AM, bruno buzzi brassesco via Glass <[hidden email]> wrote:

James,

It seems that is possible to create a new header file and includes 2 subsystems in the same GS Library.

For example sftp_server_all.h:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

Then:
| header wrapperClass wrapper |
header := CHeader path: 'sftp_server_all.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

Do you see any problem with this approach ?

The new library generated all functions now i going to check if they work.
It should work i think ... (but again not a C expert -dealing with C again after 15 years-)



regards,
Bruno

On 9/10/2020 14:53, James Foster via Glass wrote:

Bruno, 

My understanding is that the header parser follows all the #include directives, so will find everything that could be called from C code, including abs() in libc. But the process of calling a dynamic library at runtime involves doing a name lookup of entry points *for that library,* not for anything visible *to* the library, so libssh does not have an abs() entry point. For that you would need a reference to libc. 

The wrapper generator does not know what is actually in the given library, just what functions are defined by all included header files. So you have to do some selection on your own. One wrapper API has a filter as an argument, and so for a library like GemStone C Interface where all the functions begin with ‘gci’ we can filter to only generate wrappers for those functions (it is unlikely that libc has gci*() functions!).

So, yes, you need to have a different library object for each library so the system can do name lookups properly.

James

On Oct 9, 2020, at 5:40 AM, Bruno Buzzi Brassesco via Glass <[hidden email]> wrote:

Hi,

Maybe i'm wrong in some concepts here (not a C expert) but ...

If a C library (libssh) has a subsystem (sftp) in order to create a GS wrapper i can do:

| header wrapperClass wrapper |
header := CHeader path: '/usr/include/libssh/sftp.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

In this case I got all the functions of Libssh and also SFTP, so far so good.

But what happens when there is more than one C subsystem?

How can I generate only one GS wrapper for the original C library (libssh) and all its subsystems ?

Or I will have to create one wrapper per subsystem ?

regards,

bruno

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

Martin,

Thank you for the clarification !

regards,
bruno

Hi Bruno,

AFAIK, if two different wrapper classes load the same shared library, that shared library will only be loaded once, and both wrappers will be using the same handle for the library. A CPointer obtained through one wrapper class should be portable to be used through the other wrapper class.

Regards,
-Martin

On 11/26/20 10:01 AM, bruno buzzi brassesco wrote:

Martin/James,

Thank for your answers.

We are implementing a SSH Server (with SFTP also). The sftp.h header implement all related stuff to SFTP but if you load:
CHeader path: '/usr/include/libssh/sftp.h'.

Then you have the SFTP stuff but not the server capabilities. And if you load server.h header:
CHeader path: '/usr/include/libssh/server.h'.

Then you have the Server stuff but not the SFTP.

And there is almost no documentation for the server part.
(the SSH and SFTP client are working ok)

So i did this kind of workaround to have both in the same GS wrapper.

I did this because i was not sure if i can combine calls of two different wrapper in GS.

For example if i generate a sshSession (aCPointer) from LibsshServerWrapper (server.h) then Can i use it (aCPointer) as argument to call a function in LibsshSftpWrapper?

regards,
bruno

On 26/11/2020 14:42, Martin McClure wrote:

Hi Bruno, James,

libssh.so is one of the libraries that defines many functions, and the prototypes for those functions are spread over multiple header files.

When I personally deal with such libraries in GemStone I tend to make one class per header file, since that seems a bit cleaner to me. The documentation for each function says which header to include, and that tells me which one of my classes to use.

However, this is a matter of taste. Certainly it should work to include libssh.h and sftp.h into the same class, since the functions defined in both are for the same shared library. I don't know about server.h -- in my system the only headers with that name are unrelated to libssh.so.

Regards,
-Martin

On 11/26/20 9:13 AM, James Foster via Glass wrote:

Bruno,

Header files describe what is in a library. The “libssh.h” header describes what is in the “libssh” library and the “sftp.h” header describes what is in the “sftp” library. While you can create a new header file that (wrongly) claims that function A is in library B, that doesn’t mean that when we go to look for A in B that it will be found.

Just because I put down Casa Rosada as your address on my contact list doesn’t make it so. I might mail a letter, but you won’t get it. Just because you pretend that a function is in a library (making up a new header) doesn’t make it so. You may be able to create the Smalltalk wrapper, but when you call it the function won’t be found because it isn’t in the library.

The header file is simply a text file that describes what is in a binary file. Changing the text file doesn’t change the binary file.

James

On Nov 26, 2020, at 9:01 AM, bruno buzzi brassesco <[hidden email]> wrote:

James,

The new header files does not have any function declaration only "#includes" sentences.

Actually the entire new header files is:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

So i do not see how a lookup will fail. But again maybe I'm missing something :)

regards,
bruno

On 26/11/2020 13:21, [hidden email] wrote:

Bruno,

If it works, then it may be fine, but I’ll be somewhat surprised if that happens. My vague idea of the way a program interacts with a shared library is as follows: (1) you ask the OS to load a library (such as '/usr/lib64/libssh.so.4.4.0’ and the OS give you back a ‘handle’ to that library; (2) you ask the OS to look up a function in the library (you pass a string with the function name and you pass in the previously returned handle) and the OS give you back a pointer to a function in that library; (3) you use the pointer to call the function. These are the steps that GemStone handles; you will have previously created an object that encapsulates the function name, parameter types, and return types.

If you use the wrong header to generate a list of expected entry points in a library, then the Smalltalk code will still be generated and you can still load the library (step 1), but when you attempt to lookup a function from library A in library B, the OS will fail at step 2 since the function does not exist. I think that if you want to call functions in two libraries, then you need to load both libraries (step 1) and do the name lookup in the proper library (step 2). 

But my expertise in this area is several years old and was on the Smalltalk side not on the VM side.

James

On Nov 26, 2020, at 4:42 AM, bruno buzzi brassesco via Glass <[hidden email]> wrote:

James,

It seems that is possible to create a new header file and includes 2 subsystems in the same GS Library.

For example sftp_server_all.h:
#include "libssh.h"
#include "server.h"
#include "sftp.h"

Then:
| header wrapperClass wrapper |
header := CHeader path: 'sftp_server_all.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

Do you see any problem with this approach ?

The new library generated all functions now i going to check if they work.
It should work i think ... (but again not a C expert -dealing with C again after 15 years-)



regards,
Bruno

On 9/10/2020 14:53, James Foster via Glass wrote:

Bruno, 

My understanding is that the header parser follows all the #include directives, so will find everything that could be called from C code, including abs() in libc. But the process of calling a dynamic library at runtime involves doing a name lookup of entry points *for that library,* not for anything visible *to* the library, so libssh does not have an abs() entry point. For that you would need a reference to libc. 

The wrapper generator does not know what is actually in the given library, just what functions are defined by all included header files. So you have to do some selection on your own. One wrapper API has a filter as an argument, and so for a library like GemStone C Interface where all the functions begin with ‘gci’ we can filter to only generate wrappers for those functions (it is unlikely that libc has gci*() functions!).

So, yes, you need to have a different library object for each library so the system can do name lookups properly.

James

On Oct 9, 2020, at 5:40 AM, Bruno Buzzi Brassesco via Glass <[hidden email]> wrote:

Hi,

Maybe i'm wrong in some concepts here (not a C expert) but ...

If a C library (libssh) has a subsystem (sftp) in order to create a GS wrapper i can do:

| header wrapperClass wrapper |
header := CHeader path: '/usr/include/libssh/sftp.h'.
wrapperClass := header wrapperForLibraryAt:
'/usr/lib64/libssh.so.4.4.0'.
wrapperClass initializeFunctions.
UserGlobals at: wrapperClass name put: wrapperClass.

In this case I got all the functions of Libssh and also SFTP, so far so good.

But what happens when there is more than one C subsystem?

How can I generate only one GS wrapper for the original C library (libssh) and all its subsystems ?

Or I will have to create one wrapper per subsystem ?

regards,

bruno

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass