Calling DLL

I have created a DLL using Smalltalk MT. This DLL only has 1 function
call that sends a string to an IP address/port combo. For instance, I
use it to send 'GET / HTTP/1.1\n\n' to port 80 of my machine.

The DLL tested OK using Smalltalk MT. However, when I test it using
Dolphin 4 Professional, it hangs Dolphin.

The DLL takes the following parameters.
command lpstr
host lpstr
port byte
result VARIANT*

The DLL executes and puts the return in result. When testing from
Dolphin, I can use netstat to see that the socket to port 80 is open on
my machine. Thus, I know that the DLL is executing.

How do I go about debugging this? Thanks for any help.

Patrick Ma



Sent via Deja.com
http://www.deja.com/

Patrick,

The simplest reason for Dolphin to hang during an external call is that the
function does not return.  If that's the problem, you can overlap the call -
that won't necessarily fix the problem, but, it would bolster the
not-yet-returned theory.  Depending on what you are doing and how, your code
might require the message loop to be functioning, and it's possible that
overlapping the call will help.  It might help to see how you called it in
MT and in Dolphin.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

In article <92doan$4ln92$[hidden email]>,
  "Bill Schwab" <[hidden email]> wrote:
> The simplest reason for Dolphin to hang during an external call is
that the
> function does not return.  If that's the problem, you can overlap the
call -
> that won't necessarily fix the problem, but, it would bolster the
> not-yet-returned theory.  Depending on what you are doing and how,
your code
> might require the message loop to be functioning, and it's possible
that
> overlapping the call will help.  It might help to see how you called
it in
> MT and in Dolphin.

Bill,

Indeed, Dolphin is hanging waiting for the DLL to return. After posting
my initial message, I spent time narrowing down where in the DLL is not
returning. The DLL makes the socket connection OK. It also sends the
string OK. It is hanging while receiving data from the socket.

I replaced the actual receiving code in the DLL with hardcoded strings
and the DLL executes and returns successfully. I got the hardcoded
return in the VARIANT* I am passing into the call as a parameter.

Now, the question is why the DLL works within MT but not Dolphin. The
DLL is generated by MT. Perhaps something is not set correctly while
generating the DLL in MT?

I don't have my exact calling code in MT and Dolphin with me right now
to post it here. However, since I changed the data receiving code in
the DLL to return hardcoded strings and it works in Dolphin, I would
think the calling in Dolphin is fine. Basically, the calling code in
pretty much the same both in MT and Dolphin. The difference is in
getting the handle to the DLL. Once the handle is obtained, the call is
made with 4 parameters( lpstr, lpstr, byte, VARIANT* ) ( sendString,
host, port, returnPointer ).

By the way, what is 'overlapping' and 'might require the message loop
to be functioning'? I am not familiar with these terms.

Thanks.

Patrick Ma


Sent via Deja.com
http://www.deja.com/

Patrick,

> Indeed, Dolphin is hanging waiting for the DLL to return. After posting
> my initial message, I spent time narrowing down where in the DLL is not
> returning. The DLL makes the socket connection OK. It also sends the
> string OK. It is hanging while receiving data from the socket.
>
> I replaced the actual receiving code in the DLL with hardcoded strings
> and the DLL executes and returns successfully. I got the hardcoded
> return in the VARIANT* I am passing into the call as a parameter.

Ok; that suggests a likely problem.


> By the way, what is 'overlapping' and 'might require the message loop
> to be functioning'? I am not familiar with these terms.

First, overlapping: Dolphin 3.0 (IIRC) added the ability to make external
calls on separate OS threads.  Rather than burden every Smalltalk Process
with its own OS thread, Dolphin allows the programmer to explicitly request
OS threads when necessary by specifying the overlap call type.  In 4.0 at
least, you can look at KernelLibrary>>sleep:, or simply browse for methods
containg text (not references to) overlap.  This is a fairly nice
compromise, because Dolphin's lightweight threads allow for quicker context
switching and have less synchronization overhead than OS threads; more
importantly, the entire image does not have to be hardened to calls on other
OS threads.  Sometimes, you might want a call to be overlapped when the base
system defines it as a standard call: you can fix that by adding your own
method to make an overlapped call, so that's not much of a problem.  The
place where this OS threading compromise does become a problem is with COM:
you can't overlap calls to COM interface functions.  Blair has at times
mentioned a way to allow overlapped COM calls, but, I'm not aware of an
promised ship date or version to include the feature, and can't assign one
for OA.  Note that COM handles many threading problems for you; since
Dolphin is a single threaded apartment, COM automatically sequences calls
through the message queue.  Dale Rogerson's book Inside COM is a great place
to start on understanding COM.

There I went and mentioned the message queue.  The message loop pulls
Windows messages (not to be confused with Smalltalk messages) from it and
does some kind of processing on each of them.  If this is unfamiliar
territory for you, Petzold's Programming Windows is a classic, though I
would frankly advise you to get a copy from the Windows 3.x era if you can -
it's shorter reading if not also more clear.  The jump to Win32 is not a big
deal if you understand the basics and have Dolphin to help you.  Dolphin's
GUI framework handles the ugly parts for you, though it allows you to get
involved if necessary.

The reason that the message queue might be relevant to your current problem
is that, if you're using asynchronous sockets (and perhaps in other
situations too??), the socket activity ends up being announced to your
program through its message queue.  If your main GUI thread is blocked on
the DLL call, you won't be able to process messages from your message queue,
and therefore won't "notice" the socket data that's available for reading.

Overlapping the call to your DLL function might fix the problem, as could
using the Dolphin sockets connection, or perhaps the Microsoft
URLMonLibrary, much of which is already exposed in Dolphin.  For an example,
look at LiveUpdate.

Depending on how you (or MT on its own) made the call in MT, the call might
have already been on a separate OS thread.  I don't know enough about MT to
do more than guess.

One thing to consider is that any time Microsoft (or any vendor) wraps
significant network activity into a single function call, you will find
yourself at the mercy of their timeout settings.  If you want something that
is responsive to your user even when things are not going well on the
network, you will have to roll your own solution; I'd recommend using the
Dolphin Sockets Connection, especially since 4.0 overlaps connect calls.
While I wish the situation were otherwise, please think very carefully
before using DCOM.  Locally COM is great and I recommend it highly, but, to
cross machine boundaries, you'll have less trouble if you use sockets.

BTW, my experience has been that CORBA ORBs are pretty good about turning
error conditions back to the application in a timely manner; DCOM tries too
hard to fix things on its own, often with a frustrated and angry user as a
result.  As an example: did the connection fail because the machine is not
responding quickly, because the network is down or heavily loaded, or
because the user typed the wrong address?  The user often knows even as they
are trying to stop themselves from pressing the enter key<g> - the user's
self-directed anger quickly gets focused elsewhere when the computer will
not respond to an attempt to fix the mistake.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Patrick,

> Now, the question is why the DLL works within MT but not Dolphin. The
> DLL is generated by MT. Perhaps something is not set correctly while
> generating the DLL in MT?

I believe that Smalltalk MT runs one OS thread per Smalltalk Process
(MT=Multi-Threading). This means that if you call an external function that
blocks the current OS thread then it will have the expected effect of
blocking the equivalent Process on the Smalltalk side. All the other
Smalltalk Processes, including the UI, will continue to operate. This is why
your blocking call works ok in Smalltalk MT.

In Dolphin the situation is different. Dolphin runs many Smalltalk Processes
(instances of the Process class) within a single OS thread. Therefor if you
make a normal call to a blocking DLL call you will hold up all of the
processes on the Smalltalk side. Fortunately, Dolphin offers a solution to
this (in all versions 3.0 and higher) which is to make the DLL call
"overlapped". You do this by adding the "overlap" keyword after the < brace
in the function definition method. For an example see:
CRTLibrary>>_spawnvp:cmdname:argv:.

What this does is to start a new OS thread each time the DLL function is
called which, when blocked, does not hold up the entire Dolphin process
system. I suspect that addng "overlap" to your DLL method declaration will
address the "hang" that you have been seeing.

Best regards,

Andy Bower
Dolphin Support
http://www.object-arts.com

---
Visit the Dolphin Smalltalk Wiki Web
http://www.object-arts.com/wiki/html/Dolphin/FrontPage.htm
---

"Andy Bower" <[hidden email]> wrote:


Is this also true when loading up COM objects from a thread? Do you
have to do something special in the same manner? I have a problem
where a Process waits for the COM object to finish before it will
continue.  Is there a difference  (functionality wise) between Windows
98 or NT?

Thanks

Costas

Andy/Bill,

Not knowing the existence of 'overlap', I did try to enclose the DLL
calling code in a BlockingCallMonitor. Of course, it did not help so I
posted to this group for help. After reading your replies, I added
the 'overlap' keyword to the DLL call and evaluate the code in a
workspace. I was expecting it to work but to my surprise, it still
hangs Dolphin.

Thinking more about it, I think the hand makes sense. I am evaluating
the code from a workspace ( I suppose it is working off the Main
process ). Now, even when the DLL call is overlapped, the execution
path is Main --> OS thread. Since the OS thread ( running the
overlapped DLL call ) is blocking and seems to never return, Dolphin is
hung by it.

On the other hand, if I wrap the overlapped DLL call in a
BlockingCallMonitor, things become normal; Dolphin is not hung but my
DLL call still does not return. Looking at the Process Monitor ( great
tool ), I can see the BlockingCallMonitor waiting but Main is running.
Therefore, Dolphin is free.

In sum, it seems to be true that when making a blocking DLL call from
Dolphin, in order to prevent Dolphin from hanging, one must wrap the
blocking DLL call in a BlockingCallMonitor.

Something interesting just happen while I am typing this message. The
BlockingCallMonitor process just became 'dead' instead of 'waiting'.
Either the DLL returned or something timed it out. Any guess?

Andy, since Dolphin can do 'overlap' DLL calls, why isn't
BlockingCallMonitor spwaning off native threads?

Now, I still have to figure out why the DLL is 'locked up' when calling
from Dolphin but not MT now that the threading issue/difference between
Dolphin and MT is resolved.

Thanks.

Patrick Ma


Sent via Deja.com
http://www.deja.com/

Bill,

The situation is that I need to write a DLL to be called from
PowerBuilder. Instead of writing the DLL in C, I wanted to try
Smalltalk MT. I am using Dolphin to test the DLL since a DLL is a DLL
no matter what is interfacing with it.

I now understand the threading issue/difference between Dolphin and MT.
The Dolphin hang was my lack of knowledge of how Dolphin handles
Smalltalk Process and OS Threads. Now that the subject is cleared,
Dolphin does not hang anymore. ( See my reply to Andy for more detail ).

I am still stucked, though, in regard to why the 'data receiving code'
in the DLL hangs while being called from Dolphin but not MT. I don't
believe it is a threading issue anymore. Any ideas?

Ultimately, this DLL would be called from PowerBuilder. I wonder what
surprises it is waiting for me.

Regards,

Patrick Ma


Sent via Deja.com
http://www.deja.com/

Patrick

You wrote in message news:92fms6$drh$[hidden email]...
> ...
> In sum, it seems to be true that when making a blocking DLL call from
> Dolphin, in order to prevent Dolphin from hanging, one must wrap the
> blocking DLL call in a BlockingCallMonitor.

No, you just need to perform it on a Process other than the main UI process.
You could use "[blocking call here] fork" for example, which is essentially
what BlockingCallMonitor does. Dolphin hasn't "hung" either, the GUI is just
blocked, any other Dolphin processes will continue to execute while the GUI
process waits for the overlapped call to return.

> Something interesting just happen while I am typing this message. The
> BlockingCallMonitor process just became 'dead' instead of 'waiting'.
> Either the DLL returned or something timed it out. Any guess?

That would suggest that the DLL call returned and the monitor process
completed. Overlapped calls do not "time out" as such, since in the general
case they could take an arbitrarily long time to complete. It is possible to
explicitly terminate a Process waiting for an overlapped call to complete,
however, but since you haven't done that one must assume that the call did
actually return. Once the process completes, its status will be listed as
"dead", and because the Process continues to be referenced by the
BlockingCallMonitor, it will not be garbage collected and will continue to
appear in the process list.

>
> Andy, since Dolphin can do 'overlap' DLL calls, why isn't
> BlockingCallMonitor spwaning off native threads?

BlockingCallMonitor is part of the "Sockets Connection" package, and is
intended to support certain idioms described in the documentation for that
package (Evaluate 'SmalltalkSystem help: 'tcp.ipsocketsconnectivity' when
connected to the net or preferably with the Education Centre installed since
the latter appears to be the more recent publication). It was designed
before overlapped calls were introduced, and really has no intended
relationship to overlapped calls at all. Having said that, when you use
BlockingCallMonitor to wrap an overlapped call then the overall effect is
that it is spawning off native threads (the threads are pooled by the way,
as it would be terribly inefficient to actually start a new thread for each
and every external DLL call) to make the calls.

>
> Now, I still have to figure out why the DLL is 'locked up' when calling
> from Dolphin but not MT now that the threading issue/difference between
> Dolphin and MT is resolved.

Could it be an initialization issue? The MT environment may have made some
other calls which cause it to behave differently.

Why not just import the MT code and modify it to perform the operation
directly from Dolphin. This will allow you to debug through it and see what
is going on. Alternatively you could use build the MT DLL with debug
information and then use a native debugger, such as Visual Studio's, to
debug Dolphin.exe, and set a breakpoint in the MT DLL. I've never tried this
but from the MT marketing blurb it should work fine.

Regards

Blair

> No, you just need to perform it on a Process other than the main UI
process.
> You could use "[blocking call here] fork" for example, which is
essentially
> what BlockingCallMonitor does. Dolphin hasn't "hung" either, the GUI
is just
> blocked, any other Dolphin processes will continue to execute while
the GUI
> process waits for the overlapped call to return.

Of course, fork! I have been doing Java for too long ( 3 years now ) :-)

> That would suggest that the DLL call returned and the monitor process
> completed. Overlapped calls do not "time out" as such, since in the
general
> case they could take an arbitrarily long time to complete. It is
possible to
> explicitly terminate a Process waiting for an overlapped call to
complete,
> however, but since you haven't done that one must assume that the
call did
> actually return. Once the process completes, its status will be
listed as
> "dead", and because the Process continues to be referenced by the
> BlockingCallMonitor, it will not be garbage collected and will
continue to
> appear in the process list.

Why wouldn't the BlockingCallMonitor instance be collected? Who is
holding onto it? I would have assumed that once a BlockingCallMonitor
runs out of scope ( completion or error ), it would be waiting to be
collected.

> BlockingCallMonitor is part of the "Sockets Connection" package, and
is
> intended to support certain idioms described in the documentation for
that
> package (Evaluate 'SmalltalkSystem help: 'tcp.ipsocketsconnectivity'
when
> connected to the net or preferably with the Education Centre
installed since
> the latter appears to be the more recent publication). It was designed
> before overlapped calls were introduced, and really has no intended
> relationship to overlapped calls at all. Having said that, when you
use
> BlockingCallMonitor to wrap an overlapped call then the overall
effect is
> that it is spawning off native threads (the threads are pooled by the
way,
> as it would be terribly inefficient to actually start a new thread
for each
> and every external DLL call) to make the calls.

Pooled? Is it a pre-defined pool or grow as it go? Just curious.

> Could it be an initialization issue? The MT environment may have made
some
> other calls which cause it to behave differently.

I thought of that and have been hunting around the MT documentation to
discover the various aspects of generating a DLL in MT. People said
Dolphin lacks documentation. Ha! wait until they use MT.

> Why not just import the MT code and modify it to perform the operation
> directly from Dolphin. This will allow you to debug through it and

You must have not coded in MT :-) As much as MT is Smalltalk, it is
pretty bare metal, IMHO. By the way, I am not saying anything bad about
MT. I like Smalltalk. For example, the Socket class in MT requires you
to know struct SOCKADDR_IN to make connection to an IP address/port
combo. The level of abstraction in MT is far away from Dolphin. Dolphin
has SocketAbstract, Socket and ServerSocket. That is a nice abstraction
and clear for users to utilize sockets in client/server mode. In MT,
there is 1 class called Socket. It requires some studying to utilize
this class in normal application programming. Anyway, my point is the
code is not a simple import. It is more like a rewrite and I am sure a
rewrite of the function will work in Dolphin since I have done a mini
client/server socket framework using Dolphin already.

> is going on. Alternatively you could use build the MT DLL with debug
> information and then use a native debugger, such as Visual Studio's,
to
> debug Dolphin.exe, and set a breakpoint in the MT DLL. I've never
tried this
> but from the MT marketing blurb it should work fine.

That is the fun of Windows programming :-( Well, I have put a message
to ObjectConnect's support and see what they say.

Thanks to everyone for helping out.

Regards,

Patrick Ma


Sent via Deja.com
http://www.deja.com/

Patrick,

One more thought on your problem: you might want to look for a sockets
programming book by Pat Bonner.  The copy I have includes lots of screen
shots from Win 3.1, so it's dated, but, it includes a chapter on using
sockets from a DLL.  Obviously there are some issues (no doubt the ones
we've considered here and more).

Happy networking!

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

In article <92ghlt$76js2$[hidden email]>,
  "Bill Schwab" <[hidden email]> wrote:
> Patrick,
>
> One more thought on your problem: you might want to look for a sockets
> programming book by Pat Bonner.  The copy I have includes lots of
screen
> shots from Win 3.1, so it's dated, but, it includes a chapter on using
> sockets from a DLL.  Obviously there are some issues (no doubt the
ones
> we've considered here and more).
>
> Happy networking!
>

This *could be* embrassing because I don't have the code to test the
theroy. This morning I woke up thinking my claim of the "receiving
data" portion of the DLL is hanging might not be absolute. I know the
connection is made by using netstat. But, how could I claim the "send
command" was successful. It is correct that the line of code that does
the "send command" executed. But, was it successful?

Looking at the command I sent, it is a simple String ending with '\n'.
Again, after doing Java for 3 years, that is a natural line delimiter
and it never occur to me that is not true in Dolphin. On the other
hand, MT '\n' as the line delimiter.

Without the String ending with a line delimiter, the server basically
does not know the command sent was done. Therefore, there is nothing to
return. In turn, it makes the DLL wait ( not hung ).

I really think this is what is happening and I will test it when I get
back to work in a few days. Sorry for the wild goose chase but I did
learn more about calling DLLs in Dolphin.

Thanks for everyone's help.

Regards,

Patrick Ma


Sent via Deja.com
http://www.deja.com/

Blair and Andy,

<from Bill's posting>
<snip>

>The
>place where this OS threading compromise does become a problem is with COM:
>you can't overlap calls to COM interface functions.  Blair has at times
>mentioned a way to allow overlapped COM calls, but, I'm not aware of an
>promised ship date or version to include the feature, and can't assign one
>for OA.  Note that COM handles many threading problems for you; since
>Dolphin is a single threaded apartment, COM automatically sequences calls
>through the message queue.  Dale Rogerson's book Inside COM is a great place
>to start on understanding COM.

This is a big problem because many users will want to take advantage
of numerous external components.

What exactly is the difficulty with this?
Why can't it be resolved?
Is this something that I can fix or does it need changes to the VM?

Here is an example of what I have done:

I have a COM object that I invoke. I tested the object by  running an
application twice and executing the component at the same time from
each one. Each one finishes when its supposed to without halting the
other.

But with Dolphin 4.0  I have a problem. Here is what I tried

o1 := IDispatch createObject: 'data.server'
o2 := IDispatch createObject: 'data.server'

"Wait 20 seconds"
f1:=[o1 invoke: 'waits' with: 20] fork.
"Wait 10 seconds"
f2:=[o2 invoke: 'waits' with: 10] fork.

The first fork blocks and execution to the next statement will not
continue until it terminates. I understand that its because it runs in
a D4.0 VM thread.  Is there a way to change this to run on a native
Windows thread?

I even called Microsoft and asked them to check it. They did and said
the component works multithreaded and sent me the code the code they
used:

Below is the code they used to test the .DLL I sent them

#define _WIN32_DCOM 1
#include <afxole.h>
#include <rpcdce.h>
#include <stdio.h>
#include <tchar.h>
#include <objbase.h>
#include <KS.h>
#include <process.h>

#import "E:\\ashishp\\imtiazk\\fox.dll" no_namespace

#define NUM_THREADS 10

unsigned int __stdcall funcfoo(void *)
{

        CLSID wclsid;
        Isrv* punk;
        IUnknown* punk2;
        HRESULT hr;

        printf("inside thread\n");
        hr = CLSIDFromProgID(L"fox.srv", &wclsid);
        hr=CoCreateInstance(wclsid,NULL,
CLSCTX_INPROC_SERVER,IID_IUnknown,(void **) &punk2);

        hr = punk2->QueryInterface(__uuidof(Isrv),(void **) &punk);
        VARIANT seconds;
        VariantInit(&seconds);
        seconds.vt= VT_I2;
        seconds.iVal=10;
        punk->waits(&seconds);
        printf("end thread\n");
        _endthread();
        return 0;
}

int main()
{
       
        // CoInitialize(NULL);
// unsigned long handle[4];
        HRESULT hr = 0; // for testing
        hr = CoInitializeEx( NULL, COINIT_MULTITHREADED );

        //IUnknown* punk;

// _Isrv testisrv;
       

// IsrvPtr testIsrv;
// hr = testIsrv.CreateInstance(__uuidof(srv));
// testIsrv->waits(


// Isrv * x;
// x = _uuidof(Isrv);

        HANDLE hThread[NUM_THREADS];
        unsigned int threadID[NUM_THREADS];

        for(int i=0;i<NUM_THREADS;i++)
        {
                printf("thread %d\n",i);
                hThread[i] = (HANDLE)_beginthreadex(NULL, 0, &funcfoo,
NULL, 0,
           &threadID[i]);
        }

       
        WaitForMultipleObjects(
    NUM_THREADS,           // number of objects in array
    hThread,     // array of objects
    TRUE,       // wait for any
    INFINITE);   // indefinite wait

                return 0;
}

/*
        CLSID wclsid;
        hr = CLSIDFromProgID(L"fox.srv", &wclsid);
        hr=CoCreateInstance(wclsid,NULL,
CLSCTX_INPROC_SERVER,IID_IUnknown,(void **) &punk2);

        hr = punk2->QueryInterface(__uuidof(Isrv),(void **) &punk);
        VARIANT seconds;
        VariantInit(&seconds);
        seconds.vt= VT_I2;
        seconds.iVal=10;
        punk->waits(&seconds);
*/

               
// hr=CoCreateInstance(wclsid,NULL,
CLSCTX_INPROC_SERVER,IID_Isrv,(void **) &punk);
// _beginthreadex(funcfoo
                //Wait for all thread to finish.
// dwRet = WaitForMultipleObjects(4, (const HANDLE *)
handle, TRUE, INFINITE);
// _beginthread( Bounce, 0, (void *) (ch++)  );

Happy New Year everyone,

Just to close my part on this thread, the problem is resolved. Indeed,
it is the difference of how line delimiter is expressed in MT and
Dolphin.

Regards,

Patrick Ma


Sent via Deja.com
http://www.deja.com/