Broadcasting over a local network ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Broadcasting over a local network ?

FU Berlin-2
Hi,
I want to inform other users of the local network about changes on the
database. I could collect the IP-Addresses of each user by sending
individually messages but I thought that it must be simplier by
broadcasting. I spent a lot of time on it but, it is still darkness
all around.
All these Socket's stuff is very interesting but also very complicated
( for me ).
Does anybody have experiences on this item ?
Regards
Jon


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Jerome Chan
In article <[hidden email]>,
 [hidden email] (jon) wrote:

> Hi,
> I want to inform other users of the local network about changes on the
> database. I could collect the IP-Addresses of each user by sending
> individually messages but I thought that it must be simplier by
> broadcasting. I spent a lot of time on it but, it is still darkness
> all around.
> All these Socket's stuff is very interesting but also very complicated
> ( for me ).
> Does anybody have experiences on this item ?
> Regards
> Jon

Have you tried looking at remote Smalltalk?
http://www.smalltalking.net/Goodies/Dolphin/


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Chris Uppal-3
In reply to this post by FU Berlin-2
jon wrote:

> I want to inform other users of the local network about changes on the
> database. I could collect the IP-Addresses of each user by sending
> individually messages but I thought that it must be simplier by
> broadcasting. I spent a lot of time on it but, it is still darkness
> all around.

If you want to broadcast the messages reliably, then I think you'll probably
find it easier to hold a TCP/IP session open to each client and send the
notifications to each as a separate message.

If you are happy with potential for missed notifications then you could use
UDP/IP (which sends single "datagrams" rather than holding a reliable
connection open).  Dmitry Zamotkin posted an extension to allow the Dolphin
sockets stuff to talk UDP; it's in a post dated 2001/9/14, titled "UPD Socket"
and you should be able to find it in Ian's archive or possibly via Google.

Your needs might best be met by IP mutlicasting, but I know almost nothing
about that, and nothing whatever about how to do it with Dolphin's networking
support.

So, if *I* were doing this, I'd use a central server, the "database", and each
client that was interested in observing updates would connect to that server.
The server would send a separate message to each connected client whenever the
database changed.  If you need more guidance on the ways you can set up such a
server, and how to connect to it, then do ask again.

    -- chris


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Bill Dargel
Chris Uppal wrote:

> So, if *I* were doing this, I'd use a central server, the "database", and each
> client that was interested in observing updates would connect to that server.
> The server would send a separate message to each connected client whenever the
> database changed.  If you need more guidance on the ways you can set up such a
> server, and how to connect to it, then do ask again.

Chris,

I'm working on an application that want's to do something along this line. I'm a
bit of a database novice, and so aren't aware of any features might support this
directly. I was thinking of using a table in the database as sort of a change log
/ notification queue. Each connected client would need to periodically poll that
table with a query. It sounds like maybe you have something else in mind? I'm
using MySQL through odbc at the moment, if that's relevant.

thanks,
-Bill

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Bill Schwab-2
Bill,

> Chris Uppal wrote:
>
> > So, if *I* were doing this, I'd use a central server, the "database",
and each
> > client that was interested in observing updates would connect to that
server.
> > The server would send a separate message to each connected client
whenever the
> > database changed.  If you need more guidance on the ways you can set up
such a
> > server, and how to connect to it, then do ask again.
>
> I'm working on an application that want's to do something along this line.
I'm a
> bit of a database novice, and so aren't aware of any features might
support this
> directly. I was thinking of using a table in the database as sort of a
change log
> / notification queue. Each connected client would need to periodically
poll that
> table with a query. It sounds like maybe you have something else in mind?

There are people who will insist on doing this kind of thing using database
features.  Borland had it working particularly well quite some time ago.  I
think you'll find that Chris is recommending that you write software that
has clients take a big gulp at the beginning, then notifications to keep
them up to date based on messages/triggers from the server.  The server
would be the only thing that actually touches the database.  FWIW, I agree
with that recommendation.

The most difficult thing about sockets programming is error handling, and
since you will almost certainly want to use multiple threads to make it
work, synchronization issues are a close second.


> I'm
> using MySQL through odbc at the moment, if that's relevant.

Good choice, I think.  I've been using MySQL and MyODBC on test machines,
and one entering-production box.  I tried converting my cash cow, and it
stopped giving milk - in fact, it contracted mad computer disease :)  It's
back with an Access database and ODBC until I can figure out what went
wrong.  I'm too busy right now to track it down, but I'm suspicious that I
simply have it configured incorrectly.  Some (but not all) database access
caused the mysqld-nt service to jump to 99% (and then some I think) CPU
utilization.  Please let me know if you've ever heard of a similar
complaint.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Bill Dargel
Bill Schwab wrote:

> There are people who will insist on doing this kind of thing using database
> features.  Borland had it working particularly well quite some time ago.  I
> think you'll find that Chris is recommending that you write software that
> has clients take a big gulp at the beginning, then notifications to keep
> them up to date based on messages/triggers from the server.  The server
> would be the only thing that actually touches the database.  FWIW, I agree
> with that recommendation.

This sounds somewhat like what I'd like to evolve to. Right now (as far as the
application is concerned) only MySQL is running on the server, with a Dolphin
app on the client connected to it over the Internet. I'd like to migrate most of
the application to the server, leaving only the top UI layer running in the
Dolphin app on the client. But that looks like an involved undertaking, so I'm
also looking at how to use just the database for change notifications, as a
stopgap measure.

In the long run, I still like using Dolphin for the clients. Being able to
produce nice looking applications, the ease of interfacing to Windows functions
when needed, reasonably sized downloads, etc. It's all good. For the server side
of things, it's more of a conundrum. I _really_ don't want to change the server
from Linux to Windows, so that leaves Dolphin out of the picture. So I'm
considering VisualWorks and/or GemStone for the server. Though working in
multiple environments is always more of a challenge. Course that keeps things
from getting boring. :-)regards,
-Bill

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA


Reply | Threaded
Open this post in threaded view
|

MySQL CPU hogging (Re: Broadcasting over a local network)

Bill Dargel
In reply to this post by Bill Schwab-2
Bill Schwab wrote:

> Some (but not all) database access
> caused the mysqld-nt service to jump to 99% (and then some I think) CPU
> utilization.  Please let me know if you've ever heard of a similar
> complaint.

Let me pass one on first-hand! What I found caused the 99%+ CPU utilization was
triggered by doing a "drop table". As long as I avoided that, I didn't have a
problem. It was coming up quite a bit in putting together SUnit tests on the DB.
:-(  I resorted to dropping and re-creating the entire test database rather that
dropping tables in a existing one. That at least minimized the occurrences of
the problem to real database schema changes.

An example sequence from the the mysql.err file might go like this:
    030128 20:26:01  InnoDB: Warning: MySQL is trying to drop table
test/#sql2-3fc-224
    InnoDB: though there are still open handles to it.
    InnoDB: Adding the table to the background drop queue.
    030128 20:40:46  MySql: Normal shutdown
    030128 20:40:46  InnoDB: Starting shutdown...
    030128 20:40:47  InnoDB: Dropped table test/#sql2-3fc-224 in background drop
queue.
    030128 20:40:50  InnoDB: Shutdown completed
    030128 20:40:50  MySql: Shutdown Complete

Once the table was added to the background drop queue, the CPU would peg, and
that would be it. I found that if one exhibited enough patience to wait for
menus to respond and shut down the MySQL server normally, that the drops would
actually be processed during shutdown, as shown in the log above. At one time, I
tried to figure out what it might mean that there were "still open handles" to
it, but didn't have any luck.

This was with MySQL 3.23.55 on a Win2k box (connecting with MyODBC 2.50.39).
I've wondered whether MySQL 4.0 has fixed the problem, but haven't tried it. My
real solution was <drum roll> to switch to running MySQL on a Linux server,
where the issue just doesn't occur.

HTH,
-Bill

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA


Reply | Threaded
Open this post in threaded view
|

Re: MySQL CPU hogging (Re: Broadcasting over a local network)

Bill Schwab-2
Bill,

> Once the table was added to the background drop queue, the CPU would peg,

How did you identify the event?  Is it simply the last thing logged before
the problem occured, or were you able to see it happen?

> and
> that would be it. I found that if one exhibited enough patience to wait
for
> menus to respond and shut down the MySQL server normally, that the drops
would
> actually be processed during shutdown, as shown in the log above.

How long did that take to regain control?  Might it help to have a stop
button on the process in plain view before attempting to reproduce the
problem?


> At one time, I
> tried to figure out what it might mean that there were "still open
handles" to
> it, but didn't have any luck.

Did you report it?  It might make sense to the developers.  One possible
scenario would be that the only thread that can run polls the queue - of
course nothing like that ever happened to any of my code :)  Humor aside, it
might be just the observation they need to find a synchronization bug, so if
you are confident of your findings, I urge you to submit a report to MySQL
AB.


> This was with MySQL 3.23.55 on a Win2k box (connecting with MyODBC
2.50.39).
> I've wondered whether MySQL 4.0 has fixed the problem, but haven't tried
it.

I'm reasonably certain that I have 4.0 on the offending box, but I'll check.

> My
> real solution was <drum roll> to switch to running MySQL on a Linux
server,
> where the issue just doesn't occur.

Are you using ODBC over a network to get to it?  That might not be a bad
solution for us.  The only problem (and I think it is is possible) is that
I'd want the database traffic encrypted.

Still, I'm left with the questions of what clobbered the machine, and why
does it apparently not happen on the test machines?  It's ironic that your
problems centered around schema changes.  Aside from the usual
fast/stable/free benefits, and MySQL's being a network aware server, one of
the things that has pulled me into it is the ability to control the schema
from my code; Access' SQL falls just a little short in a couple of areas.  A
silly example is the ability to re-order fields the way I want; MySQL can do
it, and Access apparently cannot.  One could argue that it shouldn't matter,
but the fact is that it does matter to one of my systems.  I could maintain
a separate field order, but it's much nicer to simply get it from the
database.  Finally, if only thanks to Paul DuBois, is very well documented.

Thanks for the info!!!

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: MySQL CPU hogging (Re: Broadcasting over a local network)

Bill Dargel
Bill Schwab wrote:

> Bill Dargel wrote:
> > Once the table was added to the background drop queue, the CPU would peg,
>
> How did you identify the event?  Is it simply the last thing logged before
> the problem occured, or were you able to see it happen?

The problem with dropping tables was the only issue raised in the mysql.err log
file. (Besides starting and stopping of the server). And I just noticed the
1-to-1 correspondence between that appearing in the error log and the 100% CPU
usage.

> > and
> > that would be it. I found that if one exhibited enough patience to wait
> for
> > menus to respond and shut down the MySQL server normally, that the drops
> would
> > actually be processed during shutdown, as shown in the log above.
>
> How long did that take to regain control?  Might it help to have a stop
> button on the process in plain view before attempting to reproduce the
> problem?

I had WinMySQLAdmin running, so to stop the server I'd click on its icon in the
system tray, wait a fair number of seconds for the menu to appear, go to a
submenu, waiting a fair number of seconds again, and then selecting the stop the
service item, once more waiting a fair number of seconds for it to respond.

I did some more playing with the problem today. Your suggestion was worthwhile.
I had the properties dialog from the control panel showing for MySQL server.
Then it was one simple click on the stop button. Which incidentally took about
20 seconds to shutdown the service, where normally it takes 1 or 2 seconds.

> > At one time, I
> > tried to figure out what it might mean that there were "still open
> handles" to
> > it, but didn't have any luck.
>
> Did you report it?  It might make sense to the developers.  One possible
> scenario would be that the only thread that can run polls the queue - of
> course nothing like that ever happened to any of my code :)  Humor aside, it
> might be just the observation they need to find a synchronization bug, so if
> you are confident of your findings, I urge you to submit a report to MySQL
> AB.

I fired up the Windows MySQL server for the first time in months so that I could
play with the problem again. At first I couldn't seem to reproduce it as simply
as I recalled. I finally worked out that there must be something in the way that
ReStore communicates through MyODBC that rubs the Windows MySQL server the wrong
way. Once I've connected to the db with ReStore and either written something to
a table or done a #synchronizeAllClasses, it's primed to have a problem. I can
disconnect ReStore and even shutdown the Dolphin client application completely.
Of course once I disconnect, the "mysqladmin process" command shows the client
isn't connected at that point. But the server's still ready to blow up. If I
drop one of the tables that had been touched by ReStore, even just by using the
MySQL command line to do it (and not through MyODBC), the problem with needing
to defer the drop gets logged. And then it's about 10 seconds later that the CPU
pegs.

> > This was with MySQL 3.23.55 on a Win2k box (connecting with MyODBC
> 2.50.39).
> > I've wondered whether MySQL 4.0 has fixed the problem, but haven't tried
> it.
>
> I'm reasonably certain that I have 4.0 on the offending box, but I'll check.

Since Dolphin and ReStore aren't exactly standard tools for the MySQL folks, it
would no doubt involve looking into the debug log of MyODBC to see what sequence
of things might set it off. I think I'll wait until I get a chance to install
4.0 and the current MyODBC and see if there's still a problem before reporting
it though.

You're not using ReStore by any chance are you? Because it's doing something
through the MyODBC interface that I haven't been able to reproduce simply using
DBConnection. I'm wondering if you're having the same problem that I ran into,
or perhaps what you're doing is just uncovering the same underlying problem with
the Window's version of MySQL server? As I said before, the same sequence fed to
a Linux MySQL server doesn't cause any problem.

> > My
> > real solution was <drum roll> to switch to running MySQL on a Linux
> server,
> > where the issue just doesn't occur.
>
> Are you using ODBC over a network to get to it?  That might not be a bad
> solution for us.  The only problem (and I think it is is possible) is that
> I'd want the database traffic encrypted.

Yes. Even the Window's MySQL server tended to be on a different machine than the
Dolphin client. Found that it's real simple to get it going just by entering in
the host name, or even a full domain name into the configuration for the ODBC
data source in the control panel. With that it's easy to connect to MySQL
anywhere on the LAN or even the Internet. Though in the latter case connection
speed and probably even more important the latency become issues.

Encrypting the database traffic is something I'd be interested in as well. I
haven't looked into it yet. If you come up with anything, let me know.

regards,
-Bill

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Jerome Chan
In reply to this post by Bill Dargel
In article <[hidden email]>,
 Bill Dargel <[hidden email]> wrote:

> Bill Schwab wrote:
>
> > There are people who will insist on doing this kind of thing using database
> > features.  Borland had it working particularly well quite some time ago.  I
> > think you'll find that Chris is recommending that you write software that
> > has clients take a big gulp at the beginning, then notifications to keep
> > them up to date based on messages/triggers from the server.  The server
> > would be the only thing that actually touches the database.  FWIW, I agree
> > with that recommendation.
>
> This sounds somewhat like what I'd like to evolve to. Right now (as far as
> the
> application is concerned) only MySQL is running on the server, with a Dolphin
> app on the client connected to it over the Internet. I'd like to migrate most
> of
> the application to the server, leaving only the top UI layer running in the
> Dolphin app on the client. But that looks like an involved undertaking, so
> I'm
> also looking at how to use just the database for change notifications, as a
> stopgap measure.
>
> In the long run, I still like using Dolphin for the clients. Being able to
> produce nice looking applications, the ease of interfacing to Windows
> functions
> when needed, reasonably sized downloads, etc. It's all good. For the server
> side
> of things, it's more of a conundrum. I _really_ don't want to change the
> server
> from Linux to Windows, so that leaves Dolphin out of the picture. So I'm
> considering VisualWorks and/or GemStone for the server. Though working in
> multiple environments is always more of a challenge. Course that keeps things
> from getting boring. :-)regards,
> -Bill
>
> -------------------------------------------
> Bill Dargel            [hidden email]
> Shoshana Technologies
> 100 West Joy Road, Ann Arbor, MI 48105  USA
>
>

What about making a server (using Dolphin) that runs on Windows that
talks to the database that runs on linux?


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Chris Uppal-3
In reply to this post by Bill Dargel
Bill Dargel wrote:

> I'm working on an application that want's to do something along this
> line. I'm a bit of a database novice, and so aren't aware of any
> features might support this directly. I was thinking of using a table
> in the database as sort of a change log / notification queue. Each
> connected client would need to periodically poll that table with a
> query. It sounds like maybe you have something else in mind? I'm
> using MySQL through odbc at the moment, if that's relevant.

I'm afraid that I don't have a good solution.  I was assuming that Jon already
had a way of telling when/what updates were made to the "database" (in quotes
since I didn't get the impression that he was necessarily using an
off-the-shelf SQL db), and that the only issue was how to distribute the
notifications.

It may well be that MySQL has the means for clients to Observe (in the sense of
the Observer pattern) changes to the db, but I don't know how to do it.  In any
case, I don't think (though this is *not* something I know much about) that
standard SQL or ODBC provides a way to do it.  If there are such facilities
then you'd probably have to use a direct API to get at them.

<random noodlings>
Without knowing MySQL, it seems that your approach is as viable as anything
that doesn't require direct access to MySQL-specific APIs.  I'd be inclined to
use triggers internally to add rows to the "this has changed" table.  Depending
on the application profile you might find it better for the clients to
broadcast a notification "something has probably changed, check the DB" to each
other (possibly via a central server) when they commit changes to the DB,
rather than polling all the time.  You might even use a hybrid with polling at
low frequency (once a minute, once a day, whatever) supplemented by broadcast
notifications to improve responsiveness.

Come to think of it, you might be able to use server-side Perl/Python/whatever
to hook the changes and broadcast notifications.  If that's possible at all,
then it'd probably be pretty easy.
<random noodlings>

    -- chris


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Bill Dargel
In reply to this post by Jerome Chan
Jerome Chan wrote:

> In article <[hidden email]>,
>  Bill Dargel <[hidden email]> wrote:
> > In the long run, I still like using Dolphin for the clients. Being able to
> > produce nice looking applications, the ease of interfacing to Windows
> > functions
> > when needed, reasonably sized downloads, etc. It's all good. For the server
> > side
> > of things, it's more of a conundrum. I _really_ don't want to change the
> > server
> > from Linux to Windows, so that leaves Dolphin out of the picture. So I'm
> > considering VisualWorks and/or GemStone for the server. Though working in
> > multiple environments is always more of a challenge. Course that keeps things
> > from getting boring. :-)
>
> What about making a server (using Dolphin) that runs on Windows that
> talks to the database that runs on linux?

Certainly something to consider. Still, my confidence in using Windows as a server
platform is nowhere near as strong as that of using Linux. And in that senario, it
would still be acting as a critical server component, even if it wasn't handling
the database. Plus, since we don't have the bandwidth here, seems it would require
a minimum of two machines at a collocation facility, even to get started. In the
long run, it would be nicer to be supporting a more uniform mix of machines on any
(potential) server farm. Thanks for the idea, though.

regards,
-Bill

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA


Reply | Threaded
Open this post in threaded view
|

Re: MySQL CPU hogging (Re: Broadcasting over a local network)

Bill Schwab-2
In reply to this post by Bill Dargel
Bill,

> The problem with dropping tables was the only issue raised in the
mysql.err log
> file. (Besides starting and stopping of the server). And I just noticed
the
> 1-to-1 correspondence between that appearing in the error log and the 100%
CPU
> usage.

Thanks.


> I fired up the Windows MySQL server for the first time in months so that I
could
> play with the problem again. At first I couldn't seem to reproduce it as
simply
> as I recalled. I finally worked out that there must be something in the
way that
> ReStore communicates through MyODBC that rubs the Windows MySQL server the
wrong
> way. Once I've connected to the db with ReStore and either written
something to
> a table

That would just be insert/update kind of stuff, right?


> or done a #synchronizeAllClasses,

That sounds a little more complicated.


>  it's primed to have a problem. I can
> disconnect ReStore and even shutdown the Dolphin client application
completely.

Are you saying that it will peg the CPU even if you shut down
ReStore/Dolphin?  Sorry to parse things so carefully, but _anything_ could
be important.


> Of course once I disconnect, the "mysqladmin process" command shows the
client
> isn't connected at that point. But the server's still ready to blow up. If
I
> drop one of the tables that had been touched by ReStore, even just by
using the
> MySQL command line to do it (and not through MyODBC),

Sounds like it's the server that's hurting.  Though MyODBC still might have
caused it.


> the problem with needing
> to defer the drop gets logged. And then it's about 10 seconds later that
the CPU
> pegs.

Is it queueing the drop because there are handles open?  Could it be that
ReStore, Dolphin or MyODBC failed to close something?


> > > This was with MySQL 3.23.55 on a Win2k box (connecting with MyODBC
> > 2.50.39).
> > > I've wondered whether MySQL 4.0 has fixed the problem, but haven't
tried
> > it.
> >
> > I'm reasonably certain that I have 4.0 on the offending box, but I'll
check.

It is 4.0 - current stable release.


> Since Dolphin and ReStore aren't exactly standard tools for the MySQL
folks, it
> would no doubt involve looking into the debug log of MyODBC to see what
sequence
> of things might set it off. I think I'll wait until I get a chance to
install
> 4.0 and the current MyODBC and see if there's still a problem before
reporting
> it though.
>
> You're not using ReStore by any chance are you?

No, but I am using a home-grown tool that does some similar things (and not
terribly well).


> Because it's doing something
> through the MyODBC interface that I haven't been able to reproduce simply
using
> DBConnection. I'm wondering if you're having the same problem that I ran
into,
> or perhaps what you're doing is just uncovering the same underlying
problem with
> the Window's version of MySQL server? As I said before, the same sequence
fed to
> a Linux MySQL server doesn't cause any problem.

Do you have SQL statements that cause it?  Can you send them to me?  I'll
understand if the answer is no, because mine (so far at least) would all but
have to be to the reverse question: if it happens only with live data,
there's no way I can share it.


> Yes. Even the Window's MySQL server tended to be on a different machine
than the
> Dolphin client. Found that it's real simple to get it going just by
entering in
> the host name, or even a full domain name into the configuration for the
ODBC
> data source in the control panel. With that it's easy to connect to MySQL
> anywhere on the LAN or even the Internet. Though in the latter case
connection
> speed and probably even more important the latency become issues.

True, but if we port one of the socket-based MySQL interfaces to Dolphin,
then we could be doing things on other threads while the database server
(win2k or Linux) is busy working on a query.  Anybody?  It doesn't hurt to
ask, right? :)


> Encrypting the database traffic is something I'd be interested in as well.
I
> haven't looked into it yet. If you come up with anything, let me know.

I think it's considered a secure-shell connection.  Likewise, please post if
you beat me to it.  Legally, I could run that connection clear-text, but I
would be very reluctant to do so.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: MySQL CPU hogging (Re: Broadcasting over a local network)

Bill Dargel
Bill,

I'll gather a number of your points/questions together, and then attempt to
answer them all together, rather than piecemeal. The good news is that I've
managed to learn how to reproduce the problem simply.

Bill Schwab wrote:

> [snip]
> > or done a #synchronizeAllClasses,
> That sounds a little more complicated.
> [snip]
> Are you saying that it will peg the CPU even if you shut down
> ReStore/Dolphin?
> [snip]
> Sounds like it's the server that's hurting.  Though MyODBC still might have
> caused it.

> [snip]

> Is it queueing the drop because there are handles open?  Could it be that
> ReStore, Dolphin or MyODBC failed to close something?
> [snip]
> Do you have SQL statements that cause it?  Can you send them to me?

Okay, here goes.

The ReStore synchronizeAllClasses does sound complicated, and potentially is. It
compares the schema of the tables in the database with the application's object
model and then makes whatever adjustments are needed on the database to get them
in sync, by creating and dropping tables or columns, etc.

But, it turns out that even the simplest synchronize (where it doesn't change
anything, because it was already in sync) could be used to trigger the problem.
With that, I was finally able to reproduce it without using ReStore at all.
Turns out the key culprit in this is doing a DBConnection>>columns:, which shows
up in the MySQL general query log as a "Field List" type of entry.

The following incant will cause the MySQL server to peg the CPU (possibly after
a 5 or 10 second delay). Note that this trashes the "testtemp" database. Change
if necessary.

    (DBConnection new dsn: 'Frank-testtemp')
        exec: 'drop database if exists testtemp';
        exec: 'create database testtemp';
        exec: 'CREATE TABLE testtemp.NEXT_ID (ID_ INTEGER)';
        exec: 'use testtemp';
        columns: 'NEXT_ID';
        exec: 'drop table testtemp.NEXT_ID';
        close.

Let me know what that does, or doesn't do for you. (Or anyone else with a
Windows MySQL Server running that cares to try it).

So it seems to be something that Dolphin and/or MyODBC does with #columns: that
leaves some sort of handle open for the table on the server. The server will
then subsequently deal with the "drop table" by putting it into the background
drop queue. And that apparently puts it into the 100% CPU usage mode.

Note that by pointing the MyODBC connection at a Linux MySQL server rather than
the Windows Server, it handles the above without issue, i.e. no mention of open
handles, no mention of background drop queue, and no pegging of the CPU. So
there does seem to be bug in the Windows version of MySQL server, though perhaps
one that's being triggered by a problem in MyODBC and/or Dolphin DBConnection.

In the interest of minimizing the dependence on DBConnection/MyODBC, I found one
can use the MySQL command line to do the setup, and the final drop. So the
sequence could be:

// issue a command line to set up the database and table
    c:\mysql\bin\mysql -h frank -e "drop database if exists testtemp; create
database testtemp; CREATE TABLE testtemp.NEXT_ID (ID_ INTEGER)"

// do a Dolphin incant that apparently leaves a handle to the table open
// on the Windows MySQL server
    (DBConnection new dsn: 'Frank-testtemp')
        exec: 'use testtemp';  "<-- only needed if not set by datasource"
        columns: 'NEXT_ID';
        close.

// At this point one can even shut down Dolphin completely with no net effect.
// Note that the server is still fine.

// ask to drop the table
    c:\mysql\bin\mysql -h frank -e "drop table testtemp.NEXT_ID"

// the server puts the drop into the background drop queue
// and after a possible 5 or 10 second delay, the CPU pegs.

So there you have it. I have tried the latest MyODBC 3.51.06 in addition to the
previous 2.50.39 without seeing any difference. This has been with MySQL Max
3.23.55 using the InnoDB tables. I haven't yet had a chance to try MySQL 4.0.

regards,
-Bill

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Bill Dargel
In reply to this post by Chris Uppal-3
Chris Uppal wrote:

> I'm afraid that I don't have a good solution.
> [snip]
> It may well be that MySQL has the means for clients to Observe (in the sense of
> the Observer pattern) changes to the db, but I don't know how to do it.  In any
> case, I don't think (though this is *not* something I know much about) that
> standard SQL or ODBC provides a way to do it.

Okay. That's been my impression as well. But I figured it couldn't hurt to ask, in
case there was something that I had missed. :-)

And thanks for your "random noodlings" Chris. It got me to thinking about other
possibilities for having the clients communicate about what's going on, changing,
etc., other than just feeding it through the database. If I wrote something that
served as a communication nexus between the clients, it would become a good
opportunity to start migrating the first bit of application logic to the server,
and to learn about programming sockets.

thanks,
-Bill

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

FU Berlin-2
In reply to this post by FU Berlin-2
Chris, Jerome, Bill(1), Bill(2), Don

Thanks for your advices on my post.

Chris,
Thanks for your reference to Dimitry's post. I'll try it and then I probably
come back to your 'Server' solution, which looks interesting.


Jerome,
Thank you for your advice. I had a look on it but the 'Generic Server' classes
of ??? which look very similar to the 'SocketAbstract' classes in Dolphin. All I
know about sockets I learned through the 'Chat' example in Dolphin.

Bill Schwab,
I agree that it is not necessary to poll over databases in order to learn
something about other users. It reminds me on recent DOS days, where it
became in fashion one day to show a clock on a screen.
I am just finishing my first step to install a broadcast situation on a local
network. Probably, I'll run into the problems you mentioned of multiple
threads which have to be administered. If I fail I drop the whole thing.

-----------------------------------
I was puzzling over the UDPSocket class of Dimitry Zamotkin. I assume that
nobody has worked on it so far. If you add a 'bind' method to that class it is
easy to install a peer-to-peer connection or to broadcast over a (local)
network. I believe that it is even better to use this kind of communication on
a local network than the connection orientated one which has been designed
for communication on the internet. If somebody is interested in further
details he can send me an email. It is wasted time to work out some stuff
here if there is no need on it.

regards
Jon


Reply | Threaded
Open this post in threaded view
|

Re: Broadcasting over a local network ?

Jerome Chan
In article <[hidden email]>,
 [hidden email] (jon) wrote:

> I was puzzling over the UDPSocket class of Dimitry Zamotkin. I assume that
> nobody has worked on it so far. If you add a 'bind' method to that class it
> is
> easy to install a peer-to-peer connection or to broadcast over a (local)
> network. I believe that it is even better to use this kind of communication
> on
> a local network than the connection orientated one which has been designed
> for communication on the internet. If somebody is interested in further
> details he can send me an email. It is wasted time to work out some stuff
> here if there is no need on it.
>
> regards
> Jon

The problem with UDP multicasting is that there is no guarentee that all
clients recieved all the broadcast packets.