Smalltalk - Re: What is slowing Glorp down north of 10,000 objects in the Transaction's undoMap?

Re: What is slowing Glorp down north of 10,000 objects in the Transaction's undoMap?

Posted by jtuchel on Nov 05, 2020; 2:19pm
URL: https://forum.world.st/What-is-slowing-Glorp-down-north-of-10-000-objects-in-the-Transaction-s-undoMap-tp5124433p5124441.html

Hi Alan,

this is what is currently blocking my machine. I'm traceing a single commit of that slow kind and tracing alone took almost 4 hours ;-) I didn't expect this to take that long, otherwise I'd have started with sampling. I have a lot of time to answer my mails now ;-) Next time I'll sample first (which is what the manual says, btw, but who reads manuals ;-)) ).
The machine is now working on opening the Performance Workbench...

I'll be back with more info and very likely questions on what do do about this.

I already added some logging to the server application. The very same commit is fast for users with only a few hundred objects in their object net. The commitUOW takes between 150 and 400 msec for those users.

I am glad you suspect something similar like I do. Shows me I am learning. Maybe it is time to look up a chapter or two in my copy of Andres' Hashing book while I wait for the Workbench to open...

Joachim

Am 05.11.20 um 14:40 schrieb Alan Knight:

My first thought when seeing a non-linear slowdown on something large is hashing performance. It also might not be in the undoMap, but in the generated row maps, or somewhere else. But if the time is spent in Glorp, then a profile should help. Or even the quick and dirty profiler of pausing execution. If something is spending 90% of its time doing something, then a random pause will probably stop in that something.

On Thu, Nov 5, 2020 at 2:06 AM jtuchel <[hidden email]> wrote:

There are days when I think I still don't understand how to use Glorp right. We are looking into performance issues where teh insert of 4 rows into 3 tables and takes a up to a minute instead of a few msecs. The time is spent in Glorp, not in the database. We added a bunch of logging statements to GlorpSession and UnitOfWork and we already know most of the time is spent in Glorp before any SQL is issued to the Database.

This extreme slowdown appears only for users who have loaded lots of objects from the DB. In our current case, there are a bit more than 10,000 entries in the undoMap of the currentUnitOfWork. It seems like 10,000 is a magic number here, a few weeks ago when less data was in play, the performance was okay for this user.
Users with just a few hundred objects have very nice performance.

I want to find ot whether this is a VAST specific problem. Glorp uses an IdentityDictionary for the undoMap on both VAST and VW (and I guess in Pharo as well). This may or may not be a problem, I simply don't know. Is there anybody here on this list *not* on VA Smalltalk who has such big transactions (remember: not number of updates, just objects loeded from the DB!).

I wonder how I can go on from here? Response times of one minute and more are not acceptable...

Any ideas?

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/88932d86-9c11-4df8-b2c4-f046308a71d1n%40googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "glorp-group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/glorp-group/vtmzMOW3QSA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/CAGWHZ99W5KP2F2uuZTAt-bDEphE6S75oKSYvrWy61Ef-MY2nxQ%40mail.gmail.com.

-- 
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel          [hidden email]
Fliederweg 1                         http://www.objektfabrik.de
D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/13120648-c283-91cd-2116-9fc1e8a235dc%40objektfabrik.de.