[squeak-dev] etoys performance problems

  Sorry, I wasn't following the ticket closely.  This seems to be good
for inclusion.  Also, you can choose the compression when recording
from the sound recorder.

-- Yoshiki

  Hilaire,

> To be accurate, what happen if one drag and drop a jpeg file in an
> image, then save a project file, move the project in another computer
> without the original jpeg file? Is the jpeg file saved in the .pr zip
> file as well?

  Currently, it forgets the original format and all are treated as the
same Form.

  I contemplated that we can keep the original format, and if it was
JPEG, we could even keep the original bytes of JPEG data.  The catch
is that if the user painted on the Sketch created from JPEG,
determining whether to make sense to store the result in JPEG or not
is not trivial.

-- Yoshiki

2008/3/27, Yoshiki Ohshima <[hidden email]>:
So we have clearly an issue there !

>   I contemplated that we can keep the original format, and if it was
>  JPEG, we could even keep the original bytes of JPEG data.  The catch
>  is that if the user painted on the Sketch created from JPEG,
>  determining whether to make sense to store the result in JPEG or not
>  is not trivial.

Indeed not trivial. A simple first helpful boost will that unmodified
image (JPEG, PNG, GIF) to be kept in original format.
It is very likely other users of Squeak will complain about that
problem (pr. file getting big pretty quickly).

Hilaire



--
http://blog.ofset.org/hilaire
Smalltalkers do: [:it | All with: Class, (And love: it)]

  Hello,

  I published a few changes in last a couple of weeks; I just looked
at some inefficiency and avoid them.

  I compared the project loading time in Etoys 2.3 (before Luke's method
dictionary patch) and Etoys 3.0 #1957.  I loaded all projects in the
ExampleEtoys like this:

-------------------
| dir entries proj |
dir _ FileDirectory on: '/usr/share/etoys/ExampleEtoys'.
entries _ FileList2 projectOnlySelectionMethod: dir entries.
entries _ entries collect: [:each | Project parseProjectFileName: each first].
entries do: [:each |
        Transcript show: '\classic: ' withCRs , each first, ' ', ([ProjectLoading loadFromDir: '/usr/share/etoys/ExampleEtoys' projectName: each first] timeToRun) printString.
        proj _ (Project named: each first).
        proj ifNotNil: [proj okToChangeSilently].
].
-------------------

got result (numbers are in milliseconds and are the average of three
runs):

BallDropAnalysis1 13850.3 9971 1.38906
BetterMovieUI 9662.67 6145 1.57244
BouncingBallAnimation 4825.33 3151.67 1.53104
CarAndPen 5676.33 3504.67 1.61965
ComputerLogicGame 14902.7 7080.33 2.1048
DemonCastle1 24442.7 11497.7 2.12588
EtoysChallenge 19955.7 10435.7 1.91226
FishAndPlankton 12526 9573.33 1.30843
FollowRoad 5532.33 3869.33 1.42979
JustPaintedCar 3777 2237 1.68842
LunarLanderGame 7730 5286.67 1.46217
MakeAMovie 8909.67 5718 1.55818
MiddleOfRoad 6056 3554 1.704
ParticlesDyeInWater 11483.3 5672.33 2.02445
ParticlesEpidemic 9368.67 5116.33 1.83113
ParticlesGasModel 10688 4668.67 2.2893
RandomRacing 7013.33 4263.33 1.64504
SalmonSniff 5943 3912.33 1.51904
SimpleSprings 5563 4566.67 1.21818
SpeedAcceleration 6150.67 4421.33 1.39113
StartOfDTPDocument 4577.67 3090.67 1.48113
SteeringTheCar 6285.67 4388.67 1.43225
TurtleGeometry 6117.67 3903.33 1.56729
Welcome 22904 11830.3 1.93604

  I haven't done anything about the double compression problem,
putting media file aside, nor looking at the saving side.  But now
project loading is 20%-100% faster than before, and bigger projects
seem to benefit more.

  I have been experimenting another new format in S-expression, but
found that for bigger projects, the new format is much slower.  Unless
there is a way to optimize it, or some other reason, probably we stick
with the old format for now...

  I'll try to modify the code so that .pr file may not have to be
compressed and see if how much we gain.

-- Yoshiki

Yoshiki Ohshima wrote:
Great progress. Do the big project contain lots of forms ? Sound is very
efficiently compressed instances with the ogg plugin and maybe something
similar would be possible for forms.

Karl

>   I'll try to modify the code so that .pr file may not have to be
>  compressed and see if how much we gain.

Ah nice, that happens to be what I'm working on as well.
I've got some patches to play around with which disable either zipping
the project file or the bundle, and subsequently only delivers the .pr
file (Oh, the confusion, naming both the project bundle as well as the
project file <pr-name>.pr).

Hopefully tomorrow I'll give an account of my performance findings
while testing different combinations of zipping. As an intermediate
report, totally unzipping a .pr file almost doubles loading on my mac,
 but loading on an XO has  been slightly disappointing. Small projects
were up to 2 times as fast to load, but the bigger ones were a quarter
as slow.

Attached are some other patches related to zipping:

1) fileIn fix
In order to load a project bundle with an non-zipped .pr file in it,
you've got to change the type of stream to create in
projectStreamFromArchive from RWBinaryOrTextStream to
MultiByteBinaryOrTextStream, since that's the class the project
loading methods are defined on.

2) asUnzippedStream
changes uptoEndWithProgressBar to uptoEnd in the method with the same
name as the changeset to get a 20 percent increase in project loading.

These patches are part of my 'one patch per day' project, that will
run from now till into the not at all to distant future.

/Ties

  Hi, Ties,

> 1) fileIn fix
> In order to load a project bundle with an non-zipped .pr file in it,
> you've got to change the type of stream to create in
> projectStreamFromArchive from RWBinaryOrTextStream to
> MultiByteBinaryOrTextStream, since that's the class the project
> loading methods are defined on.

  Hmm.  What was the performance impact by this?

> 2) asUnzippedStream
> changes uptoEndWithProgressBar to uptoEnd in the method with the same
> name as the changeset to get a 20 percent increase in project loading.
>
> These patches are part of my 'one patch per day' project, that will
> run from now till into the not at all to distant future.

  One time, we thought that we would rather give a feedback while
something is going on...  25 seconds with some visible progress going
on vs. 20 seconds with frozen screen and it is still debatable whether
which would be better.

  However, is it true that this progress may not be relevant much as
other part of project loading will spend more time?

-- Yoshiki

On Apr 10, 2008, at 2:01 PM, Yoshiki Ohshima wrote:

>  One time, we thought that we would rather give a feedback while
> something is going on...  25 seconds with some visible progress going
> on vs. 20 seconds with frozen screen and it is still debatable whether
> which would be better.

I've seen this handled by poping up the visible progress screen, but  
only if say
5 or 10 seconds have gone by. So for fast operations you are not  
impacted by
the drawing, but longer operations then get the progress screen.


>
>
>  However, is it true that this progress may not be relevant much as
> other part of project loading will spend more time?
>
> -- Yoshiki
>

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================

On Fri, Apr 11, 2008 at 2:46 AM, Yoshiki Ohshima <[hidden email]> wrote:
None, but when an uncompressed .pr file sits in another archive, the
code that makes a stream for the .pr file makes the wrong kind of
stream for said .pr file. So it's a simple bugfix.

/Ties

On Fri, Apr 11, 2008 at 2:46 AM, Yoshiki Ohshima <[hidden email]> wrote:
>   One time, we thought that we would rather give a feedback while
>  something is going on...  25 seconds with some visible progress going
>  on vs. 20 seconds with frozen screen and it is still debatable whether
>  which would be better.

You mean that no info whatsoever is given when unzipping? I didn't pay
much attention to the project progress bar, but I did notice that with
big projects it takes a while to show so I presumed it was unzipping.
I thought we could just as well start showing the progress bar before
we start zipping. An empty progress bar is information enough I would
think, and the rest of the filling of the progress bar is hardwired,
after certain operations, so the bar is already incrementing in steps
(five or something I thought).

>   However, is it true that this progress may not be relevant much as
>  other part of project loading will spend more time?

You definitely notice it on the XO with 1mb+ files.

/Ties

>  -- Yoshiki
>
>

  But, this does make the current project loading slower to me.  Try
(in an updated image):

MessageTally spyOn: [ProjectLoading loadFromDir:
'/usr/share/etoys/ExampleEtoys' projectName: 'DemonCastle1'].

or such with and without the change and see the result.

  But, yes, I agree that there seems to be some potential "hanging
fruit" here.  Thank you for attacking this!

-- Yoshiki

> >   However, is it true that this progress may not be relevant much as
> >  other part of project loading will spend more time?
>
> You definitely notice it on the XO with 1mb+ files.

  Yes.  Takashi reminded me that the #upToEndProgressBar in
ReadWriteStream was when we were showing stacked progress bars.  Now
we only show one bar (that goes back and forth), it makes perfect
sense to apply your patch.

  Thank you!

-- Yoshiki