quick handling of graphics files

I have PNG files with document scans, and would like to view them as
part of an application.  I will also need to scale and/or pan them.

One file took over a minute to open when I selected "open graphic in
window" from the file browser.  (ImageReadWriter formFromFileNamed:
'myfile.png') asMorph openInHand had about the same speed.

I'm looking for advice on how to speed that up, or the information that
is not possible.  I found a similar question on the newbie list, but no
one answered it.

I'm guessing that it is the initial read and translation of the file
that's taking the time, since resizing the image is much faster than
loading it.

This is really too slow to be practical, and much slower than
imagemagick's display command, which opens in under 5 seconds.

The image is a 400dpi black and white scan of an 8.5x11" page; the file
is 144k bytes.

BTW is there any particular reason the file browser opens a SketchMorph
while asMorph on the ImageReadWrite result produces an ImageMorph?  The
former is closer to what I want since it is resizeable and not
transluscent; on the other hand, the comment in SketchMorph says they
are produced by the PaintBox, which is not true in this case.

Squeak 3.10.2 on Linux with 32 bit color in X.  I think squeak is using
32 bit color as well, but couldn't find the setting on a quick look (not
in preferences browser).

Sucks heh?

For Sophie we used quicktime document management to read documents. Well like any form of images, like say PNG files.

Right now your are reading and processing the bits in smalltalk, it's slow, also 50/50 it might give a walkback depending on what the PNG file is..

I had looked at porting the Sophie image reading logic into Pharo/Squeak as part of true graphic cut/copy,paste,
but it requires FFI which no-one seems interested in having in the base product.

On 2010-04-13, at 6:37 PM, Ross Boylan wrote:

> I have PNG files with document scans, and would like to view them as
> part of an application.  I will also need to scale and/or pan them.



--
===========================================================================
John M. McIntosh <[hidden email]>   Twitter:  squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
===========================================================================

On Tue, Apr 13, 2010 at 07:30:31PM -0700, John M McIntosh wrote:
>
> I had looked at porting the Sophie image reading logic into Pharo/Squeak
> as part of true graphic cut/copy,paste, but it requires FFI which no-one
> seems interested in having in the base product.

I for one think that FFI should be readily available, if not part of
a "base product" then as a reliably loadable package. I also think that
the 64-bit patches need to be applied to it real soon now, since 64-bit
Unix, Windows and OS X are becoming rather commonplace. So I'll probably
be nagging you about this on vm-dev sometime soon if you are at all
sympathetic to the cause ;)

Dave

Hi Ross -

Profiling is your friend. In most cases, 95% of the time are spent in 5%
of the code. From what you're saying below it sounds that you're using
one of the 'special' PNG modes (black and white, or gray-scale) that
probably have seen less attention for optimization than others. Any
chance you can post a sample image for profiling it?

Cheers,
   - Andreas

On 4/13/2010 6:37 PM, Ross Boylan wrote:

On Wednesday 14 April 2010 07:07:58 am Ross Boylan wrote:
> One file took over a minute to open when I selected "open graphic in
> window" from the file browser.  (ImageReadWriter formFromFileNamed:
> 'myfile.png') asMorph openInHand had about the same speed.
That is way too long. I am able to open 300k+ scans of A4 sheets in a couple
of seconds with Squeak 3.11 on Kubuntu 9.10.

What is special about this file?

Subbu

On Wed, 2010-04-14 at 20:45 +0530, K. K. Subramaniam wrote: I don't know, but here's some more info:
$ identify -verbose UBH20100319.png
Image: UBH20100319.png
  Format: PNG (Portable Network Graphics)
  Class: PseudoClass
  Geometry: 3360x4376+0+0
  Type: Bilevel
  Endianess: Undefined
  Colorspace: Gray
  Depth: 1-bit
  Channel depth:
    Gray: 1-bit
  Channel statistics:
    Gray:
      Min: 0 (0)
      Max: 1 (1)
      Mean: 0.914863 (0.914863)
      Standard deviation: 0.279085 (0.279085)
  Histogram:
  13451567: (255,255,255) #FFFFFF white
   1251793: (  0,  0,  0) #000000 black
  Colormap: 2
         0: (  0,  0,  0) #000000 black
         1: (255,255,255) #FFFFFF white
  Rendering intent: Undefined
  Resolution: 157.48x157.48
  Units: PixelsPerCentimeter
  Filesize: 141kb
  Interlace: None
  Background color: white
  Border color: rgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Page geometry: 3360x4376+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: Zip
  Orientation: Undefined
  Signature:
7aaf627fa1f5d1bae47c533ea395d610f975370e1c842764890d2c1115210f80
  Tainted: False
  User time: 0.550u
  Elapsed time: 0:03
  Pixels per second: 7.01111mb
  Version: ImageMagick 6.3.7 08/07/09 Q16 http://www.imagemagick.org

On 14.04.2010, at 18:01, Ross Boylan wrote:
As Andreas suggested, profile the loading code. If you don't know how to read the profile, send it here.

- Bert -

On Tue, 2010-04-13 at 21:05 -0700, Andreas Raab wrote:
> Hi Ross -
>
> Profiling is your friend. In most cases, 95% of the time are spent in 5%
> of the code. From what you're saying below it sounds that you're using
> one of the 'special' PNG modes (black and white, or gray-scale) that
> probably have seen less attention for optimization than others. Any
> chance you can post a sample image for profiling it?

I've attached a test image, which takes about 30 seconds for me.  It is
a 1 bit depth image.  Not sure if the attachment will make it
through....

Ross

On Tue, 2010-04-13 at 21:05 -0700, Andreas Raab wrote: Here are the results.  I got a code simulation error when I tried
tallyOn.  I'm using pharo-vm-0.15.2d-linux/squeak as the VM.

Aside from the PNG being 1 bit it is high resolution (400dpi) and the
displayed image is larger than the entire smalltalk window (or my
screen, I think).  
World extent -> 1704@1093

MessageTally spyOn: [(ImageReadWriter formFromFileNamed:
'/home/boylan/ScanImages/Medical/test.png') asMorph openInHand ]
except this was done with my original, larger image.

BTW, couldn't find a way to paste or fileout the results of
TimeProfileBrowser.

 - 53894 tallies, 53969 msec.

**Tree**
100.0% {53969ms} ImageReadWriter class>>formFromFileNamed:
  100.0% {53969ms} ImageReadWriter class>>formFromStream:
    100.0% {53969ms} PNGReadWriter>>nextImage
      100.0% {53969ms} PNGReadWriter>>processIDATChunk
        100.0% {53969ms} PNGReadWriter>>processNonInterlaced
          97.9% {52836ms} PNGReadWriter>>copyPixelsGray:
            43.0% {23207ms} BitBlt>>pixelAt:put:
              |25.3% {13654ms} Form>>unhibernate
              |  |12.8% {6908ms} primitives
              |  |12.5% {6746ms} Bitmap(Object)>>isMemberOf:
              |17.8% {9606ms} primitives
            27.8% {15003ms} SmallInteger>>=
              |14.5% {7826ms} primitives
              |13.3% {7178ms} SmallInteger(Integer)>>=
            13.9% {7502ms} primitives
            13.1% {7070ms} SmallInteger(Integer)>>>>

**Leaves**
17.8% {9606ms} BitBlt>>pixelAt:put:
14.5% {7826ms} SmallInteger>>=
13.9% {7502ms} PNGReadWriter>>copyPixelsGray:
13.3% {7178ms} SmallInteger(Integer)>>=
13.1% {7070ms} SmallInteger(Integer)>>>>
12.8% {6908ms} Form>>unhibernate
12.5% {6746ms} Bitmap(Object)>>isMemberOf:

**Memory**
        old +0 bytes
        young +3,823,212 bytes
        used +3,823,212 bytes
        free -3,823,212 bytes

**GCs**
        full 0 totalling 0ms (0.0% uptime)
        incr 4281 totalling 1,469ms (3.0% uptime), avg 0.0ms
        tenures 0
        root table 0 overflows

Ross Boylan wrote:
Profiling is indeed your friend.
There is some serious inefficiency there. Quickly hacking this (warning:
will only work for 1bpp):

copyPixelsGray: y
    "Handle non-interlaced grayscale color mode (colorType = 0)"
    | word base ii |
    base :=  y * form width//32 +1.
    0 to: thisScanline size-1 // 4 do: [ :i |
        ii := i * 4.
        word :=
            (thisScanline at: ii+1) << 24 bitOr: (
            (thisScanline at: ii+2) << 16 bitOr: (
            (thisScanline at: ii+3) << 8 bitOr: (
            (thisScanline at: ii+4)))).
        form bits at: base + i put: word ].

gives over 30x speed increase (from 10 seconds down to 310 mSec) on my
system. This is not a solution, just some food for thought.

Cheers,
Juan Vuletich

On Wed, 14 Apr 2010, Juan Vuletich wrote:

You can squeeze out even better performance from this, by using #* and #+
instead of #<< and #bitOr: when the result and arguments are
SmallIntegers:

         word :=
             ((thisScanline at: ii+1) * 256 +
             (thisScanline at: ii+2) * 256 +
             (thisScanline at: ii+3) bitShift: 8) bitOr:
             (thisScanline at: ii+4).

Levente

>
> Cheers,
> Juan Vuletich
>
>

Wow!  Thanks Juan and Levente; I get 30x speedup too; I think things are
now workable.

Should I open a bug on this?

Ross

At Wed, 14 Apr 2010 23:18:48 +0200 (CEST),
Levente Uzonyi wrote:
  Ah ha.  If you define a method at Bitmap that looks like:

copyFromByteArray2: byteArray to: i
        "This method should work with either byte orderings"

        | myHack byteHack |
        myHack := Form new hackBits: self.
        byteHack := Form new hackBits: byteArray.
        SmalltalkImage current  isLittleEndian ifTrue: [byteHack swapEndianness].
        byteHack displayOn: myHack at:  0@i

and make #copyPixelsGray: to be:

copyPixelsGray: y
        form bits copyFromByteArray2: thisScanline to: y * (form width // 32)

A test like this:

[(PNGReadWriter on: (FileStream readOnlyFileNamed: 'test.png')) nextImage] timeToRun

runs 2.5x faster than Juan's version on my computer.

-- Yoshiki

On 4/14/2010 1:08 PM, Juan Vuletich wrote:
> Profiling is indeed your friend.
> There is some serious inefficiency there. Quickly hacking this (warning:
> will only work for 1bpp):

[... snip ...]

> gives over 30x speed increase (from 10 seconds down to 310 mSec) on my
> system. This is not a solution, just some food for thought.

Heh, heh. Very good. But now I'm gonna get serious ...

<pokerface on>

I see your 30x improvement and raise you another ... 6x for a total of
200x speedup (from 10secs to 50 msecs). There! Take that! :-)

(but if Igor pulls out some asm I may have to fold :-)

<pokerface off>


Cheers,
   - Andreas

At Wed, 14 Apr 2010 19:54:22 -0700,
Andreas Raab wrote:
>
> I see your 30x improvement and raise you another ... 6x for a total of
> 200x speedup (from 10secs to 50 msecs). There! Take that! :-)

  Man!  Even the timestamps of these methods are the same.  How fast
can you type?! ^^;

-- Yoshiki

On 4/14/2010 8:38 PM, Yoshiki Ohshima wrote:
> At Wed, 14 Apr 2010 19:54:22 -0700,
> Andreas Raab wrote:
>>
>> I see your 30x improvement and raise you another ... 6x for a total of
>> 200x speedup (from 10secs to 50 msecs). There! Take that! :-)
>
>    Man!  Even the timestamps of these methods are the same.  How fast
> can you type?! ^^;

Time is an illusion. Lunchtime doubly so. :-)

(I update the comments before filing it out)

Cheers,
   - Andreas

On 15.04.2010, at 06:47, Andreas Raab wrote:
19:50 is lunchtime for you? That explains a lot ;)

In any case, this is a nice example of how naive code works fine for years, but if you need speed, there are certainly ways to make it efficient (if less readable).

- Bert -

> (I update the comments before filing it out)
>
> Cheers,
>  - Andreas
>
>

On 4/14/2010 11:39 PM, Bert Freudenberg wrote:
Indeed. I would vote for including Juan's version and the ZipPlugin
primitive (which for some inexplicable reason wasn't there). This still
gets you a great speedup which ought to be enough until someone else
complains :-)

Cheers,
   - Andreas

Andreas Raab wrote: This is an updated version. It includes Levente's suggestion, and
Yoshiki's and Andreas' alternatives as well, although disabled. All
three options now work for gray level images with other bit depths
besides 1 (I only tested 1 and 8). I also fixed transparent pixels for
gray level images (it was completely broken).

Cheers,
Juan Vuletich

On 15 April 2010 05:54, Andreas Raab <[hidden email]> wrote:
Yeah.. one could use MMX/SSE/SIMD instructions which will put on the
knees anything you can write
in C, not mentioning smalltalk :)

> <pokerface off>
>
>
> Cheers,
>  - Andreas
>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.