Smalltalk › Pharo › Pharo Smalltalk Users

breaking infinite loop from saved image

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Peter Uhnak

breaking infinite loop from saved image

In moment of my utter brilliance I managed to save image without noticing that I have infinite loop running...

So it obviously crashed and launching is quite problematic.

I managed to recompile infringing method from a startup script, since it seems strangly that it has higher priority, however right after that garbage collection kicks in and due to (probably) massive accumulated stack it dies again...(it is a nice 875M image -_-)

~~~~~~~~~

Smalltalk stack dump:

0xff916d80 I SmalltalkImage>lowSpaceWatcher 0xb7674470: a(n) SmalltalkImage

0xf2ae8a2c s [] in SmalltalkImage>installLowSpaceWatcher

0xf2ab50e0 s [] in BlockClosure>newProcess

stack page bytes 4096 available headroom 3300 minimum unused headroom 3524

(out of memory)

~~~~~~~~~

Is there some magic remedy for this?

It seems that there is a small window between starting the image and it crashing where I can do some little things; I managed to file out the most critical package like this, however I don't know where else I was making changes (day-ish of work)...

The loop itself is announcement-based, so maybe that's why there is some room for experimenting...

Thanks,

Peter

Paul DeBruicker

Re: breaking infinite loop from saved image

What platform are you on?

You can raise the RAM limits the VM allows on a mac by editing the Info.plist file in the vm bundle. By default the limit is set to:

<key>SqueakMaxHeapSize</key>
<integer>536870912</integer>

So maybe if you're on a Mac you can move that up to a larger number and the GC will complete.

Hope this helps

Paul

Peter Uhnák wrote

In moment of my utter brilliance I managed to save image without noticing
that I have infinite loop running...
So it obviously crashed and launching is quite problematic.
I managed to recompile infringing method from a startup script, since it
seems strangly that it has higher priority, however right after that
garbage collection kicks in and due to (probably) massive accumulated stack
it dies again...(it is a nice 875M image -_-)
~~~~~~~~~
Smalltalk stack dump:
0xff916d80 I SmalltalkImage>lowSpaceWatcher 0xb7674470: a(n) SmalltalkImage
0xf2ae8a2c s [] in SmalltalkImage>installLowSpaceWatcher
0xf2ab50e0 s [] in BlockClosure>newProcess

stack page bytes 4096 available headroom 3300 minimum unused headroom 3524

(out of memory)
~~~~~~~~~

Is there some magic remedy for this?

It seems that there is a small window between starting the image and it
crashing where I can do some little things; I managed to file out the most
critical package like this, however I don't know where else I was making
changes (day-ish of work)...

The loop itself is announcement-based, so maybe that's why there is some
room for experimenting...

Thanks,
Peter

philippeback

Re: breaking infinite loop from saved image

In reply to this post by Peter Uhnak

Gille fixed one of my locked up images with his Oz tooling.

I'd welcome that thing to be more widely usable.

No clue about the current state of affairs on that front.

Phil

On Tue, Jun 16, 2015 at 2:00 AM, Peter Uhnák <[hidden email]> wrote:

In moment of my utter brilliance I managed to save image without noticing that I have infinite loop running...
So it obviously crashed and launching is quite problematic.
I managed to recompile infringing method from a startup script, since it seems strangly that it has higher priority, however right after that garbage collection kicks in and due to (probably) massive accumulated stack it dies again...(it is a nice 875M image -_-)
~~~~~~~~~
Smalltalk stack dump:
0xff916d80 I SmalltalkImage>lowSpaceWatcher 0xb7674470: a(n) SmalltalkImage
0xf2ae8a2c s [] in SmalltalkImage>installLowSpaceWatcher
0xf2ab50e0 s [] in BlockClosure>newProcess

stack page bytes 4096 available headroom 3300 minimum unused headroom 3524

(out of memory)
~~~~~~~~~

Is there some magic remedy for this?

It seems that there is a small window between starting the image and it crashing where I can do some little things; I managed to file out the most critical package like this, however I don't know where else I was making changes (day-ish of work)...

The loop itself is announcement-based, so maybe that's why there is some room for experimenting...

Thanks,
Peter

Nicolai Hess

Re: breaking infinite loop from saved image

2015-06-16 7:41 GMT+02:00 [hidden email] <[hidden email]>:

Gille fixed one of my locked up images with his Oz tooling.

I'd welcome that thing to be more widely usable.

No clue about the current state of affairs on that front.

Phil

Did you try to open (a copy of) the changes file with a new/fresh image?

Or you can use :

ExternalChangesBrowser openOn:'FULLNAME_OF_OTHER_CHANGES_FILE'

to open a changes browser.

nicolai

On Tue, Jun 16, 2015 at 2:00 AM, Peter Uhnák <[hidden email]> wrote:
In moment of my utter brilliance I managed to save image without noticing that I have infinite loop running...
So it obviously crashed and launching is quite problematic.
I managed to recompile infringing method from a startup script, since it seems strangly that it has higher priority, however right after that garbage collection kicks in and due to (probably) massive accumulated stack it dies again...(it is a nice 875M image -_-)
~~~~~~~~~
Smalltalk stack dump:
0xff916d80 I SmalltalkImage>lowSpaceWatcher 0xb7674470: a(n) SmalltalkImage
0xf2ae8a2c s [] in SmalltalkImage>installLowSpaceWatcher
0xf2ab50e0 s [] in BlockClosure>newProcess

stack page bytes 4096 available headroom 3300 minimum unused headroom 3524

(out of memory)
~~~~~~~~~

Is there some magic remedy for this?

It seems that there is a small window between starting the image and it crashing where I can do some little things; I managed to file out the most critical package like this, however I don't know where else I was making changes (day-ish of work)...

The loop itself is announcement-based, so maybe that's why there is some room for experimenting...

Thanks,
Peter

Peter Uhnak

Re: breaking infinite loop from saved image

I ended up fileouting what I could in headless mode (had utf8 issues for whatever reason so I didn't get it all), and rewriting the rest. :/

But what happens once will happen again... so some more generic solution for this would be nice.

Yuriy mentioned Oz, and it looks viable, provided it works with current pharo version and is capable of dealing with (most likely) corrupted memory.

However as I personally prefer time traveling over archaeology, I would like having multiple snapshot of the image (10 minutes ago, 20 minutes ago) so if my image gets corrupted I loose at most 10 minutes of time. I can always dig it out, but there is always some work involved (downloading new image, setting it up, downloading work packages, digging out the old changeset, ...).

On Tue, Jun 16, 2015 at 8:45 AM, Nicolai Hess <[hidden email]> wrote:

Did you try to open (a copy of) the changes file with a new/fresh image?
Or you can use :

ExternalChangesBrowser openOn:'FULLNAME_OF_OTHER_CHANGES_FILE'
to open a changes browser.

Good to know this is available now (I am still in Pharo 4 tho), but this is still archaeology for me. :)

Anyway, I've resolved my current problem, so the question now is how to add some extra fault tolerance.

Peter

Guillermo Polito

Re: breaking infinite loop from saved image

Hi!

El mar., 16 de jun. de 2015 a la(s) 8:55 a. m., Peter Uhnák <[hidden email]> escribió:

I ended up fileouting what I could in headless mode (had utf8 issues for whatever reason so I didn't get it all), and rewriting the rest. :/
But what happens once will happen again... so some more generic solution for this would be nice.

Given command line was usable you could have used it to break the loop.

Let's say you had your method that recurses looks like

BadGuy>>recursive

self recursive

then you can do

./pharo eval "BadGuy compile: 'recursive'"

that will break your method but will help you recover the rest :)

Another useful debugging tool, maybe you knew about it, is sending the SIGUSR1 signal to the VM process. It will print out the stack and help you detect possible problems.

Yuriy mentioned Oz, and it looks viable, provided it works with current pharo version and is capable of dealing with (most likely) corrupted memory.

Well, Oz has never left the prototype stage, and I doubt it will soon with the time I have and the hackish VM requirements.

Guille

Peter Uhnak

Re: breaking infinite loop from saved image

On Tue, Jun 16, 2015 at 10:09 AM, Guillermo Polito <[hidden email]> wrote:

Hi!

El mar., 16 de jun. de 2015 a la(s) 8:55 a. m., Peter Uhnák <[hidden email]> escribió:
I ended up fileouting what I could in headless mode (had utf8 issues for whatever reason so I didn't get it all), and rewriting the rest. :/
But what happens once will happen again... so some more generic solution for this would be nice.

Given command line was usable you could have used it to break the loop.

Let's say you had your method that recurses looks like

BadGuy>>recursive
self recursive

then you can do

./pharo eval "BadGuy compile: 'recursive'"

As I have said originally, I did recompile the method to get rid of it (albeit from StartupScript and not eval), but there was already to much stuff on stack (every call fired a new announcement and the observer called the method again), so the system resources were already clogged, and even after I recompiled it it crashed shortly thereafter.

Another useful debugging tool, maybe you knew about it, is sending the SIGUSR1 signal to the VM process. It will print out the stack and help you detect possible problems.

Well on crash I did receive the stack so I knew what was the problem, but it might be helpful in the future.

Well, Oz has never left the prototype stage, and I doubt it will soon with the time I have and the hackish VM requirements.

I guess making tools for image surgery ain't easy, but it certainly is interesting. :)

Peter

Stephan Eggermont-3

Re: breaking infinite loop from saved image

On 16-06-15 10:44, Peter Uhnák wrote:

> I guess making tools for image surgery ain't easy, but it certainly is
> interesting. :)

I suppose in the long term this will get easier when we can have tools
in one image operating on another image.

Stephan