.source file

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

.source file

HilaireFernandes
For information, a iStoa user reported to me this problem, related to a
Pharo regression I already discussed about here
https://bugs.launchpad.net/istoa/+bug/1384649

I see this related bug report:
https://pharo.fogbugz.com/f/cases/10464/Always-warn-when-sources-or-changes-are-not-available

So what is the plan regarding source code, will we be forced to ship it.
It is untrue disk space is cheap, especially for desktop or tablet
application.

Hilaire

--
Dr. Geo - http://drgeo.eu
iStoa - http://istoa.drgeo.eu


Reply | Threaded
Open this post in threaded view
|

Re: .source file

Sven Van Caekenberghe-2
Have you tried using two fake empty files ?
If you are sure they are not used, that should work, right ?

Or you could try hacking SourceFileArray>>#ensureOpen

> On 23 Oct 2014, at 16:38, Hilaire <[hidden email]> wrote:
>
> For information, a iStoa user reported to me this problem, related to a
> Pharo regression I already discussed about here
> https://bugs.launchpad.net/istoa/+bug/1384649
>
> I see this related bug report:
> https://pharo.fogbugz.com/f/cases/10464/Always-warn-when-sources-or-changes-are-not-available
>
> So what is the plan regarding source code, will we be forced to ship it.
> It is untrue disk space is cheap, especially for desktop or tablet
> application.
>
> Hilaire
>
> --
> Dr. Geo - http://drgeo.eu
> iStoa - http://istoa.drgeo.eu
>
>


Reply | Threaded
Open this post in threaded view
|

Re: .source file

Esteban A. Maringolo
In reply to this post by HilaireFernandes
I've been hit by that "cannot find some file" error before.

No option other than close Pharo and restart it (without saving).


Esteban A. Maringolo


2014-10-23 11:38 GMT-03:00 Hilaire <[hidden email]>:

> For information, a iStoa user reported to me this problem, related to a
> Pharo regression I already discussed about here
> https://bugs.launchpad.net/istoa/+bug/1384649
>
> I see this related bug report:
> https://pharo.fogbugz.com/f/cases/10464/Always-warn-when-sources-or-changes-are-not-available
>
> So what is the plan regarding source code, will we be forced to ship it.
> It is untrue disk space is cheap, especially for desktop or tablet
> application.
>
> Hilaire
>
> --
> Dr. Geo - http://drgeo.eu
> iStoa - http://istoa.drgeo.eu
>
>

Reply | Threaded
Open this post in threaded view
|

Re: .source file

Sven Van Caekenberghe-2
In reply to this post by Sven Van Caekenberghe-2

> On 23 Oct 2014, at 16:55, Sven Van Caekenberghe <[hidden email]> wrote:
>
> Have you tried using two fake empty files ?
> If you are sure they are not used, that should work, right ?

I just tried and it works.
I removed the .changes file and replaced the .source file by a file containing just the word empty (a totally empty file does not seem to work).
Then the 4.0 image starts up and works OK (I only tried command line), it created an empty .changes.

Even decompilation works !

$ ./pharo Pharo.image eval '(ZnClient>>#execute) sourceCode'
'execute
        | t1 |
        self prepareRequest.
        self withProgressDo: [ t1 := self executeWithTimeout ].
        self prepareNextRequest.
        ^ t1'

Cool !

HTH,

Sven

> Or you could try hacking SourceFileArray>>#ensureOpen


Reply | Threaded
Open this post in threaded view
|

Re: .source file

HilaireFernandes
Le 23/10/2014 21:17, Sven Van Caekenberghe a écrit :

> I just tried and it works.
> I removed the .changes file and replaced the .source file by a file containing just the word empty (a totally empty file does not seem to work).
> Then the 4.0 image starts up and works OK (I only tried command line), it created an empty .changes.

Works with Pharo 3 as well.


> Even decompilation works !

Cool.
With Pharo 4 only, right? If I remember correctly decompiler was not
there in Pharo3

Thanks

>
> $ ./pharo Pharo.image eval '(ZnClient>>#execute) sourceCode'
> 'execute
> | t1 |
> self prepareRequest.
> self withProgressDo: [ t1 := self executeWithTimeout ].
> self prepareNextRequest.
> ^ t1'
>
> Cool !
>
> HTH,
>
> Sven
>
>> Or you could try hacking SourceFileArray>>#ensureOpen
>
>
>


--
Dr. Geo - http://drgeo.eu
iStoa - http://istoa.drgeo.eu


Reply | Threaded
Open this post in threaded view
|

Re: .source file

Marcus Denker-4

On 24 Oct 2014, at 10:24, Hilaire <[hidden email]> wrote:

Le 23/10/2014 21:17, Sven Van Caekenberghe a écrit :

I just tried and it works.
I removed the .changes file and replaced the .source file by a file containing just the word empty (a totally empty file does not seem to work).
Then the 4.0 image starts up and works OK (I only tried command line), it created an empty .changes.

Works with Pharo 3 as well.


Even decompilation works !

Cool.
With Pharo 4 only, right? If I remember correctly decompiler was not
there in Pharo3

yes.

Marcus

Reply | Threaded
Open this post in threaded view
|

Re: .source file

Sven Van Caekenberghe-2
In reply to this post by HilaireFernandes
I have to add that NativeBoost based code (like accessing environment variables for certain FileLocator places) breaks down when you have no sources ...

> On 24 Oct 2014, at 10:24, Hilaire <[hidden email]> wrote:
>
> Le 23/10/2014 21:17, Sven Van Caekenberghe a écrit :
>
>> I just tried and it works.
>> I removed the .changes file and replaced the .source file by a file containing just the word empty (a totally empty file does not seem to work).
>> Then the 4.0 image starts up and works OK (I only tried command line), it created an empty .changes.
>
> Works with Pharo 3 as well.
>
>
>> Even decompilation works !
>
> Cool.
> With Pharo 4 only, right? If I remember correctly decompiler was not
> there in Pharo3
>
> Thanks
>
>>
>> $ ./pharo Pharo.image eval '(ZnClient>>#execute) sourceCode'
>> 'execute
>> | t1 |
>> self prepareRequest.
>> self withProgressDo: [ t1 := self executeWithTimeout ].
>> self prepareNextRequest.
>> ^ t1'
>>
>> Cool !
>>
>> HTH,
>>
>> Sven
>>
>>> Or you could try hacking SourceFileArray>>#ensureOpen
>>
>>
>>
>
>
> --
> Dr. Geo - http://drgeo.eu
> iStoa - http://istoa.drgeo.eu


Reply | Threaded
Open this post in threaded view
|

Re: .source file

HilaireFernandes
Le 24/10/2014 10:35, Sven Van Caekenberghe a écrit :
> I have to add that NativeBoost based code (like accessing environment variables for certain FileLocator places) breaks down when you have no sources ...

Even with a fake .sources file?

--
Dr. Geo - http://drgeo.eu
iStoa - http://istoa.drgeo.eu


Reply | Threaded
Open this post in threaded view
|

Re: .source file

Sven Van Caekenberghe-2

> On 24 Oct 2014, at 10:48, Hilaire <[hidden email]> wrote:
>
> Le 24/10/2014 10:35, Sven Van Caekenberghe a écrit :
>> I have to add that NativeBoost based code (like accessing environment variables for certain FileLocator places) breaks down when you have no sources ...
>
> Even with a fake .sources file?

Yes, I believe NativeBoost needs to know the argument names of primitives to match method arguments to C primitives, and this cannot be done with decompilation or something like that
Reply | Threaded
Open this post in threaded view
|

Re: .source file

Henrik Sperre Johansen
In reply to this post by HilaireFernandes

On 24 Oct 2014, at 10:48 , Hilaire <[hidden email]> wrote:

Le 24/10/2014 10:35, Sven Van Caekenberghe a écrit :
I have to add that NativeBoost based code (like accessing environment variables for certain FileLocator places) breaks down when you have no sources ...

Even with a fake .sources file?

--
Dr. Geo - http://drgeo.eu
iStoa - http://istoa.drgeo.eu

Few NB-FFI methods will work without sources, as many rely on the automatic argument name -> signature mapping.
This happens the first time a method is called after startup, and platform-specific recompilation occurs. When the argument names are decompiled as arg1, arg2, the mapping will fail. 
(Other NB methods that generate machine code without using NB-FFI,  will  still work though)

Theoretically, one could solve this by internalizing the source of all NB-FFI calling method in the method trailer* at the end of deployment preparation process. (so no NB methods have a chance to trigger before image save) 
NB puts native code into the same trailer space, and combing different method trailer types is currently unsupported (AFAIK), but the transition 
TrailerWithEmbeddedSource** -> TrailerWithNBCode should happen only once.

Cheers,
Henry

* something along the lines of:
 | nativeMethods |
nativeMethods := (Pragma allInstances select: [ :each | each arguments = #(#primitiveNativeCall #NativeBoostPlugin) ]) collect: #method. 

nativeMethods do: [ :nativeMethod | 
nativeMethod methodClass methodDict at: nativeMethod selector put: (nativeMethod copyWithTrailerBytes: 
(CompiledMethodTrailer new sourceCode: nativeMethod sourceCode))] 

** TrailerWithEmbeddedTemps would probably be sufficient, but other than a reserved type, I can't see there's working code for it


Reply | Threaded
Open this post in threaded view
|

Re: .source file

stepharo


Few NB-FFI methods will work without sources, as many rely on the automatic argument name -> signature mapping.
This happens the first time a method is called after startup, and platform-specific recompilation occurs. When the argument names are decompiled as arg1, arg2, the mapping will fail. 
(Other NB methods that generate machine code without using NB-FFI,  will  still work though)

Theoretically, one could solve this by internalizing the source of all NB-FFI calling method in the method trailer* at the end of deployment preparation process. (so no NB methods have a chance to trigger before image save) 
NB puts native code into the same trailer space, and combing different method trailer types is currently unsupported (AFAIK), but the transition 
TrailerWithEmbeddedSource** -> TrailerWithNBCode should happen only once.

Yes it would be good to have a method for doing that.
https://pharo.fogbugz.com/default.asp?14311

Cheers,
Henry

* something along the lines of:
 | nativeMethods |
nativeMethods := (Pragma allInstances select: [ :each | each arguments = #(#primitiveNativeCall #NativeBoostPlugin) ]) collect: #method. 

nativeMethods do: [ :nativeMethod | 
nativeMethod methodClass methodDict at: nativeMethod selector put: (nativeMethod copyWithTrailerBytes: 
(CompiledMethodTrailer new sourceCode: nativeMethod sourceCode))] 

** TrailerWithEmbeddedTemps would probably be sufficient, but other than a reserved type, I can't see there's working code for it





Reply | Threaded
Open this post in threaded view
|

Re: .source file

Marcus Denker-4

On 24 Oct 2014, at 14:06, stepharo <[hidden email]> wrote:



Few NB-FFI methods will work without sources, as many rely on the automatic argument name -> signature mapping.
This happens the first time a method is called after startup, and platform-specific recompilation occurs. When the argument names are decompiled as arg1, arg2, the mapping will fail. 
(Other NB methods that generate machine code without using NB-FFI,  will  still work though)

Theoretically, one could solve this by internalizing the source of all NB-FFI calling method in the method trailer* at the end of deployment preparation process. (so no NB methods have a chance to trigger before image save) 
NB puts native code into the same trailer space, and combing different method trailer types is currently unsupported (AFAIK), but the transition 
TrailerWithEmbeddedSource** -> TrailerWithNBCode should happen only once.

Yes it would be good to have a method for doing that.
https://pharo.fogbugz.com/default.asp?14311

No, please… we need to simplify, not make everything complex.

It would be much better to spend engery on getting rid of .sources and changes… we are drowning in complexity with
the current stuff.

Marcus

Reply | Threaded
Open this post in threaded view
|

Re: .source file

HilaireFernandes
In reply to this post by Sven Van Caekenberghe-2
Le 24/10/2014 13:03, Sven Van Caekenberghe a écrit :
> Yes, I believe NativeBoost needs to know the argument names of primitives to match method arguments to C primitives, and this cannot be done with decompilation or something like that

GLURP!

--
Dr. Geo - http://drgeo.eu
iStoa - http://istao.drgeo.eu


Reply | Threaded
Open this post in threaded view
|

Re: .source file

Nicolai Hess
In reply to this post by Marcus Denker-4
2014-10-24 14:09 GMT+02:00 Marcus Denker <[hidden email]>:

On 24 Oct 2014, at 14:06, stepharo <[hidden email]> wrote:



Few NB-FFI methods will work without sources, as many rely on the automatic argument name -> signature mapping.
This happens the first time a method is called after startup, and platform-specific recompilation occurs. When the argument names are decompiled as arg1, arg2, the mapping will fail. 
(Other NB methods that generate machine code without using NB-FFI,  will  still work though)

Theoretically, one could solve this by internalizing the source of all NB-FFI calling method in the method trailer* at the end of deployment preparation process. (so no NB methods have a chance to trigger before image save) 
NB puts native code into the same trailer space, and combing different method trailer types is currently unsupported (AFAIK), but the transition 
TrailerWithEmbeddedSource** -> TrailerWithNBCode should happen only once.

Yes it would be good to have a method for doing that.
https://pharo.fogbugz.com/default.asp?14311

No, please… we need to simplify, not make everything complex.

It would be much better to spend engery on getting rid of .sources and changes… we are drowning in complexity with
the current stuff.

Marcus


Isn't that for what NativeBoost prepareForProduction is used?


Reply | Threaded
Open this post in threaded view
|

Re: .source file

Marcus Denker-4

On 24 Oct 2014, at 15:05, Nicolai Hess <[hidden email]> wrote:

2014-10-24 14:09 GMT+02:00 Marcus Denker <[hidden email]>:

On 24 Oct 2014, at 14:06, stepharo <[hidden email]> wrote:



Few NB-FFI methods will work without sources, as many rely on the automatic argument name -> signature mapping.
This happens the first time a method is called after startup, and platform-specific recompilation occurs. When the argument names are decompiled as arg1, arg2, the mapping will fail. 
(Other NB methods that generate machine code without using NB-FFI,  will  still work though)

Theoretically, one could solve this by internalizing the source of all NB-FFI calling method in the method trailer* at the end of deployment preparation process. (so no NB methods have a chance to trigger before image save) 
NB puts native code into the same trailer space, and combing different method trailer types is currently unsupported (AFAIK), but the transition 
TrailerWithEmbeddedSource** -> TrailerWithNBCode should happen only once.

Yes it would be good to have a method for doing that.
https://pharo.fogbugz.com/default.asp?14311

No, please… we need to simplify, not make everything complex.

It would be much better to spend engery on getting rid of .sources and changes… we are drowning in complexity with
the current stuff.

Marcus


Isn't that for what NativeBoost prepareForProduction is used?


Yes! Now I remember, we solved that usecase already.

Good.

Ceterum Censeo: We need to get rid of .sources and .changes.

Marcus
Reply | Threaded
Open this post in threaded view
|

Re: .source file

Sven Van Caekenberghe-2

> On 24 Oct 2014, at 15:10, Marcus Denker <[hidden email]> wrote:
>
>> Isn't that for what NativeBoost prepareForProduction is used?
>>
>>
> Yes! Now I remember, we solved that usecase already.
>
> Good.

Nice!

Maybe it should run after each build, or is that platform dependent ?

Reply | Threaded
Open this post in threaded view
|

Re: .source file

Henrik Sperre Johansen

> On 24 Oct 2014, at 3:20 , Sven Van Caekenberghe <[hidden email]> wrote:
>
>
>> On 24 Oct 2014, at 15:10, Marcus Denker <[hidden email]> wrote:
>>
>>> Isn't that for what NativeBoost prepareForProduction is used?
>>>
>>>
>> Yes! Now I remember, we solved that usecase already.
>>
>> Good.
>
> Nice!
>
> Maybe it should run after each build, or is that platform dependent ?
>


Nonononono.
The NB trailer only knows how to reinstall source from an original trailer with source pointer, so after first invocation, it will lose the source and start failing at next image startup.

Cheers,
Henry
Reply | Threaded
Open this post in threaded view
|

Re: .source file

Henrik Sperre Johansen

> On 24 Oct 2014, at 3:25 , Henrik Johansen <[hidden email]> wrote:
>
>
>> On 24 Oct 2014, at 3:20 , Sven Van Caekenberghe <[hidden email]> wrote:
>>
>>
>>> On 24 Oct 2014, at 15:10, Marcus Denker <[hidden email]> wrote:
>>>
>>>> Isn't that for what NativeBoost prepareForProduction is used?
>>>>
>>>>
>>> Yes! Now I remember, we solved that usecase already.
>>>
>>> Good.
>>
>> Nice!
>>
>> Maybe it should run after each build, or is that platform dependent ?
>>
>
>
> Nonononono.
> The NB trailer only knows how to reinstall source from an original trailer with source pointer, so after first invocation, it will lose the source and start failing at next image startup.
>
> Cheers,
> Henry


Disregard that, prepareForProduction is implemented differently from what I described, and the argument mapping no longer works they way I seemed to remembered, using the cache if present.
It should be safe to called when building, and allows dropping the changes file later on with no bad side-effects like my little hack.

Cheers,
Henry