Re: [Pharo-dev] ftp repositories broken

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] ftp repositories broken

Eliot Miranda-2
 
Hi Esteban,


On Apr 4, 2017, at 9:45 AM, Esteban Lorenzano <[hidden email]> wrote:


On 4 Apr 2017, at 18:01, Ben Coman <[hidden email]> wrote:



On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>>
>> no it doesn't: squeak does not have that primitive, that’s why it does not fails :)
>
>
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)
look, this is the way where it is generated: 

translatedPrimitives
"an assorted list of various primitives"
(ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil:
[^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:"
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(String findSubstringViaPrimitive:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:))].
^#(
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(ByteString findSubstring:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:)
)

means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. 
Now, there is a bug there (because source is in String and not in ByteString), and I can fix that by asking correctly, but: 

1) this means that sources for pharo and squeak will be different and pharo needs to be generated in pharo. This is contrary to what we are trying to do with the VM: we want all sources to be the same, and differences should come from compilation flags and/or the presence of certain plugins (this is because is a lot easier to debug the VM like that). 
2) I wonder… why we are using this primitive an squeak is not? maybe we need to deprecate the use?

I believe Esteban's point Its not so much whether the VM has it
(since they are essentially the same VM + extra C-libs for Pharo)
but whether the Squeak Image is using it.

Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String.
That was added to Squeak 2015.05.01.  Squeak String seems to have never used the primitive (since 2005). 

this is the important thing, I think… so… why can’t we use that instead the other one?
what’s the difference?

You have identified the problem.  The bug is that Pharo broke this by renaming and repositioning the primitive to findSubstringViaPrimitive:in:startingAt:matchTable:.  If it had stayed where it was things would still be working.

So my suggestions are
a) restore Pharo to using by the primitive correctly in the exactly the same configuration as Squeak
b) no one in the Pharo community changes the definition or position of a primitive without consulting a vm person (and Esteban is a vm person as is Clément and arguably Guille).
c) we measure the performance of the primitive and the equivalent non-primitive code in StackInterpreter, Cog and Sista configurations (using Spur, the current object representation) for a range of strings and see how much benefit we're getter by from the primitive; maybe we can nuke it.

But yes, it is a bad bug to position this primitive in String; it works only for ByteString.


cheers!
Esteban


 
A diff shows some differences in implementation..  
   https://www.diffchecker.com/CmBJZjB6
which anyway wouldn't account for John's observations.

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] ftp repositories broken

Eliot Miranda-2
 


On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]> wrote:

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

Not yet.  We are close. Last week I got angry with Esteban because I thought that the joe was stalled because Pharo didn't want to move to opensmalltalk-vm, but I over reacted.  One of the issues preventing the move was indeed this primitive and the fact that someone, without thinking to talk to anyone working with the VM, renamed the primitive, and then someone put it on the wrong class.  Esteban and I have spent some hours trying to work around such issues.  I wish people would be more considerate.

If issues like this can be resolved we are very close to using the opensmalltalk-vm process for stable Pharo VMs.  Esteban wants (and has built) a test build that tries to produce sources and generate Vs and runs tests every time VMMaker is committed.  On the Squeak side we only try and build VMs when opensmalltalk-vm is committed in git.  I don't want to stand in Esteban's way.  I *do* want stable VMs to be built from the opensmalltalk-vm tree.

_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] ftp repositories broken

Nicolas Cellier
 


2017-04-04 20:03 GMT+02:00 Eliot Miranda <[hidden email]>:
 


On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]> wrote:

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

Not yet.  We are close. Last week I got angry with Esteban because I thought that the joe was stalled because Pharo didn't want to move to opensmalltalk-vm, but I over reacted.  One of the issues preventing the move was indeed this primitive and the fact that someone, without thinking to talk to anyone working with the VM, renamed the primitive, and then someone put it on the wrong class.  Esteban and I have spent some hours trying to work around such issues.  I wish people would be more considerate.

If issues like this can be resolved we are very close to using the opensmalltalk-vm process for stable Pharo VMs.  Esteban wants (and has built) a test build that tries to produce sources and generate Vs and runs tests every time VMMaker is committed.  On the Squeak side we only try and build VMs when opensmalltalk-vm is committed in git.  I don't want to stand in Esteban's way.  I *do* want stable VMs to be built from the opensmalltalk-vm tree.

_,,,^..^,,,_
best, Eliot


+1
The faster we get the feedback the better.
So the Pharo VM has to be built on opensmalltalk-vm

If Pharo people wants to have a fork for mastering their release cycle, (the officially released VM) that's understandable.
If Pharo people wants to have a better automation with VMMaker code generation (maybe in a dev branch) and non regression tests that's all good.

It's just that we should integrate back any improvment and fix ASAP.
It would be even better to commit those fix in opensmalltalk-vm directly (feature branch and/or pull request)

BTW, don't forget Ronie as a referent Pharo VM developer :)

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] ftp repositories broken

Eliot Miranda-2
 


On Apr 4, 2017, at 11:56 AM, Nicolas Cellier <[hidden email]> wrote:



2017-04-04 20:03 GMT+02:00 Eliot Miranda <[hidden email]>:
 


On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]> wrote:

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

Not yet.  We are close. Last week I got angry with Esteban because I thought that the joe was stalled because Pharo didn't want to move to opensmalltalk-vm, but I over reacted.  One of the issues preventing the move was indeed this primitive and the fact that someone, without thinking to talk to anyone working with the VM, renamed the primitive, and then someone put it on the wrong class.  Esteban and I have spent some hours trying to work around such issues.  I wish people would be more considerate.

If issues like this can be resolved we are very close to using the opensmalltalk-vm process for stable Pharo VMs.  Esteban wants (and has built) a test build that tries to produce sources and generate Vs and runs tests every time VMMaker is committed.  On the Squeak side we only try and build VMs when opensmalltalk-vm is committed in git.  I don't want to stand in Esteban's way.  I *do* want stable VMs to be built from the opensmalltalk-vm tree.

_,,,^..^,,,_
best, Eliot


+1
The faster we get the feedback the better.
So the Pharo VM has to be built on opensmalltalk-vm

If Pharo people wants to have a fork for mastering their release cycle, (the officially released VM) that's understandable.
If Pharo people wants to have a better automation with VMMaker code generation (maybe in a dev branch) and non regression tests that's all good.

It's just that we should integrate back any improvment and fix ASAP.
It would be even better to commit those fix in opensmalltalk-vm directly (feature branch and/or pull request)

BTW, don't forget Ronie as a referent Pharo VM developer :)

Yes.  I was kicking myself as soon as I hit send.  Forgive me, Ronie.
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] ftp repositories broken

Nicolas Cellier
 


2017-04-04 22:32 GMT+02:00 Eliot Miranda <[hidden email]>:
 


On Apr 4, 2017, at 11:56 AM, Nicolas Cellier <[hidden email]> wrote:



2017-04-04 20:03 GMT+02:00 Eliot Miranda <[hidden email]>:
 


On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]> wrote:

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

Not yet.  We are close. Last week I got angry with Esteban because I thought that the joe was stalled because Pharo didn't want to move to opensmalltalk-vm, but I over reacted.  One of the issues preventing the move was indeed this primitive and the fact that someone, without thinking to talk to anyone working with the VM, renamed the primitive, and then someone put it on the wrong class.  Esteban and I have spent some hours trying to work around such issues.  I wish people would be more considerate.

If issues like this can be resolved we are very close to using the opensmalltalk-vm process for stable Pharo VMs.  Esteban wants (and has built) a test build that tries to produce sources and generate Vs and runs tests every time VMMaker is committed.  On the Squeak side we only try and build VMs when opensmalltalk-vm is committed in git.  I don't want to stand in Esteban's way.  I *do* want stable VMs to be built from the opensmalltalk-vm tree.

_,,,^..^,,,_
best, Eliot


+1
The faster we get the feedback the better.
So the Pharo VM has to be built on opensmalltalk-vm

If Pharo people wants to have a fork for mastering their release cycle, (the officially released VM) that's understandable.
If Pharo people wants to have a better automation with VMMaker code generation (maybe in a dev branch) and non regression tests that's all good.

It's just that we should integrate back any improvment and fix ASAP.
It would be even better to commit those fix in opensmalltalk-vm directly (feature branch and/or pull request)

BTW, don't forget Ronie as a referent Pharo VM developer :)

Yes.  I was kicking myself as soon as I hit send.  Forgive me, Ronie.


And I certainly mispelled, it must be Ronnie
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] ftp repositories broken

EstebanLM
In reply to this post by Nicolas Cellier
 

On 4 Apr 2017, at 20:56, Nicolas Cellier <[hidden email]> wrote:



2017-04-04 20:03 GMT+02:00 Eliot Miranda <[hidden email]>:
 


On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]> wrote:

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

Not yet.  We are close. Last week I got angry with Esteban because I thought that the joe was stalled because Pharo didn't want to move to opensmalltalk-vm, but I over reacted.  One of the issues preventing the move was indeed this primitive and the fact that someone, without thinking to talk to anyone working with the VM, renamed the primitive, and then someone put it on the wrong class.  Esteban and I have spent some hours trying to work around such issues.  I wish people would be more considerate.

If issues like this can be resolved we are very close to using the opensmalltalk-vm process for stable Pharo VMs.  Esteban wants (and has built) a test build that tries to produce sources and generate Vs and runs tests every time VMMaker is committed.  On the Squeak side we only try and build VMs when opensmalltalk-vm is committed in git.  I don't want to stand in Esteban's way.  I *do* want stable VMs to be built from the opensmalltalk-vm tree.

_,,,^..^,,,_
best, Eliot


+1
The faster we get the feedback the better.
So the Pharo VM has to be built on opensmalltalk-vm

If Pharo people wants to have a fork for mastering their release cycle, (the officially released VM) that's understandable.
If Pharo people wants to have a better automation with VMMaker code generation (maybe in a dev branch) and non regression tests that's all good.

It's just that we should integrate back any improvment and fix ASAP.
It would be even better to commit those fix in opensmalltalk-vm directly (feature branch and/or pull request)

BTW, don't forget Ronie as a referent Pharo VM developer :)

we all agree with that. 
thing is: 

- we still want to have a CI process running which covers all the stages of VM development: source generation, compilation, test. 
- we have a different packaging policy (basically we push to different places).

now, I do not see why that cannot coexist with opensmalltalk-vm, after all the work made this year (and believe me, it was A LOT of work). 

Esteban

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] ftp repositories broken

Eliot Miranda-2
In reply to this post by Eliot Miranda-2
 


On Apr 5, 2017, at 1:35 AM, Denis Kudriashov <[hidden email]> wrote:

Hi guys.

I could not accept your explanation because I have test which works on latest squeakVM and not works on latest pharoVM. Try filein attachment and check it on both VMs. Test not depends on any difference between squeak and pharo images. You run it on both systems (not only VM's).
I found that #primitiveFindSubstring not depends on receiver and you can put it on any class in system (in github C code there is no users of "rcvr" variable). So I just copy primitive method in test class.

But I could imaging that I not see/know something trivial. I will be glad to expand my mind :) but I see only reason: primitive is different (or absent) on latest pharoVM. 


Can we not simply compare the sources of src/plugins/MiscPrimitivesPlugin/MiscPrimitivesPlugin.c in the two VMs?

(If we were using one set of sources this issue would be moot).


2017-04-04 19:03 GMT+02:00 Esteban Lorenzano <[hidden email]>:

On 4 Apr 2017, at 19:00, Denis Kudriashov <[hidden email]> wrote:


2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

stable works because the bug ByteString>>#find… instead of String>>#find… was introduced later.