ftp repositories broken

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

ftp repositories broken

John Brant-2
I can't use a ftp Monticello repository using:

Pharo6.0
Latest update: #60315
pharo-linux-x86_64threaded-201704041239-b3f27b3

It appears that the primitive for
findSubstringViaPrimitive:in:startingAt:matchTable: has changed so that
this method now throws an error instead of returning 0. The ftp code
eventually evaluates something similar to this:

| eol |
eol := String with: Character cr with: Character lf.
eol findSubstringViaPrimitive: eol in: (String new: 4096) startingAt: 0
matchTable: (String classPool at: #CaseSensitiveOrder)

This returns 0 with an older vm, but throws an error in a new vm. Of
course, it may make sense for it to throw an error since 0 is an invalid
start index. If that is the case, then SocketStream should be changed so
that it doesn't call indexOfSubCollection:startingAt: with a 0:

SocketStream>>upToAll: aStringOrByteArray limit: nBytes
        "..."

        | index sz result searchedSoFar |
        sz := aStringOrByteArray size.
        "Look in the current inBuffer first"
        index := inBuffer indexOfSubCollection: aStringOrByteArray
                                                startingAt: (lastRead - sz + 2 max: 1).

Adding the "max: 1" allows me to connect to the ftp repository.


John Brant

Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Denis Kudriashov
Hi John.

This issue is solved in latest image 19892. But it only fixes primitive fallback code. General problem is still in latest VM 19893

2017-04-04 15:18 GMT+02:00 John Brant <[hidden email]>:
I can't use a ftp Monticello repository using:

Pharo6.0
Latest update: #60315
pharo-linux-x86_64threaded-201704041239-b3f27b3

It appears that the primitive for findSubstringViaPrimitive:in:startingAt:matchTable: has changed so that this method now throws an error instead of returning 0. The ftp code eventually evaluates something similar to this:

| eol |
eol := String with: Character cr with: Character lf.
eol findSubstringViaPrimitive: eol in: (String new: 4096) startingAt: 0 matchTable: (String classPool at: #CaseSensitiveOrder)

This returns 0 with an older vm, but throws an error in a new vm. Of course, it may make sense for it to throw an error since 0 is an invalid start index. If that is the case, then SocketStream should be changed so that it doesn't call indexOfSubCollection:startingAt: with a 0:

SocketStream>>upToAll: aStringOrByteArray limit: nBytes
        "..."

        | index sz result searchedSoFar |
        sz := aStringOrByteArray size.
        "Look in the current inBuffer first"
        index := inBuffer indexOfSubCollection: aStringOrByteArray
                                                startingAt: (lastRead - sz + 2 max: 1).

Adding the "max: 1" allows me to connect to the ftp repository.


John Brant


Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Denis Kudriashov

2017-04-04 15:36 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi John.

This issue is solved in latest image 19892. But it only fixes primitive fallback code. General problem is still in latest VM 19893

Interesting that it works fine in Squeak VM
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

EstebanLM

On 4 Apr 2017, at 15:36, Denis Kudriashov <[hidden email]> wrote:


2017-04-04 15:36 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi John.

This issue is solved in latest image 19892. But it only fixes primitive fallback code. General problem is still in latest VM 19893

Interesting that it works fine in Squeak VM

no it doesn't: squeak does not have that primitive, that’s why it does not fails :)
I do not think the way out this problem is to restore the primitive (which has some problems to make it TBH), but to clean the code of this… 

Esteban
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

John Brant-2
On 04/04/2017 08:47 AM, Esteban Lorenzano wrote:

>
>> On 4 Apr 2017, at 15:36, Denis Kudriashov <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>
>> 2017-04-04 15:36 GMT+02:00 Denis Kudriashov <[hidden email]
>> <mailto:[hidden email]>>:
>>
>>     Hi John.
>>
>>     This issue is solved in latest image 19892
>>     <https://pharo.fogbugz.com/f/cases/19892/Socket-problem-with-LatestVM>.
>>     But it only fixes primitive fallback code. General problem is
>>     still in latest VM 19893
>>     <https://pharo.fogbugz.com/f/cases/19893/primitiveFindSubstring-primitive-is-always-failed-on-latestVM>

Sorry, I thought I had the latest image, but I unzipped it in a
different directory.

>> Interesting that it works fine in Squeak VM
>
> no it doesn't: squeak does not have that primitive, that’s why it does
> not fails :)
> I do not think the way out this problem is to restore the primitive
> (which has some problems to make it TBH), but to clean the code of this…

I don't know about the latest Squeak VM, but a Squeak VM from last year
has it:

/home/brant/Smalltalk/Pharo64/lib/squeak/5.0-3732/squeak
CoInterpreter VMMaker.oscog-eem.1872 uuid:
6db6d610-b1a5-4f4d-978d-22c917bdb3e4 May 26 2016

When I use that VM, it works -- I'm running the same Pharo image file.
If you debug it, the debugger steps over the method (implying that it is
a primitive).


John Brant

Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Denis Kudriashov
In reply to this post by EstebanLM

2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email]>:
no it doesn't: squeak does not have that primitive, that’s why it does not fails :)

I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Ben Coman


On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>>
>> no it doesn't: squeak does not have that primitive, that’s why it does not fails :)
>
>
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

I believe Esteban's point Its not so much whether the VM has it
(since they are essentially the same VM + extra C-libs for Pharo)
but whether the Squeak Image is using it.

Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String.
That was added to Squeak 2015.05.01.  Squeak String seems to have never used the primitive (since 2005). 
 
A diff shows some differences in implementation..  
   https://www.diffchecker.com/CmBJZjB6
which anyway wouldn't account for John's observations.

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

EstebanLM

On 4 Apr 2017, at 18:01, Ben Coman <[hidden email]> wrote:



On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>>
>> no it doesn't: squeak does not have that primitive, that’s why it does not fails :)
>
>
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)
look, this is the way where it is generated: 

translatedPrimitives
"an assorted list of various primitives"
(ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil:
[^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:"
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(String findSubstringViaPrimitive:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:))].
^#(
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(ByteString findSubstring:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:)
)

means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. 
Now, there is a bug there (because source is in String and not in ByteString), and I can fix that by asking correctly, but: 

1) this means that sources for pharo and squeak will be different and pharo needs to be generated in pharo. This is contrary to what we are trying to do with the VM: we want all sources to be the same, and differences should come from compilation flags and/or the presence of certain plugins (this is because is a lot easier to debug the VM like that). 
2) I wonder… why we are using this primitive an squeak is not? maybe we need to deprecate the use?

I believe Esteban's point Its not so much whether the VM has it
(since they are essentially the same VM + extra C-libs for Pharo)
but whether the Squeak Image is using it.

Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String.
That was added to Squeak 2015.05.01.  Squeak String seems to have never used the primitive (since 2005). 

this is the important thing, I think… so… why can’t we use that instead the other one?
what’s the difference?

cheers!
Esteban


 
A diff shows some differences in implementation..  
   https://www.diffchecker.com/CmBJZjB6
which anyway wouldn't account for John's observations.

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Denis Kudriashov

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
still is not there, because is not the same primitive :)
look, this is the way where it is generated: 

translatedPrimitives
"an assorted list of various primitives"
(ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil:
[^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:"
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(String findSubstringViaPrimitive:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:))].
^#(
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(ByteString findSubstring:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:)
)

means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. 

Ok maybe I not understand anything :) But to clarify lets look at image side methods.
in Squeak:

ByteString>>findSubstring: key in: body startingAt: start matchTable: matchTable
<primitive: 'primitiveFindSubstring' module: 'MiscPrimitivePlugin'>

in Pharo:

String>>findSubstringViaPrimitive: key in: body startingAt: start matchTable: matchTable
<primitive: 'primitiveFindSubstring' module: 'MiscPrimitivePlugin'>

So to me they both uses same VM primitive primitiveFindSubstring. Does they are really different inside VM? 

Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Denis Kudriashov
In reply to this post by Ben Coman

2017-04-04 18:01 GMT+02:00 Ben Coman <[hidden email]>:

I believe Esteban's point Its not so much whether the VM has it
(since they are essentially the same VM + extra C-libs for Pharo)
but whether the Squeak Image is using it.

Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String.
That was added to Squeak 2015.05.01.  Squeak String seems to have never used the primitive (since 2005). 

You can make experiment. Replace fallback code with "self primitiveFailed" and call this method from comment examples. It works in SqueakVM. And it fails with latest PharoVM.

Actually just remove this method from ByteString. And many debuggers will approve that method is used

Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Eliot Miranda-2
In reply to this post by EstebanLM
Hi Esteban,


On Apr 4, 2017, at 9:45 AM, Esteban Lorenzano <[hidden email]> wrote:


On 4 Apr 2017, at 18:01, Ben Coman <[hidden email]> wrote:



On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>>
>> no it doesn't: squeak does not have that primitive, that’s why it does not fails :)
>
>
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)
look, this is the way where it is generated: 

translatedPrimitives
"an assorted list of various primitives"
(ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil:
[^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:"
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(String findSubstringViaPrimitive:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:))].
^#(
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(ByteString findSubstring:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:)
)

means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. 
Now, there is a bug there (because source is in String and not in ByteString), and I can fix that by asking correctly, but: 

1) this means that sources for pharo and squeak will be different and pharo needs to be generated in pharo. This is contrary to what we are trying to do with the VM: we want all sources to be the same, and differences should come from compilation flags and/or the presence of certain plugins (this is because is a lot easier to debug the VM like that). 
2) I wonder… why we are using this primitive an squeak is not? maybe we need to deprecate the use?

I believe Esteban's point Its not so much whether the VM has it
(since they are essentially the same VM + extra C-libs for Pharo)
but whether the Squeak Image is using it.

Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String.
That was added to Squeak 2015.05.01.  Squeak String seems to have never used the primitive (since 2005). 

this is the important thing, I think… so… why can’t we use that instead the other one?
what’s the difference?

You have identified the problem.  The bug is that Pharo broke this by renaming and repositioning the primitive to findSubstringViaPrimitive:in:startingAt:matchTable:.  If it had stayed where it was things would still be working.

So my suggestions are
a) restore Pharo to using by the primitive correctly in the exactly the same configuration as Squeak
b) no one in the Pharo community changes the definition or position of a primitive without consulting a vm person (and Esteban is a vm person as is Clément and arguably Guille).
c) we measure the performance of the primitive and the equivalent non-primitive code in StackInterpreter, Cog and Sista configurations (using Spur, the current object representation) for a range of strings and see how much benefit we're getter by from the primitive; maybe we can nuke it.

But yes, it is a bad bug to position this primitive in String; it works only for ByteString.


cheers!
Esteban


 
A diff shows some differences in implementation..  
   https://www.diffchecker.com/CmBJZjB6
which anyway wouldn't account for John's observations.

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Denis Kudriashov
In reply to this post by EstebanLM

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

EstebanLM

On 4 Apr 2017, at 19:00, Denis Kudriashov <[hidden email]> wrote:


2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

stable works because the bug ByteString>>#find… instead of String>>#find… was introduced later. 

Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Eliot Miranda-2
In reply to this post by Denis Kudriashov
Hi Denis,

On Tue, Apr 4, 2017 at 9:53 AM, Denis Kudriashov <[hidden email]> wrote:

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
still is not there, because is not the same primitive :)
look, this is the way where it is generated: 

translatedPrimitives
"an assorted list of various primitives"
(ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil:
[^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:"
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(String findSubstringViaPrimitive:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:))].
^#(
(Bitmap compress:toByteArray:)
(Bitmap decompress:fromByteArray:at:)
(Bitmap encodeBytesOf:in:at:)
(Bitmap encodeInt:in:at:)
(ByteString compare:with:collated:)
(ByteString translate:from:to:table:)
(ByteString findFirstInString:inSet:startingAt:)
(ByteString indexOfAscii:inString:startingAt:)
(ByteString findSubstring:in:startingAt:matchTable:)
(ByteArray hashBytes:startingWith:)
(SampledSound convert8bitSignedFrom:to16Bit:)
)

means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. 

Ok maybe I not understand anything :) But to clarify lets look at image side methods.
in Squeak:

ByteString>>findSubstring: key in: body startingAt: start matchTable: matchTable
<primitive: 'primitiveFindSubstring' module: 'MiscPrimitivePlugin'>

in Pharo:

String>>findSubstringViaPrimitive: key in: body startingAt: start matchTable: matchTable
<primitive: 'primitiveFindSubstring' module: 'MiscPrimitivePlugin'>

So to me they both uses same VM primitive primitiveFindSubstring. Does they are really different inside VM? 

No they're not different; they're the same.  But the primitive works only with ByteString, so it should not be on String where it will try and work on WideString and return completely incorrect results.

--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Eliot Miranda-2
In reply to this post by Denis Kudriashov


On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]> wrote:

2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

Not yet.  We are close. Last week I got angry with Esteban because I thought that the joe was stalled because Pharo didn't want to move to opensmalltalk-vm, but I over reacted.  One of the issues preventing the move was indeed this primitive and the fact that someone, without thinking to talk to anyone working with the VM, renamed the primitive, and then someone put it on the wrong class.  Esteban and I have spent some hours trying to work around such issues.  I wish people would be more considerate.

If issues like this can be resolved we are very close to using the opensmalltalk-vm process for stable Pharo VMs.  Esteban wants (and has built) a test build that tries to produce sources and generate Vs and runs tests every time VMMaker is committed.  On the Squeak side we only try and build VMs when opensmalltalk-vm is committed in git.  I don't want to stand in Esteban's way.  I *do* want stable VMs to be built from the opensmalltalk-vm tree.

_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Denis Kudriashov
In reply to this post by EstebanLM
Hi guys.

I could not accept your explanation because I have test which works on latest squeakVM and not works on latest pharoVM. Try filein attachment and check it on both VMs. Test not depends on any difference between squeak and pharo images. You run it on both systems (not only VM's).
I found that #primitiveFindSubstring not depends on receiver and you can put it on any class in system (in github C code there is no users of "rcvr" variable). So I just copy primitive method in test class.

But I could imaging that I not see/know something trivial. I will be glad to expand my mind :) but I see only reason: primitive is different (or absent) on latest pharoVM. 

2017-04-04 19:03 GMT+02:00 Esteban Lorenzano <[hidden email]>:

On 4 Apr 2017, at 19:00, Denis Kudriashov <[hidden email]> wrote:


2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

stable works because the bug ByteString>>#find… instead of String>>#find… was introduced later. 



FindStringPrimitiveTests.st (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Henrik Sperre Johansen
In reply to this post by Eliot Miranda-2
Eliot Miranda-2 wrote
On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]>
wrote:

>
> 2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>
>> > I think you are wrong because I check it carefully in Squeak and I
>> found it on github https://github.com/OpenSmallta
>> lk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.
>>
>>
>> still is not there, because is not the same primitive :)
>>
>
> Does it also means that latest VM is built differently then stable?
> (primitive works on stable pharo). I just thought that we already moved to
> opensmalltalk build process.
>

Not yet.  We are close. Last week I got angry with Esteban because I
thought that the joe was stalled because Pharo didn't want to move to
opensmalltalk-vm, but I over reacted.  One of the issues preventing the
move was indeed this primitive and the fact that someone, without thinking
to talk to anyone working with the VM, renamed the primitive, and then
someone put it on the wrong class.  Esteban and I have spent some hours
trying to work around such issues.  I wish people would be more considerate.
,,,^..^,,,_
best, Eliot
You'd have to talk to the Sophie people ;)
http://forum.world.st/VMMaker-Inlining-loop-with-MiscPrimitivesPlugin-on-Pharo-tp360979p361075.html

And "not talking to anyone working on the vm" is somewhat of an exaggeration, there were multiple other threads on the topic at the time, including the one where Andreas fixed the bug in the primitive the workaround wrapper/rename was meant to avoid;
http://forum.world.st/latest-vmmaker-wont-run-with-pharo-1-1-1-dev-td3030221.html

IIRC, I kept checking periodically if stable released vm's including the fix were available on all platforms it so the workaround could be removed, but after a year and a half or so without a new stable windows release, I sort of gave up/forgot about it :(

+1 to renaming it back now and removing the fluff in the Pharo image.
It's been 7 years since the issue in the primitive (which the extra wrapping of findSubstring: worked around) was fixed, and since opensmalltalk-vm got off the ground, there hasn't been a VM in general use that didn't include it.

Cheers,
Henry
Reply | Threaded
Open this post in threaded view
|

Re: ftp repositories broken

Eliot Miranda-2
In reply to this post by Denis Kudriashov


On Apr 5, 2017, at 1:35 AM, Denis Kudriashov <[hidden email]> wrote:

Hi guys.

I could not accept your explanation because I have test which works on latest squeakVM and not works on latest pharoVM. Try filein attachment and check it on both VMs. Test not depends on any difference between squeak and pharo images. You run it on both systems (not only VM's).
I found that #primitiveFindSubstring not depends on receiver and you can put it on any class in system (in github C code there is no users of "rcvr" variable). So I just copy primitive method in test class.

But I could imaging that I not see/know something trivial. I will be glad to expand my mind :) but I see only reason: primitive is different (or absent) on latest pharoVM. 


Can we not simply compare the sources of src/plugins/MiscPrimitivesPlugin/MiscPrimitivesPlugin.c in the two VMs?

(If we were using one set of sources this issue would be moot).


2017-04-04 19:03 GMT+02:00 Esteban Lorenzano <[hidden email]>:

On 4 Apr 2017, at 19:00, Denis Kudriashov <[hidden email]> wrote:


2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
> I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=.

still is not there, because is not the same primitive :)

Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process.

stable works because the bug ByteString>>#find… instead of String>>#find… was introduced later. 


Reply | Threaded
Open this post in threaded view
|

Software maintenance costs in the real world (Re: ftp repositories broken)

David T. Lewis
In reply to this post by EstebanLM
For students of software engineering, I offer this as an example of one of the
most common and costly types of error that you will encounter in real life.

A small error may be introduced early in the development process, possibly
slipping through the cracks through lack of review (as was the case in this
example). The change is motivated by good intentions, but some aspect of the
overall system design is overlooked.

Once the mistake has been made, it becomes increasingly difficult to change
as time goes by, and increasingly costly to support as new people need to
understand the issue and work around its consequences. Over the life of the
software, the accumulated costs of effort to understand and address the
problem (typically budgeted separately under "software maintenance") far
exceed the costs associated with the original change (typically budgeted
as "development").

In addition to cost of maintenance, this kind of error can also defeat the
intentions of the original developer. In the example we have here, the
original change was motivated by an interest in making the system cleaner
and easier to understand. But nearly a decade later, we see workarounds
such as the #translatedPrimitives method below that are far worse than the
original problem that someone was trying to solve.

For the historical background on this particular exammple, see the earlier
post by Henrik Sperre Johansen in this thread.

Dave

p.s. I am not a software engineer :-)


On Tue, Apr 04, 2017 at 06:45:27PM +0200, Esteban Lorenzano wrote:

>
> > On 4 Apr 2017, at 18:01, Ben Coman <[hidden email]> wrote:
> >
> >
> >
> > On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <[hidden email] <mailto:[hidden email]>> wrote:
> > >
> > >
> > > 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email] <mailto:[hidden email]>>:
> > >>
> > >> no it doesn't: squeak does not have that primitive, that???s why it does not fails :)
> > >
> > >
> > > I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=???&q=primitiveFindSubstring&type= <https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=%E2%9C%93&q=primitiveFindSubstring&type=>.
>
> still is not there, because is not the same primitive :)
> look, this is the way where it is generated:
>
> translatedPrimitives
> "an assorted list of various primitives"
> (ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil:
> [^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:"
> (Bitmap compress:toByteArray:)
> (Bitmap decompress:fromByteArray:at:)
> (Bitmap encodeBytesOf:in:at:)
> (Bitmap encodeInt:in:at:)
> (ByteString compare:with:collated:)
> (ByteString translate:from:to:table:)
> (ByteString findFirstInString:inSet:startingAt:)
> (ByteString indexOfAscii:inString:startingAt:)
> (String findSubstringViaPrimitive:in:startingAt:matchTable:)
> (ByteArray hashBytes:startingWith:)
> (SampledSound convert8bitSignedFrom:to16Bit:))].
> ^#(
> (Bitmap compress:toByteArray:)
> (Bitmap decompress:fromByteArray:at:)
> (Bitmap encodeBytesOf:in:at:)
> (Bitmap encodeInt:in:at:)
> (ByteString compare:with:collated:)
> (ByteString translate:from:to:table:)
> (ByteString findFirstInString:inSet:startingAt:)
> (ByteString indexOfAscii:inString:startingAt:)
> (ByteString findSubstring:in:startingAt:matchTable:)
> (ByteArray hashBytes:startingWith:)
> (SampledSound convert8bitSignedFrom:to16Bit:)
> )
>
> means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:.
> Now, there is a bug there (because source is in String and not in ByteString), and I can fix that by asking correctly, but:
>
> 1) this means that sources for pharo and squeak will be different and pharo needs to be generated in pharo. This is contrary to what we are trying to do with the VM: we want all sources to be the same, and differences should come from compilation flags and/or the presence of certain plugins (this is because is a lot easier to debug the VM like that).
> 2) I wonder??? why we are using this primitive an squeak is not? maybe we need to deprecate the use?
>
> > I believe Esteban's point Its not so much whether the VM has it
> > (since they are essentially the same VM + extra C-libs for Pharo)
> > but whether the Squeak Image is using it.
> >
> > Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String.
> > That was added to Squeak 2015.05.01.  Squeak String seems to have never used the primitive (since 2005).
>
> this is the important thing, I think??? so??? why can???t we use that instead the other one?
> what???s the difference?
>
> cheers!
> Esteban
>
>
> >  
> > A diff shows some differences in implementation..  
> >    https://www.diffchecker.com/CmBJZjB6 <https://www.diffchecker.com/CmBJZjB6>
> > which anyway wouldn't account for John's observations.
> >
> > cheers -ben
>