ftp repositories broken
Locked
 
|
I can't use a ftp Monticello repository using:
Pharo6.0 Latest update: #60315 pharo-linux-x86_64threaded-201704041239-b3f27b3 It appears that the primitive for findSubstringViaPrimitive:in:startingAt:matchTable: has changed so that this method now throws an error instead of returning 0. The ftp code eventually evaluates something similar to this: | eol | eol := String with: Character cr with: Character lf. eol findSubstringViaPrimitive: eol in: (String new: 4096) startingAt: 0 matchTable: (String classPool at: #CaseSensitiveOrder) This returns 0 with an older vm, but throws an error in a new vm. Of course, it may make sense for it to throw an error since 0 is an invalid start index. If that is the case, then SocketStream should be changed so that it doesn't call indexOfSubCollection:startingAt: with a 0: SocketStream>>upToAll: aStringOrByteArray limit: nBytes "..." | index sz result searchedSoFar | sz := aStringOrByteArray size. "Look in the current inBuffer first" index := inBuffer indexOfSubCollection: aStringOrByteArray startingAt: (lastRead - sz + 2 max: 1). Adding the "max: 1" allows me to connect to the ftp repository. John Brant |
|
|
Hi John. 2017-04-04 15:18 GMT+02:00 John Brant <[hidden email]>: I can't use a ftp Monticello repository using: |
|
|
Interesting that it works fine in Squeak VM |
|
|
no it doesn't: squeak does not have that primitive, that’s why it does not fails :) I do not think the way out this problem is to restore the primitive (which has some problems to make it TBH), but to clean the code of this… Esteban
|
|
|
On 04/04/2017 08:47 AM, Esteban Lorenzano wrote:
> >> On 4 Apr 2017, at 15:36, Denis Kudriashov <[hidden email] >> <mailto:[hidden email]>> wrote: >> >> >> 2017-04-04 15:36 GMT+02:00 Denis Kudriashov <[hidden email] >> <mailto:[hidden email]>>: >> >> Hi John. >> >> This issue is solved in latest image 19892 >> <https://pharo.fogbugz.com/f/cases/19892/Socket-problem-with-LatestVM>. >> But it only fixes primitive fallback code. General problem is >> still in latest VM 19893 >> <https://pharo.fogbugz.com/f/cases/19893/primitiveFindSubstring-primitive-is-always-failed-on-latestVM> Sorry, I thought I had the latest image, but I unzipped it in a different directory. >> Interesting that it works fine in Squeak VM > > no it doesn't: squeak does not have that primitive, that’s why it does > not fails :) > I do not think the way out this problem is to restore the primitive > (which has some problems to make it TBH), but to clean the code of this… I don't know about the latest Squeak VM, but a Squeak VM from last year has it: /home/brant/Smalltalk/Pharo64/lib/squeak/5.0-3732/squeak CoInterpreter VMMaker.oscog-eem.1872 uuid: 6db6d610-b1a5-4f4d-978d-22c917bdb3e4 May 26 2016 When I use that VM, it works -- I'm running the same Pharo image file. If you debug it, the debugger steps over the method (implying that it is a primitive). John Brant |
|
|
In reply to this post by EstebanLM
2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email]>: no it doesn't: squeak does not have that primitive, that’s why it does not fails :) I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=. |
|
|
On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <[hidden email]> wrote: > > > 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email]>: >> >> no it doesn't: squeak does not have that primitive, that’s why it does not fails :) > > > I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=✓&q=primitiveFindSubstring&type=. I believe Esteban's point Its not so much whether the VM has it (since they are essentially the same VM + extra C-libs for Pharo) but whether the Squeak Image is using it. Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String. That was added to Squeak 2015.05.01. Squeak String seems to have never used the primitive (since 2005). which anyway wouldn't account for John's observations. cheers -ben |
|
|
still is not there, because is not the same primitive :) look, this is the way where it is generated: translatedPrimitives "an assorted list of various primitives" (ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil: [^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:" (Bitmap compress:toByteArray:) (Bitmap decompress:fromByteArray:at:) (Bitmap encodeBytesOf:in:at:) (Bitmap encodeInt:in:at:) (ByteString compare:with:collated:) (ByteString translate:from:to:table:) (ByteString findFirstInString:inSet:startingAt:) (ByteString indexOfAscii:inString:startingAt:) (String findSubstringViaPrimitive:in:startingAt:matchTable:) (ByteArray hashBytes:startingWith:) (SampledSound convert8bitSignedFrom:to16Bit:))]. ^#( (Bitmap compress:toByteArray:) (Bitmap decompress:fromByteArray:at:) (Bitmap encodeBytesOf:in:at:) (Bitmap encodeInt:in:at:) (ByteString compare:with:collated:) (ByteString translate:from:to:table:) (ByteString findFirstInString:inSet:startingAt:) (ByteString indexOfAscii:inString:startingAt:) (ByteString findSubstring:in:startingAt:matchTable:) (ByteArray hashBytes:startingWith:) (SampledSound convert8bitSignedFrom:to16Bit:) ) means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. Now, there is a bug there (because source is in String and not in ByteString), and I can fix that by asking correctly, but: 1) this means that sources for pharo and squeak will be different and pharo needs to be generated in pharo. This is contrary to what we are trying to do with the VM: we want all sources to be the same, and differences should come from compilation flags and/or the presence of certain plugins (this is because is a lot easier to debug the VM like that). 2) I wonder… why we are using this primitive an squeak is not? maybe we need to deprecate the use?
this is the important thing, I think… so… why can’t we use that instead the other one? what’s the difference? cheers! Esteban
|
|
|
2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
Ok maybe I not understand anything :) But to clarify lets look at image side methods. in Squeak:
in Pharo:
So to me they both uses same VM primitive primitiveFindSubstring. Does they are really different inside VM? |
|
|
In reply to this post by Ben Coman
2017-04-04 18:01 GMT+02:00 Ben Coman <[hidden email]>:
You can make experiment. Replace fallback code with "self primitiveFailed" and call this method from comment examples. It works in SqueakVM. And it fails with latest PharoVM. Actually just remove this method from ByteString. And many debuggers will approve that method is used |
|
|
In reply to this post by EstebanLM
Hi Esteban,
So my suggestions are a) restore Pharo to using by the primitive correctly in the exactly the same configuration as Squeak b) no one in the Pharo community changes the definition or position of a primitive without consulting a vm person (and Esteban is a vm person as is Clément and arguably Guille). c) we measure the performance of the primitive and the equivalent non-primitive code in StackInterpreter, Cog and Sista configurations (using Spur, the current object representation) for a range of strings and see how much benefit we're getter by from the primitive; maybe we can nuke it. But yes, it is a bad bug to position this primitive in String; it works only for ByteString.
|
|
|
In reply to this post by EstebanLM
2017-04-04 18:45 GMT+02:00 Esteban Lorenzano <[hidden email]>:
Does it also means that latest VM is built differently then stable? (primitive works on stable pharo). I just thought that we already moved to opensmalltalk build process. |
|
|
stable works because the bug ByteString>>#find… instead of String>>#find… was introduced later. |
|
|
In reply to this post by Denis Kudriashov
Hi Denis,
On Tue, Apr 4, 2017 at 9:53 AM, Denis Kudriashov <[hidden email]> wrote: --
No they're not different; they're the same. But the primitive works only with ByteString, so it should not be on String where it will try and work on WideString and return completely incorrect results. _,,,^..^,,,_ best, Eliot |
|
|
In reply to this post by Denis Kudriashov
On Tue, Apr 4, 2017 at 10:00 AM, Denis Kudriashov <[hidden email]> wrote:
Not yet. We are close. Last week I got angry with Esteban because I thought that the joe was stalled because Pharo didn't want to move to opensmalltalk-vm, but I over reacted. One of the issues preventing the move was indeed this primitive and the fact that someone, without thinking to talk to anyone working with the VM, renamed the primitive, and then someone put it on the wrong class. Esteban and I have spent some hours trying to work around such issues. I wish people would be more considerate. If issues like this can be resolved we are very close to using the opensmalltalk-vm process for stable Pharo VMs. Esteban wants (and has built) a test build that tries to produce sources and generate Vs and runs tests every time VMMaker is committed. On the Squeak side we only try and build VMs when opensmalltalk-vm is committed in git. I don't want to stand in Esteban's way. I *do* want stable VMs to be built from the opensmalltalk-vm tree. _,,,^..^,,,_ best, Eliot |
|
|
In reply to this post by EstebanLM
Hi guys. I could not accept your explanation because I have test which works on latest squeakVM and not works on latest pharoVM. Try filein attachment and check it on both VMs. Test not depends on any difference between squeak and pharo images. You run it on both systems (not only VM's). I found that #primitiveFindSubstring not depends on receiver and you can put it on any class in system (in github C code there is no users of "rcvr" variable). So I just copy primitive method in test class. But I could imaging that I not see/know something trivial. I will be glad to expand my mind :) but I see only reason: primitive is different (or absent) on latest pharoVM. 2017-04-04 19:03 GMT+02:00 Esteban Lorenzano <[hidden email]>:
|
FindStringPrimitiveTests.st (1K) 
Re: ftp repositories broken
|
|
In reply to this post by Eliot Miranda-2
You'd have to talk to the Sophie people ;) http://forum.world.st/VMMaker-Inlining-loop-with-MiscPrimitivesPlugin-on-Pharo-tp360979p361075.html And "not talking to anyone working on the vm" is somewhat of an exaggeration, there were multiple other threads on the topic at the time, including the one where Andreas fixed the bug in the primitive the workaround wrapper/rename was meant to avoid; http://forum.world.st/latest-vmmaker-wont-run-with-pharo-1-1-1-dev-td3030221.html IIRC, I kept checking periodically if stable released vm's including the fix were available on all platforms it so the workaround could be removed, but after a year and a half or so without a new stable windows release, I sort of gave up/forgot about it :( +1 to renaming it back now and removing the fluff in the Pharo image. It's been 7 years since the issue in the primitive (which the extra wrapping of findSubstring: worked around) was fixed, and since opensmalltalk-vm got off the ground, there hasn't been a VM in general use that didn't include it. Cheers, Henry |
|
|
In reply to this post by Denis Kudriashov
(If we were using one set of sources this issue would be moot).
|
Software maintenance costs in the real world (Re: ftp repositories broken)
|
|
In reply to this post by EstebanLM
For students of software engineering, I offer this as an example of one of the
most common and costly types of error that you will encounter in real life. A small error may be introduced early in the development process, possibly slipping through the cracks through lack of review (as was the case in this example). The change is motivated by good intentions, but some aspect of the overall system design is overlooked. Once the mistake has been made, it becomes increasingly difficult to change as time goes by, and increasingly costly to support as new people need to understand the issue and work around its consequences. Over the life of the software, the accumulated costs of effort to understand and address the problem (typically budgeted separately under "software maintenance") far exceed the costs associated with the original change (typically budgeted as "development"). In addition to cost of maintenance, this kind of error can also defeat the intentions of the original developer. In the example we have here, the original change was motivated by an interest in making the system cleaner and easier to understand. But nearly a decade later, we see workarounds such as the #translatedPrimitives method below that are far worse than the original problem that someone was trying to solve. For the historical background on this particular exammple, see the earlier post by Henrik Sperre Johansen in this thread. Dave p.s. I am not a software engineer :-) On Tue, Apr 04, 2017 at 06:45:27PM +0200, Esteban Lorenzano wrote: > > > On 4 Apr 2017, at 18:01, Ben Coman <[hidden email]> wrote: > > > > > > > > On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <[hidden email] <mailto:[hidden email]>> wrote: > > > > > > > > > 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <[hidden email] <mailto:[hidden email]>>: > > >> > > >> no it doesn't: squeak does not have that primitive, that???s why it does not fails :) > > > > > > > > > I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=???&q=primitiveFindSubstring&type= <https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=%E2%9C%93&q=primitiveFindSubstring&type=>. > > still is not there, because is not the same primitive :) > look, this is the way where it is generated: > > translatedPrimitives > "an assorted list of various primitives" > (ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil: > [^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:" > (Bitmap compress:toByteArray:) > (Bitmap decompress:fromByteArray:at:) > (Bitmap encodeBytesOf:in:at:) > (Bitmap encodeInt:in:at:) > (ByteString compare:with:collated:) > (ByteString translate:from:to:table:) > (ByteString findFirstInString:inSet:startingAt:) > (ByteString indexOfAscii:inString:startingAt:) > (String findSubstringViaPrimitive:in:startingAt:matchTable:) > (ByteArray hashBytes:startingWith:) > (SampledSound convert8bitSignedFrom:to16Bit:))]. > ^#( > (Bitmap compress:toByteArray:) > (Bitmap decompress:fromByteArray:at:) > (Bitmap encodeBytesOf:in:at:) > (Bitmap encodeInt:in:at:) > (ByteString compare:with:collated:) > (ByteString translate:from:to:table:) > (ByteString findFirstInString:inSet:startingAt:) > (ByteString indexOfAscii:inString:startingAt:) > (ByteString findSubstring:in:startingAt:matchTable:) > (ByteArray hashBytes:startingWith:) > (SampledSound convert8bitSignedFrom:to16Bit:) > ) > > means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. > Now, there is a bug there (because source is in String and not in ByteString), and I can fix that by asking correctly, but: > > 1) this means that sources for pharo and squeak will be different and pharo needs to be generated in pharo. This is contrary to what we are trying to do with the VM: we want all sources to be the same, and differences should come from compilation flags and/or the presence of certain plugins (this is because is a lot easier to debug the VM like that). > 2) I wonder??? why we are using this primitive an squeak is not? maybe we need to deprecate the use? > > > I believe Esteban's point Its not so much whether the VM has it > > (since they are essentially the same VM + extra C-libs for Pharo) > > but whether the Squeak Image is using it. > > > > Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String. > > That was added to Squeak 2015.05.01. Squeak String seems to have never used the primitive (since 2005). > > this is the important thing, I think??? so??? why can???t we use that instead the other one? > what???s the difference? > > cheers! > Esteban > > > > > > A diff shows some differences in implementation.. > > https://www.diffchecker.com/CmBJZjB6 <https://www.diffchecker.com/CmBJZjB6> > > which anyway wouldn't account for John's observations. > > > > cheers -ben > |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |