Windows 64bit, long filename issue but after a second load, how to categorise and report this?
Locked
Windows 64bit, long filename issue but after a second load, how to categorise and report this?
 
|
We noticed a weird problem when testing the pharo exercism project.
When loading on Windows10 in a fresh 64bit image with the following evaluate in the playground: Metacello new baseline: 'Exercism'; repository: 'github://exercism/pharo-smalltalk:master/dev/src'; load. The project seemed to load correctly (but we seemed to hit an issue with an http submit giving a 404 error - which weirdly works in 32bit and OSX 64/32). Anyway - the real question here is that when trying to reproduce this in a second fresh 64bit image 2 users now have reported getting the long filename error: LGit_GIT_EEXISTS: Failed to stat file 'C:/Users/sam/Documents/Pharo/images/exercism-test-run-7.0-32...stance/testRaiseUnexpectedMessageWhenTooMuchMessages.st': The filename or extension is too long. And the second user got the same but with a different filename: ruleRBUtlitiesMethodsRuleV1FalsePositve.st I know there is a long standing problem with Windows and file path lengths - but why would it work the first time and then subsequently fail in fresh installs? Is something being cached on windows that isn’t on other platforms? (Writing this - I’m wondering if there is that shared pharo Iceberg setting for Share repos that might be implicated?) Also - given Exercism is in Tonel format and those files aren’t in exercism, and in fact I’m struggling to find where they are defined (my images don’t have them) - there is something very weird going on. I wonder if we have another error that is masking all of this? Ultimately my question is - where is the Pharo issue that is tracking this problem? I’ve seen it mentioned lots of times here, but can’t see anything in the issues that I can tack the above onto? For reference - our exercism issue is: https://github.com/exercism/pharo-smalltalk/issues/140 Tim |
|
|
On Thu, 21 Feb 2019 at 20:36, Tim Mackinnon <[hidden email]> wrote: We noticed a weird problem when testing the pharo exercism project. These files are both method names which indicates FileTree format rather than Tonel (and these "failed to stat file" errors on Windows is one of the reasons for the move to Tonel). It may be from one of the dependencies that are pulled in. To identify which, search for that file under `pharo-local/iceberg` directory. I believe the repo may have been cloned successfully, its just the libgit binary can't `stat` long filenames. If that turns out the be untrue, use commandline git to complete the clone and you will likely see it.
as above. (my images don’t have them) - there is something very weird going on. Filetree on Windows is constrained by long filename issues with libgit. cheers -ben |
Re: Windows 64bit, long filename issue but after a second load, how to categorise and report this?
|
|
Hi Ben - while I understand the description below - why would it work the first time (no errors) and then fail the second time in a fresh image? Are we really sure its just down to non-determinism of the load process (possibly - but it seems very suspicious to me). There is something more fishy about windows going on I suspect…
I will track down the Exercism dependencies to see if any of them are FileTree - having said that, on OSX (where it loads cleanly - I would expect to be able to find one of those methods in my image - but again I can’t? They look like valid methods - but why aren’t they in my image? Its like the load process is downloading other stuff on its own and then not loading that into the image? I’m just worried that we keep blaming windows (and yes its dumb having a file length restriction I this day and age) - but maybe we are missing another problem too? Tim
|
Re: Windows 64bit, long filename issue but after a second load, how to categorise and report this?
|
|
Tim Mackinnon wrote
> Hi Ben - while I understand the description below - why would it work the > first time (no errors) and then fail the second time in a fresh image? Are > we really sure its just down to non-determinism of the load process > (possibly - but it seems very suspicious to me). There is something more > fishy about windows going on I suspect… The Iceberg cache path is relative to the image folder, so it's at least possible to have success in one image, located in C:\Pharo\testImage\, while another image, reciding in something like C:\Users\sam\Documents\Pharo\images\exercism-test-run-7.0-32\ , fails to load the exact same filetree package... Tim Mackinnon wrote > I will track down the Exercism dependencies to see if any of them are > FileTree - having said that, on OSX (where it loads cleanly - I would > expect to be able to find one of those methods in my image - but again I > can’t? They look like valid methods - but why aren’t they in my image? Its > like the load process is downloading other stuff on its own and then not > loading that into the image? Yeaaah, that sounds fishy. But at least the first method sounds like a test, which may be part of the packages in the git repo that gets cloned without being loaded by Metacello, if the Metacello dependency specifies a *Core group which does not include tests. Tim Mackinnon wrote > I’m just worried that we keep blaming windows (and yes its dumb having a > file length restriction I this day and age) - but maybe we are missing > another problem too? > > Tim To be pendantic, (and not of much actual help, sorry :/ ) the problem isn't really with Windows, which provides API's for getting file info that handle long paths just fine, but with libgit, which is probably compiled against some POSIX stat implementation provided by Cygwin/mingw, which does not. Cheers, Henry -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html |
Re: Windows 64bit, long filename issue but after a second load, how to categorise and report this?
|
|
Some interesting thoughts below - but my understanding is these images are being created by Pharo Launcher , so they should be isolated from one another?
I also don’t understand why you can load with 32bit (multiple images) but only once with 64bit. Do we load radically different git code with longer paths on 64bit? (Strange isn’t it?) Possibly worth caveating this is with 2-3 users so far, and only noticed this pattern with the last one. Also - has anyone created an issue for these path problems - I’m curious if my results match anything else. Tim Sent from my iPhone > On 21 Feb 2019, at 16:01, Henrik Sperre Johansen <[hidden email]> wrote: > > Tim Mackinnon wrote >> Hi Ben - while I understand the description below - why would it work the >> first time (no errors) and then fail the second time in a fresh image? Are >> we really sure its just down to non-determinism of the load process >> (possibly - but it seems very suspicious to me). There is something more >> fishy about windows going on I suspect… > > The Iceberg cache path is relative to the image folder, so it's at least > possible to have success in one image, located in C:\Pharo\testImage\, while > another image, reciding in something like > C:\Users\sam\Documents\Pharo\images\exercism-test-run-7.0-32\ , fails to > load the exact same filetree package... > > > Tim Mackinnon wrote >> I will track down the Exercism dependencies to see if any of them are >> FileTree - having said that, on OSX (where it loads cleanly - I would >> expect to be able to find one of those methods in my image - but again I >> can’t? They look like valid methods - but why aren’t they in my image? Its >> like the load process is downloading other stuff on its own and then not >> loading that into the image? > > Yeaaah, that sounds fishy. > But at least the first method sounds like a test, which may be part of the > packages in the git repo that gets cloned without being loaded by Metacello, > if the Metacello dependency specifies a *Core group which does not include > tests. > > > Tim Mackinnon wrote >> I’m just worried that we keep blaming windows (and yes its dumb having a >> file length restriction I this day and age) - but maybe we are missing >> another problem too? >> >> Tim > > To be pendantic, (and not of much actual help, sorry :/ ) the problem isn't > really with Windows, which provides API's for getting file info that handle > long paths just fine, but with libgit, which is probably compiled against > some POSIX stat implementation provided by Cygwin/mingw, which does not. > > Cheers, > Henry > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > |
Re: Windows 64bit, long filename issue but after a second load, how to categorise and report this?
|
|
Just to add some additional notes on this (and I will also file an issue, as I don’t think one has been submitted, and I don’t think this is fully down to file path length alone) -
After some more testing, we have also seen the error on Windows 32bit as well (so while easier to get in 64bit, it does occur in 32bit like others have reported). My real concern however, is that a simple project (which admittedly pulls in a single Filetree prerequisite - PharoWebBrowser) seems to trigger this error - but confusingly references files that don’t appear to be in the image. Tracing through PharoWebBrowser - there are some file tree based pre-reques and some that are mcz - and none of them have files like: testRaiseUnexpectedMessageWhenTooMuchMessages.st And this method is not in the image that gets loaded into OSX. So I am wondering where these long files are coming from? So yes, while there is a long filename issue - is there another hidden issue underneath all of this where files we aren’t even using are getting reference somehow? Tim > On 22 Feb 2019, at 00:01, Tim Mackinnon <[hidden email]> wrote: > > Some interesting thoughts below - but my understanding is these images are being created by Pharo Launcher , so they should be isolated from one another? > > I also don’t understand why you can load with 32bit (multiple images) but only once with 64bit. Do we load radically different git code with longer paths on 64bit? (Strange isn’t it?) > > Possibly worth caveating this is with 2-3 users so far, and only noticed this pattern with the last one. > > Also - has anyone created an issue for these path problems - I’m curious if my results match anything else. > > Tim > > Sent from my iPhone > >> On 21 Feb 2019, at 16:01, Henrik Sperre Johansen <[hidden email]> wrote: >> >> Tim Mackinnon wrote >>> Hi Ben - while I understand the description below - why would it work the >>> first time (no errors) and then fail the second time in a fresh image? Are >>> we really sure its just down to non-determinism of the load process >>> (possibly - but it seems very suspicious to me). There is something more >>> fishy about windows going on I suspect… >> >> The Iceberg cache path is relative to the image folder, so it's at least >> possible to have success in one image, located in C:\Pharo\testImage\, while >> another image, reciding in something like >> C:\Users\sam\Documents\Pharo\images\exercism-test-run-7.0-32\ , fails to >> load the exact same filetree package... >> >> >> Tim Mackinnon wrote >>> I will track down the Exercism dependencies to see if any of them are >>> FileTree - having said that, on OSX (where it loads cleanly - I would >>> expect to be able to find one of those methods in my image - but again I >>> can’t? They look like valid methods - but why aren’t they in my image? Its >>> like the load process is downloading other stuff on its own and then not >>> loading that into the image? >> >> Yeaaah, that sounds fishy. >> But at least the first method sounds like a test, which may be part of the >> packages in the git repo that gets cloned without being loaded by Metacello, >> if the Metacello dependency specifies a *Core group which does not include >> tests. >> >> >> Tim Mackinnon wrote >>> I’m just worried that we keep blaming windows (and yes its dumb having a >>> file length restriction I this day and age) - but maybe we are missing >>> another problem too? >>> >>> Tim >> >> To be pendantic, (and not of much actual help, sorry :/ ) the problem isn't >> really with Windows, which provides API's for getting file info that handle >> long paths just fine, but with libgit, which is probably compiled against >> some POSIX stat implementation provided by Cygwin/mingw, which does not. >> >> Cheers, >> Henry >> >> >> >> -- >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >> > > |
Re: Windows 64bit, long filename issue but after a second load, how to categorise and report this?
|
|
I’ve created - https://github.com/pharo-vcs/iceberg/issues/1204
> On 25 Feb 2019, at 17:12, Tim Mackinnon <[hidden email]> wrote: > > Just to add some additional notes on this (and I will also file an issue, as I don’t think one has been submitted, and I don’t think this is fully down to file path length alone) - > > After some more testing, we have also seen the error on Windows 32bit as well (so while easier to get in 64bit, it does occur in 32bit like others have reported). > > My real concern however, is that a simple project (which admittedly pulls in a single Filetree prerequisite - PharoWebBrowser) seems to trigger this error - but confusingly references files that don’t appear to be in the image. Tracing through PharoWebBrowser - there are some file tree based pre-reques and some that are mcz - and none of them have files like: testRaiseUnexpectedMessageWhenTooMuchMessages.st > > And this method is not in the image that gets loaded into OSX. So I am wondering where these long files are coming from? > > So yes, while there is a long filename issue - is there another hidden issue underneath all of this where files we aren’t even using are getting reference somehow? > > Tim > >> On 22 Feb 2019, at 00:01, Tim Mackinnon <[hidden email]> wrote: >> >> Some interesting thoughts below - but my understanding is these images are being created by Pharo Launcher , so they should be isolated from one another? >> >> I also don’t understand why you can load with 32bit (multiple images) but only once with 64bit. Do we load radically different git code with longer paths on 64bit? (Strange isn’t it?) >> >> Possibly worth caveating this is with 2-3 users so far, and only noticed this pattern with the last one. >> >> Also - has anyone created an issue for these path problems - I’m curious if my results match anything else. >> >> Tim >> >> Sent from my iPhone >> >>> On 21 Feb 2019, at 16:01, Henrik Sperre Johansen <[hidden email]> wrote: >>> >>> Tim Mackinnon wrote >>>> Hi Ben - while I understand the description below - why would it work the >>>> first time (no errors) and then fail the second time in a fresh image? Are >>>> we really sure its just down to non-determinism of the load process >>>> (possibly - but it seems very suspicious to me). There is something more >>>> fishy about windows going on I suspect… >>> >>> The Iceberg cache path is relative to the image folder, so it's at least >>> possible to have success in one image, located in C:\Pharo\testImage\, while >>> another image, reciding in something like >>> C:\Users\sam\Documents\Pharo\images\exercism-test-run-7.0-32\ , fails to >>> load the exact same filetree package... >>> >>> >>> Tim Mackinnon wrote >>>> I will track down the Exercism dependencies to see if any of them are >>>> FileTree - having said that, on OSX (where it loads cleanly - I would >>>> expect to be able to find one of those methods in my image - but again I >>>> can’t? They look like valid methods - but why aren’t they in my image? Its >>>> like the load process is downloading other stuff on its own and then not >>>> loading that into the image? >>> >>> Yeaaah, that sounds fishy. >>> But at least the first method sounds like a test, which may be part of the >>> packages in the git repo that gets cloned without being loaded by Metacello, >>> if the Metacello dependency specifies a *Core group which does not include >>> tests. >>> >>> >>> Tim Mackinnon wrote >>>> I’m just worried that we keep blaming windows (and yes its dumb having a >>>> file length restriction I this day and age) - but maybe we are missing >>>> another problem too? >>>> >>>> Tim >>> >>> To be pendantic, (and not of much actual help, sorry :/ ) the problem isn't >>> really with Windows, which provides API's for getting file info that handle >>> long paths just fine, but with libgit, which is probably compiled against >>> some POSIX stat implementation provided by Cygwin/mingw, which does not. >>> >>> Cheers, >>> Henry >>> >>> >>> >>> -- >>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >>> >> >> > > |
Re: Windows 64bit, long filename issue but after a second load, how to categorise and report this?
|
|
I’ve been able to find out more information about this - and have now got the full filename and error in question:
`'C:/Users/sam/Documents/Pharo/images/exercism-test-run-7.0-32bit/pharo-local/iceberg/dionisiydk/StateSpecs/StateSpecs-DSL-PragmaBasedShouldExpressions-Tests.package/SpecOfShouldExpressionDriverTests.class/instance/testRaiseUnexpectedMessageWhenTooMuchMessages.st': The filename or extension is too long.` So the path is 262 characters long. (hence the error) The package this file is in, is a downstream project prerequisite for PharoWebBrowser - which is dragged in because that project doesn't have different groupings for code and tests (something I am working with Sean to try to get resolved - although I’m scratching my head on how you can test baselines?). We don't see this in the image in OSX because the included baseline doesn't actually pre-req these pragma based tests? However from below, have I now understood that when we reference a project, we end up cloning the whole repository for that repo on disk (so one big read), and then metacello picks out the relevant files from the baseline. So possibly, this might be some orphaned files/packages in the repo - that iceberg no longer knows about, but git does? The longer term fix is to convert PharoWebBrowser to Tonel and see if we can improve the grouping. Tim > On 25 Feb 2019, at 17:19, Tim Mackinnon <[hidden email]> wrote: > > I’ve created - https://github.com/pharo-vcs/iceberg/issues/1204 > >> On 25 Feb 2019, at 17:12, Tim Mackinnon <[hidden email]> wrote: >> >> Just to add some additional notes on this (and I will also file an issue, as I don’t think one has been submitted, and I don’t think this is fully down to file path length alone) - >> >> After some more testing, we have also seen the error on Windows 32bit as well (so while easier to get in 64bit, it does occur in 32bit like others have reported). >> >> My real concern however, is that a simple project (which admittedly pulls in a single Filetree prerequisite - PharoWebBrowser) seems to trigger this error - but confusingly references files that don’t appear to be in the image. Tracing through PharoWebBrowser - there are some file tree based pre-reques and some that are mcz - and none of them have files like: testRaiseUnexpectedMessageWhenTooMuchMessages.st >> >> And this method is not in the image that gets loaded into OSX. So I am wondering where these long files are coming from? >> >> So yes, while there is a long filename issue - is there another hidden issue underneath all of this where files we aren’t even using are getting reference somehow? >> >> Tim >> >>> On 22 Feb 2019, at 00:01, Tim Mackinnon <[hidden email]> wrote: >>> >>> Some interesting thoughts below - but my understanding is these images are being created by Pharo Launcher , so they should be isolated from one another? >>> >>> I also don’t understand why you can load with 32bit (multiple images) but only once with 64bit. Do we load radically different git code with longer paths on 64bit? (Strange isn’t it?) >>> >>> Possibly worth caveating this is with 2-3 users so far, and only noticed this pattern with the last one. >>> >>> Also - has anyone created an issue for these path problems - I’m curious if my results match anything else. >>> >>> Tim >>> >>> Sent from my iPhone >>> >>>> On 21 Feb 2019, at 16:01, Henrik Sperre Johansen <[hidden email]> wrote: >>>> >>>> Tim Mackinnon wrote >>>>> Hi Ben - while I understand the description below - why would it work the >>>>> first time (no errors) and then fail the second time in a fresh image? Are >>>>> we really sure its just down to non-determinism of the load process >>>>> (possibly - but it seems very suspicious to me). There is something more >>>>> fishy about windows going on I suspect… >>>> >>>> The Iceberg cache path is relative to the image folder, so it's at least >>>> possible to have success in one image, located in C:\Pharo\testImage\, while >>>> another image, reciding in something like >>>> C:\Users\sam\Documents\Pharo\images\exercism-test-run-7.0-32\ , fails to >>>> load the exact same filetree package... >>>> >>>> >>>> Tim Mackinnon wrote >>>>> I will track down the Exercism dependencies to see if any of them are >>>>> FileTree - having said that, on OSX (where it loads cleanly - I would >>>>> expect to be able to find one of those methods in my image - but again I >>>>> can’t? They look like valid methods - but why aren’t they in my image? Its >>>>> like the load process is downloading other stuff on its own and then not >>>>> loading that into the image? >>>> >>>> Yeaaah, that sounds fishy. >>>> But at least the first method sounds like a test, which may be part of the >>>> packages in the git repo that gets cloned without being loaded by Metacello, >>>> if the Metacello dependency specifies a *Core group which does not include >>>> tests. >>>> >>>> >>>> Tim Mackinnon wrote >>>>> I’m just worried that we keep blaming windows (and yes its dumb having a >>>>> file length restriction I this day and age) - but maybe we are missing >>>>> another problem too? >>>>> >>>>> Tim >>>> >>>> To be pendantic, (and not of much actual help, sorry :/ ) the problem isn't >>>> really with Windows, which provides API's for getting file info that handle >>>> long paths just fine, but with libgit, which is probably compiled against >>>> some POSIX stat implementation provided by Cygwin/mingw, which does not. >>>> >>>> Cheers, >>>> Henry >>>> >>>> >>>> >>>> -- >>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >>>> >>> >>> >> >> > > |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |