issue with large file path on windows

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

issue with large file path on windows

Nicolai Hess
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai



Reply | Threaded
Open this post in threaded view
|

Re: issue with large file path on windows

Tudor Girba-2
This is great news!

I had tons of problems last year with that and I did not know what to do about this.

Doru

On Sun, Aug 23, 2015 at 1:44 PM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--

"Every thing has its own flow"
Reply | Threaded
Open this post in threaded view
|

Re: issue with large file path on windows

stepharo
In reply to this post by Nicolai Hess
Thanks for trying to fix this issue!


Le 23/8/15 13:44, Nicolai Hess a écrit :

> For those who had problems with pharo on windows and github based
> repositories,
> I built a windows vm with support for long paths:
>
> https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing
>
>
> For browsing directories with large paths (FileList or Inspect),
> you may need one additional change in the image (But I am not really
> sure about that) :
>
> DiskStore>>initialize
>     super initialize.
>     maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
>
>
> please test and give feedback.
>
> This wasn't as easy as I thought, and I had to make some more changes
> for the file permissions (the stat-functions don't work for files with
> long paths).
> Please test other file/folder operations.
>
>
> nicolai
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: issue with large file path on windows

Nicolai Hess
In reply to this post by Nicolai Hess
And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename


2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai




Reply | Threaded
Open this post in threaded view
|

Re: issue with large file path on windows

hernanmd
In reply to this post by Nicolai Hess
Nicolai

2015-08-23 8:44 GMT-03:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.


Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore.

However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :(

Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limitation such Win Unicode API would let VM to use the 32767 bricks


This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


Thank you Nicolai for your effort, I know it's hard to dive into depths with C.
 
Hernán


nicolai




Reply | Threaded
Open this post in threaded view
|

Re: issue with large file path on windows

Nicolai Hess
Hi Hernán
Thanks for your feedback

2015-08-24 8:13 GMT+02:00 Hernán Morales Durand <[hidden email]>:
Nicolai

2015-08-23 8:44 GMT-03:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.


Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore.

However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :(

I tried Aconcagua from Catalogbrowser in pharo 5.0 and it worked for me.
Can you show what path is the base path, the working directory of your pharo package-cache/github directory. Is there something special in your path name or the disk?
 

Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limitation such Win Unicode API would let VM to use the 32767 bricks

I am not sure, we are using all file/directory operations that should use the unicode version with multibyte char (CreateFileW / FindFirstFileW / ...)
 


This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


Thank you Nicolai for your effort, I know it's hard to dive into depths with C.
 
Hernán


nicolai





Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] issue with large file path on windows

Henrik Sperre Johansen
In reply to this post by Nicolai Hess

On 23 Aug 2015, at 6:09 , Nicolai Hess <[hidden email]> wrote:

And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename


2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai





+ #define CONVERT_MULTIBYTE_TO_WIDECHAR_PATH(buffer, size, fileNameString, fileNameLength) { \
+ buffer = (WCHAR*)alloca((size+4+1)*sizeof(WCHAR));\
+ buffer[0] = L'\\';buffer[1] = L'\\'; buffer[2] = L'?'; buffer[3] = L'\\';\
+ MultiByteToWideChar(CP_UTF8, 0, fileNameString, fileNameLength, buffer + 4, size);\
+ buffer[size + 4] = 0;\
+ size += 4;}


- Is an alloca version really worth it for the potential problems?
- Also, LONG_PATH_PREFIX_SIZE is defined above in the header, should probably use this instead of 4 in the last two lines?
- The comment on lines 111/164/265/381, while not modified, are somewhat incorrect, null-terminated C-string indicates UTF8, which we are converting from, not into.

I'm curious what the "<a href="smb://?/" class="">\\?\" prefix does to hasCaseSensitiveDuplicate (adding a new ? segment to the path), but I cba to understand that code, and considering
int caseSensitiveFileMode = 0;
if(!caseSensitiveFileMode) return 0;

I guess it doesn't really matter.

Otherwise, looks good to me!

Cheers,
Henry
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] issue with large file path on windows

Nicolai Hess


2015-08-24 13:34 GMT+02:00 Henrik Johansen <[hidden email]>:

On 23 Aug 2015, at 6:09 , Nicolai Hess <[hidden email]> wrote:

And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename


2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






Thank you Henrik, for looking at this

 
+ #define CONVERT_MULTIBYTE_TO_WIDECHAR_PATH(buffer, size, fileNameString, fileNameLength) { \
+ buffer = (WCHAR*)alloca((size+4+1)*sizeof(WCHAR));\
+ buffer[0] = L'\\';buffer[1] = L'\\'; buffer[2] = L'?'; buffer[3] = L'\\';\
+ MultiByteToWideChar(CP_UTF8, 0, fileNameString, fileNameLength, buffer + 4, size);\
+ buffer[size + 4] = 0;\
+ size += 4;}


- Is an alloca version really worth it for the potential problems?

It is what Marcel used for the long path support in the squeak vm (I would have copy and paste the whole change, but I couldn't find the appropriate change for directory functions).
 
- Also, LONG_PATH_PREFIX_SIZE is defined above in the header, should probably use this instead of 4 in the last two lines?

Yes
 
- The comment on lines 111/164/265/381, while not modified, are somewhat incorrect, null-terminated C-string indicates UTF8, which we are converting from, not into.

Right, I ll chang that

I'm curious what the "\\?\" prefix does to hasCaseSensitiveDuplicate (adding a new ? segment to the path), but I cba to understand that code, and considering
int caseSensitiveFileMode = 0;
if(!caseSensitiveFileMode) return 0;

I guess it doesn't really matter.

Yes, I tried to make this change in a way that does not break the case sensitive duplicate check, but even without my change,
enabling the caseSenstiveFileMode makes the windows vm unusable. So, at least, it is no worse.

 

Otherwise, looks good to me!

Cheers,
Henry

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] issue with large file path on windows

Ben Coman


On Mon, Aug 24, 2015 at 8:44 PM, Nicolai Hess <[hidden email]> wrote:


2015-08-24 13:34 GMT+02:00 Henrik Johansen <[hidden email]>:

On 23 Aug 2015, at 6:09 , Nicolai Hess <[hidden email]> wrote:

And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename


2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


- Is an alloca version really worth it for the potential problems?

It is what Marcel used for the long path support in the squeak vm (I would have copy and paste the whole change, but I couldn't find the appropriate change for directory functions).
 

 "Potential problems" sounded like something interesting I could learn.  Here is a summary... 

On the use and abuse of alloca
  * alloca five times faster than malloc (but is that necessary given file operations likely take longer anyway?)
  * most concerns with alloca no problem here e.g. its a very short lifespan; the pointer is not returned from the function. 

The Perils of alloca()
  * remaining concern is potential stack exhaustion...
     * is MAX_LONG_FILE_PATH=32kB significant today (but also consider embedded application) ?
     * counter argument, primitive functions are not likely to entered recursively ?

Microsoft advises alloca [1] deprecated in favour of malloca [2] which allocates either stack or heap depending on requested size

An example of a DIY malloca...
  * might have the advantage of tuning MAX_ALLOCA_CUTOFF post-compilation
  * minGW gcc might not have malloca
  * if large allocations go on the heap, maybe MAX_LONG_FILE_PATH test not required, just MAX_ALLOCA_CUTOFF

----------
btw, perhaps rename CONVERT_MULTIBYTE_TO_WIDECHAR_PATH
to ALLOC_WIDECHAR_PATH_FROM_MULTIBYTE or similar 
so its more obviously symmetrical with FREE_WIDECHAR_PATH.

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: issue with large file path on windows

Nicolai Hess
In reply to this post by Nicolai Hess
Oh, no. Long path prefix can not work for relative paths!

Since my latest additions for long path support for the windows vm, you can not
open an image file if the supplied path is not an absolute path.

If you start Pharo with "Pharo.exe Pharo.image" an open-file-dialog shows up and you have to choose
the image in this dialog. (If you have only one image file in the current directory, this image *is* found,
this is because the windows vm searches the current directory for a single .image file)

Sorry for that, I'll fix it.


nicolai


2015-09-04 22:19 GMT+02:00 Nicolai Hess <[hidden email]>:
Thank you all for your feedback.
@ben,
- yes malloca is not supported by mingw ( or I did not get it to work)
- and we don't have to consider recursive calls
I think we are save by using alloca. I did some tests with awfully long path names.

I made some more little changes for retrieving the file permissions. Don't use the stat() function, because
this did not work for long path names ( and it doesn't worked well for paths with non ascii characters).

I created a pull request for the pharo-vm git project.

nicolai



2015-08-25 8:35 GMT+02:00 Hernán Morales Durand <[hidden email]>:


2015-08-24 4:31 GMT-03:00 Nicolai Hess <[hidden email]>:
Hi Hernán
Thanks for your feedback

2015-08-24 8:13 GMT+02:00 Hernán Morales Durand <[hidden email]>:
Nicolai

2015-08-23 8:44 GMT-03:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.


Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore.

However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :(

I tried Aconcagua from Catalogbrowser in pharo 5.0 and it worked for me.
Can you show what path is the base path, the working directory of your pharo package-cache/github directory. Is there something special in your path name or the disk?

Nicolai, the path is the one which I reported here http://forum.world.st/Loading-Chalten-amp-Aconcagua-error-td4844694.html
I have nothing special in the path name, not special characters or so.

I tried today again with your VM, same path, same image, but before loading the configuration I removed the whole github-cache/* contents, and it worked.... so I am confused. So I checked with old VM again and yep the FileDoesNotExists came back. Conclusion: Your VM actually fixes the issue, but cleaning github-cache did something, there could be another bug there...

 
 

Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limitation such Win Unicode API would let VM to use the 32767 bricks

I am not sure, we are using all file/directory operations that should use the unicode version with multibyte char (CreateFileW / FindFirstFileW / ...)
 

Ok, thanks for checking.
 


This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


Thank you Nicolai for your effort, I know it's hard to dive into depths with C.
 
Hernán


nicolai








Reply | Threaded
Open this post in threaded view
|

Re: issue with large file path on windows

Eliot Miranda-2
In reply to this post by Nicolai Hess
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Nicolai Hess


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).
But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?

I would love if someone else got this working.

nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Eliot Miranda-2
Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this.  I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions.  I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
#   define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?


I'm sorry I didn't see your message.  Did you ask on vm-dev?

 
I would love if someone else got this working.

:-)  I hear you :-)

 

nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Nicolai Hess


2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this.  I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions.  I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
#   define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution
 

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?


Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there
be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?


 

I'm sorry I didn't see your message.  Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list
 

 
I would love if someone else got this working.

:-)  I hear you :-)

 

nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Nicolai Hess


2015-09-11 9:43 GMT+02:00 Nicolai Hess <[hidden email]>:


2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this.  I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions.  I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
#   define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution
 

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?


Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there
be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?

Ok, found it. It is in the trunk branch :)

 


 

I'm sorry I didn't see your message.  Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list
 

 
I would love if someone else got this working.

:-)  I hear you :-)

 

nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot



Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Nicolai Hess


2015-09-11 9:44 GMT+02:00 Nicolai Hess <[hidden email]>:


2015-09-11 9:43 GMT+02:00 Nicolai Hess <[hidden email]>:


2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this.  I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions.  I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
#   define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution
 

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?


Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there
be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?

Ok, found it. It is in the trunk branch :)


No, wait, now I am confused. The change to sqWin32FilePrims.c is only in the trunk branch and the change to sqWin32Directory.c is
only in the Cog branch. What branch includes the solution for both (sqWin32Directory.c and sqWin32FilePrims.c) and should be
used for a merge into pharos vm repository?

 

 


 

I'm sorry I didn't see your message.  Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list
 

 
I would love if someone else got this working.

:-)  I hear you :-)

 

nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot




Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Eliot Miranda-2
Hi Nicolai,

On Fri, Sep 11, 2015 at 12:48 AM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-11 9:44 GMT+02:00 Nicolai Hess <[hidden email]>:


2015-09-11 9:43 GMT+02:00 Nicolai Hess <[hidden email]>:


2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this.  I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions.  I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
#   define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution
 

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?


Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there
be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?

Ok, found it. It is in the trunk branch :)


No, wait, now I am confused. The change to sqWin32FilePrims.c is only in the trunk branch and the change to sqWin32Directory.c is
only in the Cog branch. What branch includes the solution for both (sqWin32Directory.c and sqWin32FilePrims.c) and should be
used for a merge into pharos vm repository?

There are two "externals" in the Cog branch:

X       platforms/win32/plugins
X       platforms/Cross/plugins




OK?


 

 


 

I'm sorry I didn't see your message.  Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list
 

 
I would love if someone else got this working.

:-)  I hear you :-)

 

nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot








--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Nicolai Hess
In reply to this post by Eliot Miranda-2


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source.
I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the
squeak vm main branch (wrapped with pharo vm ifdefs).

I'll create a push request.


nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Eliot Miranda-2
Thanks Nicolai, that's great!  I'll merge in your changes.  Which files do I need to look at?

On Sat, Sep 12, 2015 at 8:00 AM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source.
I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the
squeak vm main branch (wrapped with pharo vm ifdefs).

I'll create a push request.


nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: issue with large file path on windows

Nicolai Hess


2015-09-14 23:10 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Thanks Nicolai, that's great!  I'll merge in your changes.  Which files do I need to look at?

I changed (removed my code, added Marcels)
platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c
platforms/win32/vm/sqWin32.h
platforms/win32/vm/sqWin32Directory.c

I added (should be the same as in squeak vm repository)
platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c


But my changes only about that code that already is in squeaks repository.

If we want to merge some of the pharo-vm code into squeaks branch, we
first need to wrap that code with some #ifdefs

(For the win32 file/directory stuff, this would be code in
platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c
platforms/win32/vm/sqWin32.h
platforms/win32/vm/sqWin32Directory.c
and
platforms/Cross/plugins/FilePlugin/FilePlugin.h)

Maybe Estaban should look at this, and tell which path he would follow  to
merge those changes.



 

On Sat, Sep 12, 2015 at 8:00 AM, Nicolai Hess <[hidden email]> wrote:
 


2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolai,

    I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog.  Long filename support for win32 was recently added by Marcel Taumel in May of this year.  Are you tracking that?  How can we keep the sources harmonized?

Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source.
I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the
squeak vm main branch (wrapped with pharo vm ifdefs).

I'll create a push request.


nicolai

 

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai






--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot


12