Smalltalk › Pharo › Pharo Smalltalk Developers

issue with large file path on windows

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

25 messages Options

Nicolai Hess

issue with large file path on windows

For those who had problems with pharo on windows and github based repositories,

I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes

for the file permissions (the stat-functions don't work for files with long paths).

Please test other file/folder operations.

nicolai

Tudor Girba-2

Re: issue with large file path on windows

This is great news!

I had tons of problems last year with that and I did not know what to do about this.

Doru

On Sun, Aug 23, 2015 at 1:44 PM, Nicolai Hess <[hidden email]> wrote:

For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

www.tudorgirba.com

"Every thing has its own flow"

stepharo

Re: issue with large file path on windows

In reply to this post by Nicolai Hess

Thanks for trying to fix this issue!

Le 23/8/15 13:44, Nicolai Hess a écrit :

> For those who had problems with pharo on windows and github based
> repositories,
> I built a windows vm with support for long paths:
>
> https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing
>
>
> For browsing directories with large paths (FileList or Inspect),
> you may need one additional change in the image (But I am not really
> sure about that) :
>
> DiskStore>>initialize
> super initialize.
> maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
>
>
> please test and give feedback.
>
> This wasn't as easy as I thought, and I had to make some more changes
> for the file permissions (the stat-functions don't work for files with
> long paths).
> Please test other file/folder operations.
>
>
> nicolai
>
>
>

Nicolai Hess

Re: issue with large file path on windows

In reply to this post by Nicolai Hess

And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename

2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:

For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

hernanmd

Re: issue with large file path on windows

In reply to this post by Nicolai Hess

Nicolai

2015-08-23 8:44 GMT-03:00 Nicolai Hess <[hidden email]>:

For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore.

However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :(

Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limitation such Win Unicode API would let VM to use the 32767 bricks

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

Thank you Nicolai for your effort, I know it's hard to dive into depths with C.

Hernán

nicolai

Nicolai Hess

Re: issue with large file path on windows

Hi Hernán

Thanks for your feedback

2015-08-24 8:13 GMT+02:00 Hernán Morales Durand <[hidden email]>:

Nicolai

2015-08-23 8:44 GMT-03:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore.

However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :(

I tried Aconcagua from Catalogbrowser in pharo 5.0 and it worked for me.

Can you show what path is the base path, the working directory of your pharo package-cache/github directory. Is there something special in your path name or the disk?

Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limitation such Win Unicode API would let VM to use the 32767 bricks

I am not sure, we are using all file/directory operations that should use the unicode version with multibyte char (CreateFileW / FindFirstFileW / ...)

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

Thank you Nicolai for your effort, I know it's hard to dive into depths with C.

Hernán

nicolai

Henrik Sperre Johansen

Re: [Pharo-users] issue with large file path on windows

 				In reply to this post by Nicolai Hess
			
	
		
On 23 Aug 2015, at 6:09 , Nicolai Hess <[hidden email]> wrote:

And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename


2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing


For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
    super initialize.
    maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].


please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.


nicolai







+  #define CONVERT_MULTIBYTE_TO_WIDECHAR_PATH(buffer, size, fileNameString, fileNameLength)  {	\
+    buffer = (WCHAR*)alloca((size+4+1)*sizeof(WCHAR));\
+    buffer[0] = L'\\';buffer[1] = L'\\'; buffer[2] = L'?'; buffer[3] = L'\\';\
+    MultiByteToWideChar(CP_UTF8, 0, fileNameString, fileNameLength, buffer + 4, size);\
+    buffer[size + 4] = 0;\
+    size += 4;}


- Is an alloca version really worth it for the potential problems?
- Also, LONG_PATH_PREFIX_SIZE is defined above in the header, should probably use this instead of 4 in the last two lines?
- The comment on lines 111/164/265/381, while not modified, are somewhat incorrect, null-terminated C-string indicates UTF8, which we are converting from, not into.

I'm curious what the "<a href="smb://?/" class="">\\?\" prefix does to hasCaseSensitiveDuplicate (adding a new ? segment to the path), but I cba to understand that code, and considering
int caseSensitiveFileMode = 0;
if(!caseSensitiveFileMode) return 0;

I guess it doesn't really matter.

Otherwise, looks good to me!

Cheers,
Henry
++  #define CONVERT_MULTIBYTE_TO_WIDECHAR_PATH(buffer, size, fileNameString, fileNameLength)  {	\
++    buffer = (WCHAR*)alloca((size+4+1)*sizeof(WCHAR));\
++    buffer[0] = L'\\';buffer[1] = L'\\'; buffer[2] = L'?'; buffer[3] = L'\\';\
++    MultiByteToWideChar(CP_UTF8, 0, fileNameString, fileNameLength, buffer + 4, size);\
++    buffer[size + 4] = 0;\
++    size += 4;}

Nicolai Hess

Re: [Pharo-users] issue with large file path on windows

2015-08-24 13:34 GMT+02:00 Henrik Johansen <[hidden email]>:

On 23 Aug 2015, at 6:09 , Nicolai Hess <[hidden email]> wrote:

And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename

2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

Thank you Henrik, for looking at this

+ #define CONVERT_MULTIBYTE_TO_WIDECHAR_PATH(buffer, size, fileNameString, fileNameLength) { \
+ buffer = (WCHAR*)alloca((size+4+1)*sizeof(WCHAR));\
+ buffer[0] = L'\\';buffer[1] = L'\\'; buffer[2] = L'?'; buffer[3] = L'\\';\
+ MultiByteToWideChar(CP_UTF8, 0, fileNameString, fileNameLength, buffer + 4, size);\
+ buffer[size + 4] = 0;\
+ size += 4;}

- Is an alloca version really worth it for the potential problems?

It is what Marcel used for the long path support in the squeak vm (I would have copy and paste the whole change, but I couldn't find the appropriate change for directory functions).

- Also, LONG_PATH_PREFIX_SIZE is defined above in the header, should probably use this instead of 4 in the last two lines?

Yes

- The comment on lines 111/164/265/381, while not modified, are somewhat incorrect, null-terminated C-string indicates UTF8, which we are converting from, not into.

Right, I ll chang that

I'm curious what the "\\?\" prefix does to hasCaseSensitiveDuplicate (adding a new ? segment to the path), but I cba to understand that code, and considering
int caseSensitiveFileMode = 0;
if(!caseSensitiveFileMode) return 0;

I guess it doesn't really matter.

Yes, I tried to make this change in a way that does not break the case sensitive duplicate check, but even without my change,

enabling the caseSenstiveFileMode makes the windows vm unusable. So, at least, it is no worse.

Otherwise, looks good to me!

Cheers,
Henry

Ben Coman

Re: [Pharo-users] issue with large file path on windows

On Mon, Aug 24, 2015 at 8:44 PM, Nicolai Hess <[hidden email]> wrote:

2015-08-24 13:34 GMT+02:00 Henrik Johansen <[hidden email]>:

On 23 Aug 2015, at 6:09 , Nicolai Hess <[hidden email]> wrote:

And If you want to review the changes:

https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-long-filename

2015-08-23 13:44 GMT+02:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

- Is an alloca version really worth it for the potential problems?

It is what Marcel used for the long path support in the squeak vm (I would have copy and paste the whole change, but I couldn't find the appropriate change for directory functions).

"Potential problems" sounded like something interesting I could learn. Here is a summary...

http://stackoverflow.com/questions/5807612/on-the-use-and-abuse-of-alloca

On the use and abuse of alloca

* alloca five times faster than malloc (but is that necessary given file operations likely take longer anyway?)

* most concerns with alloca no problem here e.g. its a very short lifespan; the pointer is not returned from the function.

http://www.strchr.com/alloca

The Perils of alloca()

* remaining concern is potential stack exhaustion...

* is MAX_LONG_FILE_PATH=32kB significant today (but also consider embedded application) ?

* counter argument, primitive functions are not likely to entered recursively ?

Microsoft advises alloca [1] deprecated in favour of malloca [2] which allocates either stack or heap depending on requested size

[1] https://msdn.microsoft.com/en-us/library/wb1s57t5(v=vs.80).aspx

[2] https://msdn.microsoft.com/en-us/library/5471dc8s(VS.80).aspx

https://sourceware.org/ml/libc-alpha/2012-06/msg00246.html

An example of a DIY malloca...

* might have the advantage of tuning MAX_ALLOCA_CUTOFF post-compilation

* minGW gcc might not have malloca

* if large allocations go on the heap, maybe MAX_LONG_FILE_PATH test not required, just MAX_ALLOCA_CUTOFF

----------

btw, perhaps rename CONVERT_MULTIBYTE_TO_WIDECHAR_PATH

to ALLOC_WIDECHAR_PATH_FROM_MULTIBYTE or similar

so its more obviously symmetrical with FREE_WIDECHAR_PATH.

cheers -ben

Nicolai Hess

Re: issue with large file path on windows

In reply to this post by Nicolai Hess

Oh, no. Long path prefix can not work for relative paths!

Since my latest additions for long path support for the windows vm, you can not
open an image file if the supplied path is not an absolute path.

If you start Pharo with "Pharo.exe Pharo.image" an open-file-dialog shows up and you have to choose
the image in this dialog. (If you have only one image file in the current directory, this image *is* found,
this is because the windows vm searches the current directory for a single .image file)

Sorry for that, I'll fix it.

nicolai

2015-09-04 22:19 GMT+02:00 Nicolai Hess <[hidden email]>:

Thank you all for your feedback.
@ben,
- yes malloca is not supported by mingw ( or I did not get it to work)
- and we don't have to consider recursive calls
I think we are save by using alloca. I did some tests with awfully long path names.

I made some more little changes for retrieving the file permissions. Don't use the stat() function, because
this did not work for long path names ( and it doesn't worked well for paths with non ascii characters).

I created a pull request for the pharo-vm git project.

nicolai

2015-08-25 8:35 GMT+02:00 Hernán Morales Durand <[hidden email]>:

2015-08-24 4:31 GMT-03:00 Nicolai Hess <[hidden email]>:
Hi Hernán
Thanks for your feedback

2015-08-24 8:13 GMT+02:00 Hernán Morales Durand <[hidden email]>:
Nicolai

2015-08-23 8:44 GMT-03:00 Nicolai Hess <[hidden email]>:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore.

However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :(

I tried Aconcagua from Catalogbrowser in pharo 5.0 and it worked for me.
Can you show what path is the base path, the working directory of your pharo package-cache/github directory. Is there something special in your path name or the disk?

Nicolai, the path is the one which I reported here http://forum.world.st/Loading-Chalten-amp-Aconcagua-error-td4844694.html
I have nothing special in the path name, not special characters or so.

I tried today again with your VM, same path, same image, but before loading the configuration I removed the whole github-cache/* contents, and it worked.... so I am confused. So I checked with old VM again and yep the FileDoesNotExists came back. Conclusion: Your VM actually fixes the issue, but cleaning github-cache did something, there could be another bug there...

Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limitation such Win Unicode API would let VM to use the 32767 bricks

I am not sure, we are using all file/directory operations that should use the unicode version with multibyte char (CreateFileW / FindFirstFileW / ...)

Ok, thanks for checking.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

Thank you Nicolai for your effort, I know it's hard to dive into depths with C.

Hernán

nicolai

Eliot Miranda-2

Re: issue with large file path on windows

In reply to this post by Nicolai Hess

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:

For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

_,,,^..^,,,_

best, Eliot

Nicolai Hess

Re: [Vm-dev] Re: issue with large file path on windows

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or

if it is not possible because we have already other changes (pharos file plugin

provides addition attributes (the file permissions)).

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change

that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find

the changed source, only an updated vm.

Is this change already comitted?

I would love if someone else got this working.

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

Eliot Miranda-2

Re: [Vm-dev] Re: issue with large file path on windows

Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out

#define PharoVM 0

...

#if PharoVM

instead of

#if !defined(PharoVM)

# define PharoVM 0

#endif

...

#if PharoVM

but I'll fix it v soon.

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?

Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Directory.c

I'm sorry I didn't see your message. Did you ask on vm-dev?

I would love if someone else got this working.

:-) I hear you :-)

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

_,,,^..^,,,_

best, Eliot

Nicolai Hess

Re: [Vm-dev] Re: issue with large file path on windows

2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
# define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?

Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Directory.c

Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there

be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?

I'm sorry I didn't see your message. Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list

I would love if someone else got this working.

:-) I hear you :-)

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

--
_,,,^..^,,,_
best, Eliot

Nicolai Hess

Re: [Vm-dev] Re: issue with large file path on windows

2015-09-11 9:43 GMT+02:00 Nicolai Hess <[hidden email]>:

2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
# define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?

Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Directory.c

Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there
be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?

Ok, found it. It is in the trunk branch :)

I'm sorry I didn't see your message. Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list

I would love if someone else got this working.

:-) I hear you :-)

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

--
_,,,^..^,,,_
best, Eliot

Nicolai Hess

Re: [Vm-dev] Re: issue with large file path on windows

2015-09-11 9:44 GMT+02:00 Nicolai Hess <[hidden email]>:

2015-09-11 9:43 GMT+02:00 Nicolai Hess <[hidden email]>:

2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
# define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?

Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Directory.c

Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there
be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?

Ok, found it. It is in the trunk branch :)

No, wait, now I am confused. The change to sqWin32FilePrims.c is only in the trunk branch and the change to sqWin32Directory.c is

only in the Cog branch. What branch includes the solution for both (sqWin32Directory.c and sqWin32FilePrims.c) and should be

used for a merge into pharos vm repository?

I'm sorry I didn't see your message. Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list

I would love if someone else got this working.

:-) I hear you :-)

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

--
_,,,^..^,,,_
best, Eliot

Eliot Miranda-2

Re: [Vm-dev] Re: issue with large file path on windows

Hi Nicolai,

On Fri, Sep 11, 2015 at 12:48 AM, Nicolai Hess <[hidden email]> wrote:

2015-09-11 9:44 GMT+02:00 Nicolai Hess <[hidden email]>:

2015-09-11 9:43 GMT+02:00 Nicolai Hess <[hidden email]>:

2015-09-10 23:07 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <[hidden email]> wrote:

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Yes, I know, and I already asked on the ML if we can take that change or
if it is not possible because we have already other changes (pharos file plugin
provides addition attributes (the file permissions)).

Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out

#define PharoVM 0
...
#if PharoVM

instead of

#if !defined(PharoVM)
# define PharoVM 0
#endif
...
#if PharoVM

but I'll fix it v soon.

OK, I'll try to copy marcels solution

But I got no response, so I tried to adopt Marcels change.

And I only found the change for sqWin32FilePrims.c, but missed the other change
that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find
the changed source, only an updated vm.
Is this change already comitted?

Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Directory.c

Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there
be a "plugins" folder in Cog/platforms/win32/?
Where are the changes to sqWin32FilePrims.c now?

Ok, found it. It is in the trunk branch :)

No, wait, now I am confused. The change to sqWin32FilePrims.c is only in the trunk branch and the change to sqWin32Directory.c is
only in the Cog branch. What branch includes the solution for both (sqWin32Directory.c and sqWin32FilePrims.c) and should be
used for a merge into pharos vm repository?

There are two "externals" in the Cog branch:

X platforms/win32/plugins

X platforms/Cross/plugins

So http://www.squeakvm.org/svn/squeak/branches/Cog is the Cog repository, but

http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/plugins is in fact

http://squeakvm.org/svn/squeak/trunk/platforms/win32/plugins

and http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/Cross/plugins is in fact

http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins

OK?

I'm sorry I didn't see your message. Did you ask on vm-dev?

I am not sure, maybe I only send it in the squeak-dev list

I would love if someone else got this working.

:-) I hear you :-)

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

--
_,,,^..^,,,_
best, Eliot

_,,,^..^,,,_

best, Eliot

Nicolai Hess

Re: [Vm-dev] Re: issue with large file path on windows

In reply to this post by Eliot Miranda-2

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source.

I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the

squeak vm main branch (wrapped with pharo vm ifdefs).

github branch: https://github.com/nicolaihess/pharo-vm/tree/harmonize_win_file_plugin

I'll create a push request.

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

Eliot Miranda-2

Re: [Vm-dev] Re: issue with large file path on windows

Thanks Nicolai, that's great! I'll merge in your changes. Which files do I need to look at?

On Sat, Sep 12, 2015 at 8:00 AM, Nicolai Hess <[hidden email]> wrote:

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source.
I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the
squeak vm main branch (wrapped with pharo vm ifdefs).

github branch: https://github.com/nicolaihess/pharo-vm/tree/harmonize_win_file_plugin
I'll create a push request.

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

_,,,^..^,,,_

best, Eliot

Nicolai Hess

Re: [Vm-dev] Re: issue with large file path on windows

2015-09-14 23:10 GMT+02:00 Eliot Miranda <[hidden email]>:

Thanks Nicolai, that's great! I'll merge in your changes. Which files do I need to look at?

I changed (removed my code, added Marcels)
platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c
platforms/win32/vm/sqWin32.h
platforms/win32/vm/sqWin32Directory.c

I added (should be the same as in squeak vm repository)
platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c

But my changes only about that code that already is in squeaks repository.

If we want to merge some of the pharo-vm code into squeaks branch, we

first need to wrap that code with some #ifdefs

(For the win32 file/directory stuff, this would be code in
platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c
platforms/win32/vm/sqWin32.h
platforms/win32/vm/sqWin32Directory.c

and

platforms/Cross/plugins/FilePlugin/FilePlugin.h)

Maybe Estaban should look at this, and tell which path he would follow to

merge those changes.

On Sat, Sep 12, 2015 at 8:00 AM, Nicolai Hess <[hidden email]> wrote:

2015-09-10 21:31 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?

Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source.
I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the
squeak vm main branch (wrapped with pharo vm ifdefs).

github branch: https://github.com/nicolaihess/pharo-vm/tree/harmonize_win_file_plugin
I'll create a push request.

nicolai

On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <[hidden email]> wrote:
For those who had problems with pharo on windows and github based repositories,
I built a windows vm with support for long paths:

https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharing

For browsing directories with large paths (FileList or Inspect),
you may need one additional change in the image (But I am not really sure about that) :

DiskStore>>initialize
super initialize.
maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].

please test and give feedback.

This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths).
Please test other file/folder operations.

nicolai

--
_,,,^..^,,,_
best, Eliot

--
_,,,^..^,,,_
best, Eliot