InterpreterProxy>>ioFilename:fromString:ofLength:resolveAliases:?

>> From: tim Rowledge <[hidden email]>
>> Subject: Re: New VMMaker/svn release
>> Date: Fri, 30 Dec 2005 17:14:06 -0800
>> To: [hidden email]
>>
>> Hokay,
>> After taking Andreas' suggestion to dump the tacky old
>> sqFilenameFromString macro to heart, SM has a new release 11 -
>> 3.8b5-64B and SVN has level 1284 for you to play with. Read the SM
>> page comments.  It may well break things but I think it gets the
>> basic facility cleaner. I also cleaned out some ugly arg-typing
>> floobs while I was at it.
>>
>> Note that it looks to me as if AsynchFilePlugin and DeflatePlugin
>> could do with some looking at wrt 64bit cleanliness.

> tim Rowledge wrote:
>> On 25-Mar-06, at 10:33 PM, Andreas Raab wrote:
>>> This function is news to me - can somebody explain what it's good  
>>> for, where it should be used and (most importantly) why we'd  
>>> export this for plugins but don't use in the VM itself (e.g., no  
>>> senders)?
>> See mantis 2044 for some details and the original rationale. It  
>> also relates somewhat to :-
>
> I guess you mean http://bugs.impara.de/view.php?id=2404 yes? One  
> question about this: Do we have a way of doing that kind of file  
> name conversion from the image itself? It strikes me as somewhat  
> odd that a plugin needs to do that conversion given that the image  
> can just pass a well-formed version of the string as the argument, no?

> Am 26.03.2006 um 09:11 schrieb Andreas Raab:
> > tim Rowledge wrote:
> >> On 25-Mar-06, at 10:33 PM, Andreas Raab wrote:
> >>> This function is news to me - can somebody explain what it's good  
> >>> for, where it should be used and (most importantly) why we'd  
> >>> export this for plugins but don't use in the VM itself (e.g., no  
> >>> senders)?
> >> See mantis 2044 for some details and the original rationale. It  
> >> also relates somewhat to :-
> >
> > I guess you mean http://bugs.impara.de/view.php?id=2404 yes? One  
> > question about this: Do we have a way of doing that kind of file  
> > name conversion from the image itself? It strikes me as somewhat  
> > odd that a plugin needs to do that conversion given that the image  
> > can just pass a well-formed version of the string as the argument, no?
>
> Not if the VM creates the illusion of some other naming than the  
> actual system uses. Like the use of HFS file names in the Mac VM  
> (with : separators), where the VM translates to unix path names  
> nowadays. The problem with the macro was that it called (on the Mac  
> at least) a non-exported VM function. But in the freetype plugin I  
> needed to be able to convert from the path name the image uses to the  
> actual system path name. So just using the macro didn't work, which  
> is why we added that function.

> tim Rowledge wrote:
>> On 25-Mar-06, at 10:33 PM, Andreas Raab wrote:
>>> This function is news to me - can somebody explain what it's good  
>>> for, where it should be used and (most importantly) why we'd  
>>> export this for plugins but don't use in the VM itself (e.g., no  
>>> senders)?
>> See mantis 2044 for some details and the original rationale. It  
>> also relates somewhat to :-
>
> I guess you mean http://bugs.impara.de/view.php?id=2404 yes?

>> One question about this: Do we have a way of doing that kind of file
>> name conversion from the image itself? It strikes me as somewhat odd
>> that a plugin needs to do that conversion given that the image can
>> just pass a well-formed version of the string as the argument, no?
> Actually, no. Until and unless we ever get some decent file handling
> code in the image we simply can't rely on a passed in filename being
> even vaguely decent. My preference is to call a VM routine to
> canonicalise the passed in string for every usage simply because you can
> rely (for some value of 'rely') on the OS getting it right in working
> out that '../../foo/bar/bim.bmp' is actually
> '/usr/mounts/someOtherComputer/viaALink/mumble/frotz.jpeg'.

> tim Rowledge wrote:
>>> One question about this: Do we have a way of doing that kind of  
>>> file name conversion from the image itself? It strikes me as  
>>> somewhat odd that a plugin needs to do that conversion given that  
>>> the image can just pass a well-formed version of the string as  
>>> the argument, no?
>> Actually, no. Until and unless we ever get some decent file  
>> handling code in the image we simply can't rely on a passed in  
>> filename being even vaguely decent. My preference is to call a VM  
>> routine to canonicalise the passed in string for every usage  
>> simply because you can rely (for some value of 'rely') on the OS  
>> getting it right in working out that '../../foo/bar/bim.bmp' is  
>> actually '/usr/mounts/someOtherComputer/viaALink/mumble/frotz.jpeg'.
>
> All right, you guys know this better than I do (since I don't have  
> this problem ;-) So, just for verification, the following is a  
> legal implementation for the method, yes?
>
> int sqGetFilenameFromString(char *buf, char *fname, int length, int  
> b) {
>   memcpy(buf, fname, length);
>   return 1;
> }
>
>> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
>> When all else fails, let a = 7.  If that doesn't help, then read  
>> the manual.
>
> How fitting ;-) It was just how I felt when writing the above. Are  
> you sure this is random?
>
> Cheers,
>   - Andreas

> John M McIntosh wrote:
>> I believe fname here and length is technically the smalltalk  
>> object, and it's length. If you copy over from the fname it may  
>> not have
>> a terminating 0x00.
>
> So. Where. Is. This. Used??? The comment in sq.h merely states:
>
>   "New filename converting function; used by the interpreterProxy  
> function ioFilenamefromStringofLengthresolveAliases. Most platforms  
> can ignore the resolveAlias boolean - it seems to only be of use by  
> OSX but is crucial there."
>
> and unfortunately, this doesn't give much guidance about arguments  
> or return values. Without an example (which I have asked for,  
> remember?) it is *impossible* to implement that function correctly  
> based on the information I have.
>
> Cheers,
>   - Andreas

> Usage of it in the standard unix based VMs can be seen in the
>
> Cross/sqFilePluginBasicPrims.c
>
>     /* copy the file name into a null-terminated C string */
>     if (sqFileNameSize > 1000) {
>         return interpreterProxy->success(false);
>     }
>     interpreterProxy->ioFilenamefromStringofLengthresolveAliases(cFileName,
> sqFileName, sqFileNameSize, true);

>
>> Usage of it in the standard unix based VMs can be seen in the
>> Cross/sqFilePluginBasicPrims.c
>>     /* copy the file name into a null-terminated C string */
>>     if (sqFileNameSize > 1000) {
>>         return interpreterProxy->success(false);
>>     }
>>     interpreterProxy->ioFilenamefromStringofLengthresolveAliases
>> (cFileName, sqFileName, sqFileNameSize, true);
>
> I see. If the other usages are the same then the one thing that's  
> missing from my implementation is a terminating zero character, yes?

> The mpeg plugin uses
> sqFilenameFromStringOpen()
>
> which I declare as
>
> #include "sqVirtualMachine.h"
>
> extern struct VirtualMachine * interpreterProxy;
>
> void        sqFilenameFromStringOpen(char *buffer,long fileIndex, long
> fileLength, int flag) {
>     interpreterProxy->ioFilenamefromStringofLengthresolveAliases(buffer,fileIndex,
> fileLength, flag);
> }

>>> The api  then converts the given file name in VM platform character
>>> type (macroman/utf-8/latin-1) into a string that the host operating
>>> system can use to act upon using host operating system file calls.
>>
>> Which, again, makes me want to ask why we actually burden the plugin
>> with figuring that out instead of giving it a filename that works?
>
> Well we need a primitive that would translate the image file string to
> the host api file name.
> ioFilenamefromStringofLengthresolveAliases comes to mind.

>   Hello,
>
>> Well, my point is that if we expose that primitive to the image,  
>> we will
>> never have the need to do it from a plugin because you can always  
>> pass a
>> valid file name into it. It's really strange to me that the VM does
>> translation for stuff which is (supposedly) handled in the image
>> already. I could understand this if we would shoot for a file/
>> directory
>> abstraction inside the image but the way it's right now we're hiding
>> stuff from the image that the image supposedly is responsible for
>> (platform file name handling) and that's Just Wrong (tm).
>
>   Since Squeak3.8, FilePath class is supposed to do the conversion.  I
> wonder if how ioFilename:fromString:ofLength:resolveAliases: interacts
> with FilePath...
>
> -- Yoshiki

>
> Ah, poorly.
>
>  From what I've seen the in image encoding assumes latin-1. It gets  
> macroman from the VM and mangles, demangles that ok, however
> displays the characters then wrong in the file browser.  If you  
> switch the VM to use latin-1 then characters are displayed correctly,  
> however
> it is possible to create file names that contains characters that  
> UTF-8 allows but has not latin-1 mapping, thus the internal VM  
> mapping from that character to latin-1 fails.
>
> Keeping the VM in UTF-8 seems to work, of course FilePath mangles  
> that as latin-1 and the names are not displayed correctly.
>
>
> On 29-Mar-06, at 6:42 PM, Yoshiki Ohshima wrote:
>
> >   Hello,
> >
> >> Well, my point is that if we expose that primitive to the image,  
> >> we will
> >> never have the need to do it from a plugin because you can always  
> >> pass a
> >> valid file name into it. It's really strange to me that the VM does
> >> translation for stuff which is (supposedly) handled in the image
> >> already. I could understand this if we would shoot for a file/
> >> directory
> >> abstraction inside the image but the way it's right now we're hiding
> >> stuff from the image that the image supposedly is responsible for
> >> (platform file name handling) and that's Just Wrong (tm).
> >
> >   Since Squeak3.8, FilePath class is supposed to do the conversion.  I
> > wonder if how ioFilename:fromString:ofLength:resolveAliases: interacts
> > with FilePath...
> >
> > -- Yoshiki
>
> --
> ========================================================================
> ===
> John M. McIntosh <[hidden email]> 1-800-477-2659
> Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
> ========================================================================
> ===
>

>   John,
>
>   It may be done poorly.  But the idea is to switch the encoding in
> the image by providing the right fileNameConverter.
>
>   I'm not sure what you mean by "in image encoding assumes latin-1".
> If the file browser doesn't show the right file name, the right fix is
> to change the fileNameConverter, not changing the VM.
>
> -- Yoshiki
>
> At Wed, 29 Mar 2006 21:35:47 -0800,
> John M McIntosh wrote:
>>
>> Ah, poorly.
>>
>>  From what I've seen the in image encoding assumes latin-1. It gets
>> macroman from the VM and mangles, demangles that ok, however
>> displays the characters then wrong in the file browser.  If you
>> switch the VM to use latin-1 then characters are displayed correctly,
>> however
>> it is possible to create file names that contains characters that
>> UTF-8 allows but has not latin-1 mapping, thus the internal VM
>> mapping from that character to latin-1 fails.
>>
>> Keeping the VM in UTF-8 seems to work, of course FilePath mangles
>> that as latin-1 and the names are not displayed correctly.
>>
>>
>> On 29-Mar-06, at 6:42 PM, Yoshiki Ohshima wrote:
>>
>>>   Hello,
>>>
>>>> Well, my point is that if we expose that primitive to the image,
>>>> we will
>>>> never have the need to do it from a plugin because you can always
>>>> pass a
>>>> valid file name into it. It's really strange to me that the VM does
>>>> translation for stuff which is (supposedly) handled in the image
>>>> already. I could understand this if we would shoot for a file/
>>>> directory
>>>> abstraction inside the image but the way it's right now we're  
>>>> hiding
>>>> stuff from the image that the image supposedly is responsible for
>>>> (platform file name handling) and that's Just Wrong (tm).
>>>
>>>   Since Squeak3.8, FilePath class is supposed to do the  
>>> conversion.  I
>>> wonder if how ioFilename:fromString:ofLength:resolveAliases:  
>>> interacts
>>> with FilePath...
>>>
>>> -- Yoshiki
>>
>> --
>> =====================================================================
>> ===
>> ===
>> John M. McIntosh <[hidden email]> 1-800-477-2659
>> Corporate Smalltalk Consulting Ltd.  http://
>> www.smalltalkconsulting.com
>> =====================================================================
>> ===
>> ===
>>

>   John,
>
>   It may be done poorly.  But the idea is to switch the encoding in
> the image by providing the right fileNameConverter.
>
>   I'm not sure what you mean by "in image encoding assumes latin-1".
> If the file browser doesn't show the right file name, the right fix is
> to change the fileNameConverter, not changing the VM.
>
> -- Yoshiki
>
> At Wed, 29 Mar 2006 21:35:47 -0800,
> John M McIntosh wrote:
>>
>> Ah, poorly.
>>
>>  From what I've seen the in image encoding assumes latin-1. It gets
>> macroman from the VM and mangles, demangles that ok, however
>> displays the characters then wrong in the file browser.  If you
>> switch the VM to use latin-1 then characters are displayed correctly,
>> however
>> it is possible to create file names that contains characters that
>> UTF-8 allows but has not latin-1 mapping, thus the internal VM
>> mapping from that character to latin-1 fails.
>>
>> Keeping the VM in UTF-8 seems to work, of course FilePath mangles
>> that as latin-1 and the names are not displayed correctly.
>>
>>
>> On 29-Mar-06, at 6:42 PM, Yoshiki Ohshima wrote:
>>
>>>   Hello,
>>>
>>>> Well, my point is that if we expose that primitive to the image,
>>>> we will
>>>> never have the need to do it from a plugin because you can always
>>>> pass a
>>>> valid file name into it. It's really strange to me that the VM does
>>>> translation for stuff which is (supposedly) handled in the image
>>>> already. I could understand this if we would shoot for a file/
>>>> directory
>>>> abstraction inside the image but the way it's right now we're  
>>>> hiding
>>>> stuff from the image that the image supposedly is responsible for
>>>> (platform file name handling) and that's Just Wrong (tm).
>>>
>>>   Since Squeak3.8, FilePath class is supposed to do the  
>>> conversion.  I
>>> wonder if how ioFilename:fromString:ofLength:resolveAliases:  
>>> interacts
>>> with FilePath...
>>>
>>> -- Yoshiki
>>
>> --
>> =====================================================================
>> ===
>> ===
>> John M. McIntosh <[hidden email]> 1-800-477-2659
>> Corporate Smalltalk Consulting Ltd.  http://
>> www.smalltalkconsulting.com
>> =====================================================================
>> ===
>> ===
>>

> Well the VM is given a info.plist entry or cmd line that defines what  
> encoding to use. This information needs to be gotten from
> the VM in order to set the proper filenameConverter. My understanding  
> was that the default for the mac carbon vm was
> MacRoman, but the fileNameConverter used in 3.8? is latin-1. If I  
> choose latin-1 there are characters in the utf-8 character set  
> allowed in
> os-x that can't be translated to latin-1 so latin-1 technically is  
> not a viable choice. I understand UTF-8 is viable, but somehow the  
> image should configure that from the Vm without manual intervention.