New: X11UnicodeClipboard
Locked
12
12
 
|
I've published a package X11UnicodeClipboard to the public Store, to
allow VW to paste Unicode characters from other X11 apps, and (with a little work) vice versa: When using the X clipboard/selection, VMs up to at least 7.9 can only successfully transfer ISO-8859-1 characters. This package corrects that to be able to read any Unicode character from the X clipboard. It also partially corrects writing Unicode characters to the X clipboard by escaping them in the standard X format for unsupported characters. These escaped characters look like \u2aff, and cannot be pasted directly, but can be easily transformed into the real characters: e.g. paste in 3rd field and press Convert here: http://unicode.online-toolz.com/tools/text-unicode-entities-convertor.ph p Of course, the real correction is to change the VM: x11Sel.c needs to call XConvertSelection using UTF8_STRING rather than XA_STRING. Ralf Propach referred to this as AR 54602: http://www.parcplace.net/list/vwnc-archive/0901/threads.html#00150 Share and enjoy, Steve _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
Thanks a lot, Steve!
if only VW would utilize UTF8_STRING *sigh*
|
Re: New: X11UnicodeClipboard
|
|
If only, but this is actually the same problem as Xft support -- of our supported platforms, several are old enough that their X11 does not include UTF8_STRING for the clipboard. I hope I'm not mistaken here. I'm pretty sure this exact change had been recommended for X11 in the past but was put on hold due to platform support issues.
Michael On 20/09/2012, at 7:36 AM, Holger Kleinsorgen <[hidden email]> wrote: > Thanks a lot, Steve! > > if only VW would utilize UTF8_STRING *sigh* > > > Steven Kelly wrote >> I've published a package X11UnicodeClipboard to the public Store, to >> allow VW to paste Unicode characters from other X11 apps, and (with a >> little work) vice versa: > > > > > > -- > View this message in context: http://forum.world.st/New-X11UnicodeClipboard-tp4648042p4648105.html > Sent from the VisualWorks mailing list archive at Nabble.com. > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
A couple thoughts... first, the clipboard is handled via primitives. If
changing from XA_STRING to UTF8_STRING involves behavior changes in existing primitives, then we can't do it until the next major release due to backward compatibility concerns. So, assuming, there are behavior changes implied, we would need new primitives, or handle it entirely via the image if possible, etc. With that said, there are important prerequisites to take care of before thinking what to do. For example, if I were doing the work, one of the first things that I'd be concerned with is the specification background. I hear that UTF8_STRING requests were declined before because of platform support concerns, so I assume this means UTF8_STRING wasn't available on some *nix platform. However, I do not know when those requests were made. Was it 2008, or 2002? And what platforms had problems? I have no idea. In the last few years we dropped support for IRIX and HP-UX on PA-RISC. How does that change the picture? These observations hint that perhaps it's time to go through the X11 specs and see when UTF8_STRING was standardized, and with that go through our X11 platforms and see when they started shipping the first officially supported version of X11 that has UTF8_STRING support. Then we can compare the versions of each platform we support. For each platform, do we only support versions that shipped with official UTF8_STRING support? This is critical information to start considering what to do as noted above. Andres. On 9/19/2012 5:15 PM, Michael Lucas-Smith wrote: > If only, but this is actually the same problem as Xft support -- of > our supported platforms, several are old enough that their X11 does > not include UTF8_STRING for the clipboard. I hope I'm not mistaken > here. I'm pretty sure this exact change had been recommended for X11 > in the past but was put on hold due to platform support issues. > > Michael > > On 20/09/2012, at 7:36 AM, Holger Kleinsorgen > <[hidden email]> wrote: > >> Thanks a lot, Steve! >> >> if only VW would utilize UTF8_STRING *sigh* >> >> >> Steven Kelly wrote >>> I've published a package X11UnicodeClipboard to the public Store, >>> to allow VW to paste Unicode characters from other X11 apps, and >>> (with a little work) vice versa: >> >> >> >> >> >> -- View this message in context: >> http://forum.world.st/New-X11UnicodeClipboard-tp4648042p4648105.html >> >> >> _______________________________________________ vwnc mailing list >> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > > _______________________________________________ vwnc mailing list > [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
Andres,
Please allow me a comment. You wrote: "first, the clipboard is handled via primitives." Yes, this is how it was done in early 1980ies, but there is no need in the vm for the clipboard, thus there is no need to keep it there. Many years ago, I implemented a clipboard support for Windows in the image (by the way for exactly the same reason), thus I know, it can be done. In Smalltalk you can do all the nice things you want, and still have a fallback to the current implementation in the vm, if you need. Georg Georg Heeg eK, Dortmund und Köthen, HR Dortmund A 12812 Wallstraße 22, 06366 Köthen Tel. +49-3496-214328, Fax +49-3496-214712 -----Ursprüngliche Nachricht----- Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Andres Valloud Gesendet: Donnerstag, 20. September 2012 03:47 An: Michael Lucas-Smith Cc: [hidden email]; Holger Kleinsorgen Betreff: Re: [vwnc] New: X11UnicodeClipboard A couple thoughts... first, the clipboard is handled via primitives. If changing from XA_STRING to UTF8_STRING involves behavior changes in existing primitives, then we can't do it until the next major release due to backward compatibility concerns. So, assuming, there are behavior changes implied, we would need new primitives, or handle it entirely via the image if possible, etc. With that said, there are important prerequisites to take care of before thinking what to do. For example, if I were doing the work, one of the first things that I'd be concerned with is the specification background. I hear that UTF8_STRING requests were declined before because of platform support concerns, so I assume this means UTF8_STRING wasn't available on some *nix platform. However, I do not know when those requests were made. Was it 2008, or 2002? And what platforms had problems? I have no idea. In the last few years we dropped support for IRIX and HP-UX on PA-RISC. How does that change the picture? These observations hint that perhaps it's time to go through the X11 specs and see when UTF8_STRING was standardized, and with that go through our X11 platforms and see when they started shipping the first officially supported version of X11 that has UTF8_STRING support. Then we can compare the versions of each platform we support. For each platform, do we only support versions that shipped with official UTF8_STRING support? This is critical information to start considering what to do as noted above. Andres. On 9/19/2012 5:15 PM, Michael Lucas-Smith wrote: > If only, but this is actually the same problem as Xft support -- of > our supported platforms, several are old enough that their X11 does > not include UTF8_STRING for the clipboard. I hope I'm not mistaken > here. I'm pretty sure this exact change had been recommended for X11 > in the past but was put on hold due to platform support issues. > > Michael > > On 20/09/2012, at 7:36 AM, Holger Kleinsorgen > <[hidden email]> wrote: > >> Thanks a lot, Steve! >> >> if only VW would utilize UTF8_STRING *sigh* >> >> >> Steven Kelly wrote >>> I've published a package X11UnicodeClipboard to the public Store, to >>> allow VW to paste Unicode characters from other X11 apps, and (with >>> a little work) vice versa: >> >> >> >> >> >> -- View this message in context: >> http://forum.world.st/New-X11UnicodeClipboard-tp4648042p4648105.html >> >> >> _______________________________________________ vwnc mailing list >> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > > _______________________________________________ vwnc mailing list > [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
In reply to this post by Michael Lucas-Smith-2
Hello,
if UTF8_STRING is not supported by a platform, VW can still fall back to using XA_STRING.
|
Re: New: X11UnicodeClipboard
|
|
In reply to this post by Andres Valloud-6
What should definitely be taken care of is the current situation Linux. I'm starting to believe that it's not widely known enough that it is not just a matter of inconvencience or lack of features. The clipboard is utterly broken on this platform, see the examples below. So whenever I hear about preserving stability or support issues I'm wondering what that means in this context. Our case regarding the clipboard is now open for almost three years. Examples: - in VW, copy the umlaut "ö" to the clipboard, then paste the contents in a text editor -> a garbled (utf8-encoded) string appears: "ö" - in a text editor, copy the umlaut "ö" to the clipboard, then paste the contents in VW -> Unhandled exception: Unhandled exception: Strings only store Characters ByteString(Object)>>error: ByteString>>at:put: WriteStream>>nextPut: String class>>fromIntegerArray:encoding: ByteArray(IntegerArray)>>asStringEncoding: ByteArray>>asStringEncoding: IOAccessor class>>convertSimpleStringFromPlatform: IOAccessor class>>convertStringFromPlatform: UnixSystemSupport class(OSSystemSupport class)>>decodeSelection: optimized [] in Screen>>getExternalSelectionIfEmpty: <quote author="Michael Lucas-Smith-2"> If only, but this is actually the same problem as Xft support -- of our supported platforms, several are old enough that their X11 does not include UTF8_STRING for the clipboard. I hope I'm not mistaken here. I'm pretty sure this exact change had been recommended for X11 in the past but was put on hold due to platform support issues. |
|
|
In reply to this post by Michael Lucas-Smith-2
Michael Lucas-Smith wrote 20.9.2012 3:16
> If only, but this is actually the same problem as Xft support -- of our > supported platforms, several are old enough that their X11 does not > include UTF8_STRING for the clipboard. <rant> I lose count of the number of times this has been quoted as a reason for not doing something. Similarly for any change that loses backwards compatibility with old VW code. I absolutely agree with the principles of supporting platforms and keeping backwards compatibility, but I believe the extent of their application in Cincom might need re-evaluating. How many paying customers do you actually have on these old platforms, or using ancient code? And how many of them are also on the latest VW version (or moving there)? (If they're stuck on an older one, they won't be hurt by changes like this.) My guess is the answer is a lot smaller than the number of current customers on current platforms, who are constantly hurt by all the ways in which VW is hamstrung by these decisions. And of course any new evaluators aren't going to be particularly impressed either. Please Cincom, don't think that when Holger releases Xft or Windows7LookPolicy it means there is less pressure on you to bring VW up to date. It should actually prove to you that there is more pressure, as you realize somebody is in such pain from the current state that they will spend weeks of their own time doing the work Cincom should be doing. Of course I understand your resources are as painfully limited as ours: I'm not demanding more, just wondering if they should be distributed differently. So, if VW is surviving on payments from BigCompanyX on some ancient platform, that's fine - but let us know. That way we'll be able to understand your decisions better. At the moment you'll hopefully understand why in our darker moments, we sometimes think you might be hiding behind the 'backwards compatibility' arguments. </rant> All the best, Steve _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
> we sometimes think you might be
> hiding behind the 'backwards compatibility' arguments. Actually strike that - I don't think that's the reason. But I do think there is a culture that says "we can't make it perfect for everyone everywhere, so we'd better defer it". I'd much rather have an 80% solution today, while I wait for a 100% solution that may appear in some later version. And I'm willing to pay the double cost of integrating the 80% solution now, and again integrating a possibly different 100% solution later. The benefits of the 80% solution in the interim are much greater than the additional cost. All the best, Steve _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
In reply to this post by Georg Heeg
Well, that's how it is done now, and that's how it has to be done until
the next major release to maintain backwards compatibility. On 9/19/2012 11:23 PM, Georg Heeg wrote: > Andres, > > Please allow me a comment. You wrote: "first, the clipboard is handled via > primitives." Yes, this is how it was done in early 1980ies, but there is no > need in the vm for the clipboard, thus there is no need to keep it there. > Many years ago, I implemented a clipboard support for Windows in the image > (by the way for exactly the same reason), thus I know, it can be done. In > Smalltalk you can do all the nice things you want, and still have a fallback > to the current implementation in the vm, if you need. > > Georg > > Georg Heeg eK, Dortmund und Köthen, HR Dortmund A 12812 > Wallstraße 22, 06366 Köthen > Tel. +49-3496-214328, Fax +49-3496-214712 > > -----Ursprüngliche Nachricht----- > Von: [hidden email] [mailto:[hidden email]] Im Auftrag > von Andres Valloud > Gesendet: Donnerstag, 20. September 2012 03:47 > An: Michael Lucas-Smith > Cc: [hidden email]; Holger Kleinsorgen > Betreff: Re: [vwnc] New: X11UnicodeClipboard > > A couple thoughts... first, the clipboard is handled via primitives. If > changing from XA_STRING to UTF8_STRING involves behavior changes in existing > primitives, then we can't do it until the next major release due to backward > compatibility concerns. So, assuming, there are behavior changes implied, > we would need new primitives, or handle it entirely via the image if > possible, etc. > > With that said, there are important prerequisites to take care of before > thinking what to do. For example, if I were doing the work, one of the > first things that I'd be concerned with is the specification background. > I hear that UTF8_STRING requests were declined before because of platform > support concerns, so I assume this means UTF8_STRING wasn't available on > some *nix platform. However, I do not know when those requests were made. > Was it 2008, or 2002? And what platforms had problems? I have no idea. In > the last few years we dropped support for IRIX and HP-UX on PA-RISC. How > does that change the picture? These observations hint that perhaps it's > time to go through the X11 specs and see when UTF8_STRING was standardized, > and with that go through our X11 platforms and see when they started > shipping the first officially supported version of X11 that has UTF8_STRING > support. Then we can compare the versions of each platform we support. For > each platform, do we only support versions that shipped with official > UTF8_STRING support? > This is critical information to start considering what to do as noted > above. > > Andres. > > On 9/19/2012 5:15 PM, Michael Lucas-Smith wrote: >> If only, but this is actually the same problem as Xft support -- of >> our supported platforms, several are old enough that their X11 does >> not include UTF8_STRING for the clipboard. I hope I'm not mistaken >> here. I'm pretty sure this exact change had been recommended for X11 >> in the past but was put on hold due to platform support issues. >> >> Michael >> >> On 20/09/2012, at 7:36 AM, Holger Kleinsorgen >> <[hidden email]> wrote: >> >>> Thanks a lot, Steve! >>> >>> if only VW would utilize UTF8_STRING *sigh* >>> >>> >>> Steven Kelly wrote >>>> I've published a package X11UnicodeClipboard to the public Store, to >>>> allow VW to paste Unicode characters from other X11 apps, and (with >>>> a little work) vice versa: >>> >>> >>> >>> >>> >>> -- View this message in context: >>> http://forum.world.st/New-X11UnicodeClipboard-tp4648042p4648105.html >>> >>> > Sent from the VisualWorks mailing list archive at Nabble.com. >>> _______________________________________________ vwnc mailing list >>> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc >> >> >> _______________________________________________ vwnc mailing list >> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc >> > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
In reply to this post by Holger Kleinsorgen
We should separate two things...
One is the existing implementation. Since old images depend on the current behavior for backwards compatibility, what we cannot change is the behavior of the current primitives. If using UTF8_STRING changes the behavior of the primitives as seen from the image, then we can't do that. Now if the idea is to do something else, that's great. And in any case that's when all the research about versions, support, etc comes in. On 9/20/2012 12:22 AM, Holger Kleinsorgen wrote: > Hello, > > if UTF8_STRING is not supported by a platform, VW can still fall back to > using XA_STRING. > > > Michael Lucas-Smith-2 wrote >> If only, but this is actually the same problem as Xft support -- of our >> supported platforms, several are old enough that their X11 does not >> include UTF8_STRING for the clipboard. I hope I'm not mistaken here. I'm >> pretty sure this exact change had been recommended for X11 in the past but >> was put on hold due to platform support issues. > > > > > > > -- > View this message in context: http://forum.world.st/New-X11UnicodeClipboard-tp4648042p4648125.html > Sent from the VisualWorks mailing list archive at Nabble.com. > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Andres Valloud-6
You do not understand. You may keep the primitive as is as long as you want and add the new stuff in Smalltalk now.
There is no issue. Georg Von meinem iPhone gesendet Am 20.09.2012 um 20:22 schrieb Andres Valloud <[hidden email]>: > Well, that's how it is done now, and that's how it has to be done until the next major release to maintain backwards compatibility. > > On 9/19/2012 11:23 PM, Georg Heeg wrote: >> Andres, >> >> Please allow me a comment. You wrote: "first, the clipboard is handled via >> primitives." Yes, this is how it was done in early 1980ies, but there is no >> need in the vm for the clipboard, thus there is no need to keep it there. >> Many years ago, I implemented a clipboard support for Windows in the image >> (by the way for exactly the same reason), thus I know, it can be done. In >> Smalltalk you can do all the nice things you want, and still have a fallback >> to the current implementation in the vm, if you need. >> >> Georg >> >> Georg Heeg eK, Dortmund und Köthen, HR Dortmund A 12812 >> Wallstraße 22, 06366 Köthen >> Tel. +49-3496-214328, Fax +49-3496-214712 >> >> -----Ursprüngliche Nachricht----- >> Von: [hidden email] [mailto:[hidden email]] Im Auftrag >> von Andres Valloud >> Gesendet: Donnerstag, 20. September 2012 03:47 >> An: Michael Lucas-Smith >> Cc: [hidden email]; Holger Kleinsorgen >> Betreff: Re: [vwnc] New: X11UnicodeClipboard >> >> A couple thoughts... first, the clipboard is handled via primitives. If >> changing from XA_STRING to UTF8_STRING involves behavior changes in existing >> primitives, then we can't do it until the next major release due to backward >> compatibility concerns. So, assuming, there are behavior changes implied, >> we would need new primitives, or handle it entirely via the image if >> possible, etc. >> >> With that said, there are important prerequisites to take care of before >> thinking what to do. For example, if I were doing the work, one of the >> first things that I'd be concerned with is the specification background. >> I hear that UTF8_STRING requests were declined before because of platform >> support concerns, so I assume this means UTF8_STRING wasn't available on >> some *nix platform. However, I do not know when those requests were made. >> Was it 2008, or 2002? And what platforms had problems? I have no idea. In >> the last few years we dropped support for IRIX and HP-UX on PA-RISC. How >> does that change the picture? These observations hint that perhaps it's >> time to go through the X11 specs and see when UTF8_STRING was standardized, >> and with that go through our X11 platforms and see when they started >> shipping the first officially supported version of X11 that has UTF8_STRING >> support. Then we can compare the versions of each platform we support. For >> each platform, do we only support versions that shipped with official >> UTF8_STRING support? >> This is critical information to start considering what to do as noted >> above. >> >> Andres. >> >> On 9/19/2012 5:15 PM, Michael Lucas-Smith wrote: >>> If only, but this is actually the same problem as Xft support -- of >>> our supported platforms, several are old enough that their X11 does >>> not include UTF8_STRING for the clipboard. I hope I'm not mistaken >>> here. I'm pretty sure this exact change had been recommended for X11 >>> in the past but was put on hold due to platform support issues. >>> >>> Michael >>> >>> On 20/09/2012, at 7:36 AM, Holger Kleinsorgen >>> <[hidden email]> wrote: >>> >>>> Thanks a lot, Steve! >>>> >>>> if only VW would utilize UTF8_STRING *sigh* >>>> >>>> >>>> Steven Kelly wrote >>>>> I've published a package X11UnicodeClipboard to the public Store, to >>>>> allow VW to paste Unicode characters from other X11 apps, and (with >>>>> a little work) vice versa: >>>> >>>> >>>> >>>> >>>> >>>> -- View this message in context: >>>> http://forum.world.st/New-X11UnicodeClipboard-tp4648042p4648105.html >>>> >>>> >> Sent from the VisualWorks mailing list archive at Nabble.com. >>>> _______________________________________________ vwnc mailing list >>>> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc >>> >>> >>> _______________________________________________ vwnc mailing list >>> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc >>> >> _______________________________________________ >> vwnc mailing list >> [hidden email] >> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc >> >> >> > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
In reply to this post by Steven Kelly
Please note the backwards compatibility argument applies only when we
change the existing behavior. Another example of how this played out was the Windows Unicode VM. So, yes, now it supports Unicode, but of course if you try to run an older image that expects ANSI, it won't work. So what we did was to add a switch in the VM such that it pretends to be ANSI until the image requests Unicode behavior. In that way, we could preserve the old implementation that older images depend on without breaking stuff. We get to break backwards compatibility at major releases (e.g. 8.0, as opposed to 7.x). This is also a unique opportunity to simplify. So, as as soon as the next major release comes around, we're throwing out all the ANSI code, the switch, and we're redoing the Unicode stuff so it's better than what we can do right now. This means a number of things. If there are ways around using the existing primitives, then great and backwards compatibility doesn't apply. If we decide to put in a switch, then we (narrowly) avoid backwards compatibility issues. And if we deal with stuff from the image, then obviously backwards compatibility is not an issue. So, with that in mind, what to do with UTF8_STRING and Xft etc? Well, we still have to compile those things on some Unix platform, so one of the things we need to know is whether the support is there for the platforms we have to run on. Another consideration is whether the functionality will be there in the long run. In other words, it would be a bad idea to add support for Xfoobar or CoreBellsAndWhistles if their expected lifetimes are e.g. 2 years. This is because then, in 5 years, we'd be stuck with another backwards compatibility problem. Then, what should be the plan? I'd think that the first order of business is to do the research into what options are available to us independently of how we actually implement them (e.g. primitives vs the image). Once we have viable options from a technological point of view, then we can move forward and start working on an implementation strategy. Does that make sense? On 9/20/2012 3:15 AM, Steven Kelly wrote: >> we sometimes think you might be >> hiding behind the 'backwards compatibility' arguments. > > Actually strike that - I don't think that's the reason. But I do think > there is a culture that says "we can't make it perfect for everyone > everywhere, so we'd better defer it". I'd much rather have an 80% > solution today, while I wait for a 100% solution that may appear in some > later version. And I'm willing to pay the double cost of integrating the > 80% solution now, and again integrating a possibly different 100% > solution later. The benefits of the 80% solution in the interim are much > greater than the additional cost. > > All the best, > Steve > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
The current image code is broken for anything other than ASCII, since it uses the wrong encoding - as can be seen when pasting an ö into VW. So what do we lose if the primitive is changed to use UTF8_STRING (falling back to XA_STRING where necessary, but always sending to the image in UTF-8)? The ASCII will work still, even in old images, since it's a subset of UTF-8. So nobody is any worse off.
In fact, the old images will start to work better, because on a modern Linux they probably actually expect UTF-8 (Locale current defaultStreamEncoder), rather than the ISO-8859-1 that XA_STRING provides. To find someone who would be hurt by this change, you'd need an old image, the new VM, a Unix that supports UTF8_STRING but is in ISO-8859-1 encoding, and a clipboard containing characters from the top half of ISO-8859-1. I submit that this is a fairly small set compared to the set of people who will be helped by the change. Steve > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Andres Valloud > Sent: 20. syyskuuta 2012 22:41 > To: [hidden email] > Subject: Re: [vwnc] New: X11UnicodeClipboard > > Please note the backwards compatibility argument applies only when we > change the existing behavior. Another example of how this played out > was the Windows Unicode VM. So, yes, now it supports Unicode, but of > course if you try to run an older image that expects ANSI, it won't > work. So what we did was to add a switch in the VM such that it > pretends to be ANSI until the image requests Unicode behavior. In that > way, we could preserve the old implementation that older images depend > on without breaking stuff. > > We get to break backwards compatibility at major releases (e.g. 8.0, as > opposed to 7.x). This is also a unique opportunity to simplify. So, > as > as soon as the next major release comes around, we're throwing out all > the ANSI code, the switch, and we're redoing the Unicode stuff so it's > better than what we can do right now. > > This means a number of things. If there are ways around using the > existing primitives, then great and backwards compatibility doesn't > apply. If we decide to put in a switch, then we (narrowly) avoid > backwards compatibility issues. And if we deal with stuff from the > image, then obviously backwards compatibility is not an issue. > > So, with that in mind, what to do with UTF8_STRING and Xft etc? Well, > we still have to compile those things on some Unix platform, so one of > the things we need to know is whether the support is there for the > platforms we have to run on. Another consideration is whether the > functionality will be there in the long run. In other words, it would > be a bad idea to add support for Xfoobar or CoreBellsAndWhistles if > their expected lifetimes are e.g. 2 years. This is because then, in 5 > years, we'd be stuck with another backwards compatibility problem. > > Then, what should be the plan? I'd think that the first order of > business is to do the research into what options are available to us > independently of how we actually implement them (e.g. primitives vs the > image). Once we have viable options from a technological point of view, > then we can move forward and start working on an implementation > strategy. Does that make sense? > > On 9/20/2012 3:15 AM, Steven Kelly wrote: > >> we sometimes think you might be > >> hiding behind the 'backwards compatibility' arguments. > > > > Actually strike that - I don't think that's the reason. But I do > think > > there is a culture that says "we can't make it perfect for everyone > > everywhere, so we'd better defer it". I'd much rather have an 80% > > solution today, while I wait for a 100% solution that may appear in > some > > later version. And I'm willing to pay the double cost of integrating > the > > 80% solution now, and again integrating a possibly different 100% > > solution later. The benefits of the 80% solution in the interim are > much > > greater than the additional cost. > > > > All the best, > > Steve > > > > _______________________________________________ > > vwnc mailing list > > [hidden email] > > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
If you want we can talk about the new functionality we want. However,
at the end of the day, we cannot change the old one because of our support obligations. Even if that set is small, all it takes is for some critical bit of an app that has been running for a long time without problems to break, and now our story of backwards compatibility sounds really wrong. None of this means we shouldn't look into a) exactly what features are available where and since when, then b) looking at a way to implement them without breaking backwards compatibility. On 9/20/2012 4:11 PM, Steven Kelly wrote: > The current image code is broken for anything other than ASCII, since > it uses the wrong encoding - as can be seen when pasting an ö into > VW. So what do we lose if the primitive is changed to use UTF8_STRING > (falling back to XA_STRING where necessary, but always sending to the > image in UTF-8)? The ASCII will work still, even in old images, since > it's a subset of UTF-8. So nobody is any worse off. > > In fact, the old images will start to work better, because on a > modern Linux they probably actually expect UTF-8 (Locale current > defaultStreamEncoder), rather than the ISO-8859-1 that XA_STRING > provides. > > To find someone who would be hurt by this change, you'd need an old > image, the new VM, a Unix that supports UTF8_STRING but is in > ISO-8859-1 encoding, and a clipboard containing characters from the > top half of ISO-8859-1. I submit that this is a fairly small set > compared to the set of people who will be helped by the change. > > Steve > >> -----Original Message----- From: [hidden email] >> [mailto:[hidden email]] On Behalf Of Andres Valloud Sent: >> 20. syyskuuta 2012 22:41 To: [hidden email] Subject: Re: [vwnc] >> New: X11UnicodeClipboard >> >> Please note the backwards compatibility argument applies only when >> we change the existing behavior. Another example of how this >> played out was the Windows Unicode VM. So, yes, now it supports >> Unicode, but of course if you try to run an older image that >> expects ANSI, it won't work. So what we did was to add a switch in >> the VM such that it pretends to be ANSI until the image requests >> Unicode behavior. In that way, we could preserve the old >> implementation that older images depend on without breaking stuff. >> >> We get to break backwards compatibility at major releases (e.g. >> 8.0, as opposed to 7.x). This is also a unique opportunity to >> simplify. So, as as soon as the next major release comes around, >> we're throwing out all the ANSI code, the switch, and we're redoing >> the Unicode stuff so it's better than what we can do right now. >> >> This means a number of things. If there are ways around using the >> existing primitives, then great and backwards compatibility >> doesn't apply. If we decide to put in a switch, then we (narrowly) >> avoid backwards compatibility issues. And if we deal with stuff >> from the image, then obviously backwards compatibility is not an >> issue. >> >> So, with that in mind, what to do with UTF8_STRING and Xft etc? >> Well, we still have to compile those things on some Unix platform, >> so one of the things we need to know is whether the support is >> there for the platforms we have to run on. Another consideration >> is whether the functionality will be there in the long run. In >> other words, it would be a bad idea to add support for Xfoobar or >> CoreBellsAndWhistles if their expected lifetimes are e.g. 2 years. >> This is because then, in 5 years, we'd be stuck with another >> backwards compatibility problem. >> >> Then, what should be the plan? I'd think that the first order of >> business is to do the research into what options are available to >> us independently of how we actually implement them (e.g. primitives >> vs the image). Once we have viable options from a technological >> point of view, then we can move forward and start working on an >> implementation strategy. Does that make sense? >> >> On 9/20/2012 3:15 AM, Steven Kelly wrote: >>>> we sometimes think you might be hiding behind the 'backwards >>>> compatibility' arguments. >>> >>> Actually strike that - I don't think that's the reason. But I do >> think >>> there is a culture that says "we can't make it perfect for >>> everyone everywhere, so we'd better defer it". I'd much rather >>> have an 80% solution today, while I wait for a 100% solution that >>> may appear in >> some >>> later version. And I'm willing to pay the double cost of >>> integrating >> the >>> 80% solution now, and again integrating a possibly different >>> 100% solution later. The benefits of the 80% solution in the >>> interim are >> much >>> greater than the additional cost. >>> >>> All the best, Steve >>> >>> _______________________________________________ vwnc mailing >>> list [hidden email] >>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc >>> >> _______________________________________________ vwnc mailing list >> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ vwnc mailing list > [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
Let's say that out of 100 existing VW Unix/Linux users (the numbers don't matter, I'm after the principle, not just this case):
* 60 are fine because they are only in ASCII anyway * 10 are fine because they need the upper half of ISO-8859-1, and their platform locale is ISO-8859-1 * 30 are hurting because the image assumes the platform locale encoding, whereas the VM delivers ISO-8859-1, and either those don't match or they need something other than ISO-8859-1 Are you saying that your policy is not to implement a fix that would improve that 70-30 split to 90-10, because there will be some who are worse off, even if three times as many are helped and most are unchanged? And those 10 who are hurt could simply choose not to upgrade the VM until they've also upgraded their image? I agree the 10 may be angry, but at least they have a choice to avoid the pain. If you leave it unfixed, the 30 are angry and have no choice but to continue taking the pain. Steve > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Andres Valloud > Sent: 21. syyskuuta 2012 2:19 > To: [hidden email] > Subject: Re: [vwnc] New: X11UnicodeClipboard > > If you want we can talk about the new functionality we want. However, > at the end of the day, we cannot change the old one because of our > support obligations. Even if that set is small, all it takes is for > some critical bit of an app that has been running for a long time > without problems to break, and now our story of backwards compatibility > sounds really wrong. > > None of this means we shouldn't look into a) exactly what features are > available where and since when, then b) looking at a way to implement > them without breaking backwards compatibility. > > On 9/20/2012 4:11 PM, Steven Kelly wrote: > > The current image code is broken for anything other than ASCII, since > > it uses the wrong encoding - as can be seen when pasting an ö into > > VW. So what do we lose if the primitive is changed to use UTF8_STRING > > (falling back to XA_STRING where necessary, but always sending to the > > image in UTF-8)? The ASCII will work still, even in old images, since > > it's a subset of UTF-8. So nobody is any worse off. > > > > In fact, the old images will start to work better, because on a > > modern Linux they probably actually expect UTF-8 (Locale current > > defaultStreamEncoder), rather than the ISO-8859-1 that XA_STRING > > provides. > > > > To find someone who would be hurt by this change, you'd need an old > > image, the new VM, a Unix that supports UTF8_STRING but is in > > ISO-8859-1 encoding, and a clipboard containing characters from the > > top half of ISO-8859-1. I submit that this is a fairly small set > > compared to the set of people who will be helped by the change. > > > > Steve > > > >> -----Original Message----- From: [hidden email] > >> [mailto:[hidden email]] On Behalf Of Andres Valloud Sent: > >> 20. syyskuuta 2012 22:41 To: [hidden email] Subject: Re: [vwnc] > >> New: X11UnicodeClipboard > >> > >> Please note the backwards compatibility argument applies only when > >> we change the existing behavior. Another example of how this > >> played out was the Windows Unicode VM. So, yes, now it supports > >> Unicode, but of course if you try to run an older image that > >> expects ANSI, it won't work. So what we did was to add a switch in > >> the VM such that it pretends to be ANSI until the image requests > >> Unicode behavior. In that way, we could preserve the old > >> implementation that older images depend on without breaking stuff. > >> > >> We get to break backwards compatibility at major releases (e.g. > >> 8.0, as opposed to 7.x). This is also a unique opportunity to > >> simplify. So, as as soon as the next major release comes around, > >> we're throwing out all the ANSI code, the switch, and we're redoing > >> the Unicode stuff so it's better than what we can do right now. > >> > >> This means a number of things. If there are ways around using the > >> existing primitives, then great and backwards compatibility > >> doesn't apply. If we decide to put in a switch, then we (narrowly) > >> avoid backwards compatibility issues. And if we deal with stuff > >> from the image, then obviously backwards compatibility is not an > >> issue. > >> > >> So, with that in mind, what to do with UTF8_STRING and Xft etc? > >> Well, we still have to compile those things on some Unix platform, > >> so one of the things we need to know is whether the support is > >> there for the platforms we have to run on. Another consideration > >> is whether the functionality will be there in the long run. In > >> other words, it would be a bad idea to add support for Xfoobar or > >> CoreBellsAndWhistles if their expected lifetimes are e.g. 2 years. > >> This is because then, in 5 years, we'd be stuck with another > >> backwards compatibility problem. > >> > >> Then, what should be the plan? I'd think that the first order of > >> business is to do the research into what options are available to > >> us independently of how we actually implement them (e.g. primitives > >> vs the image). Once we have viable options from a technological > >> point of view, then we can move forward and start working on an > >> implementation strategy. Does that make sense? > >> > >> On 9/20/2012 3:15 AM, Steven Kelly wrote: > >>>> we sometimes think you might be hiding behind the 'backwards > >>>> compatibility' arguments. > >>> > >>> Actually strike that - I don't think that's the reason. But I do > >> think > >>> there is a culture that says "we can't make it perfect for > >>> everyone everywhere, so we'd better defer it". I'd much rather > >>> have an 80% solution today, while I wait for a 100% solution that > >>> may appear in > >> some > >>> later version. And I'm willing to pay the double cost of > >>> integrating > >> the > >>> 80% solution now, and again integrating a possibly different > >>> 100% solution later. The benefits of the 80% solution in the > >>> interim are > >> much > >>> greater than the additional cost. > >>> > >>> All the best, Steve > >>> > >>> _______________________________________________ vwnc mailing > >>> list [hidden email] > >>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > >>> > >> _______________________________________________ vwnc mailing list > >> [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > > > _______________________________________________ vwnc mailing list > > [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
On 9/20/2012 4:41 PM, Steven Kelly wrote:
> Let's say that out of 100 existing VW Unix/Linux users (the numbers > don't matter, I'm after the principle, not just this case): * 60 are > fine because they are only in ASCII anyway * 10 are fine because they > need the upper half of ISO-8859-1, and their platform locale is > ISO-8859-1 * 30 are hurting because the image assumes the platform > locale encoding, whereas the VM delivers ISO-8859-1, and either those > don't match or they need something other than ISO-8859-1 > > Are you saying that your policy is not to implement a fix that would > improve that 70-30 split to 90-10, because there will be some who are > worse off, even if three times as many are helped and most are > unchanged? And those 10 who are hurt could simply choose not to > upgrade the VM until they've also upgraded their image? > > I agree the 10 may be angry, but at least they have a choice to avoid > the pain. If you leave it unfixed, the 30 are angry and have no > choice but to continue taking the pain. So why can't we have it such that everyone is happy? That's what I'm trying to go after. I'll use your numbers for the sake of the below but I note we don't really know how many are in each group. What I am saying is: * Leave the existing functionality unchanged. In other words, if it works for you then great, you have to do nothing at all. Those would be the 10 + 60 you suggest. * Provide new functionality for the 30 that are hurting. In fact, let's do it well: figure out what this new functionality is, where it is available in supported status etc, and make it available in as many platforms as possible. Indeed, if we can make it available on *every* X11 platform then that's even better because likely we won't even have to write conditional code that we will have to maintain later. In short, write a small bit code that you know Works(TM), and write it once and only once. As to the specific details of *how* to make the new functionality available, there are a variety of technical options and some of those depend on the particulars of how UTF8_STRING came to be (which right now I am totally unaware of). And, if we look just a bit further beyond UTF8_STRING, could any UTF8_STRING changes be made such they can be part of a larger picture, and without painting ourselves into a corner? So what I am saying is let's figure out the details of UTF8_STRING and see what we can do with that. Does that make sense now? Andres. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
> So why can't we have it such that everyone is happy? That's what I'm
> trying to go after. ... > Indeed, if we can make it available on *every* X11 platform then > that's even better .... > And, if we look just a bit further beyond UTF8_STRING, could any > UTF8_STRING changes be made such they can be part of a larger picture Actually, I think you may have just illustrated what I was saying, far better than I could have said it myself. The desire to get to "everyone being happy" is such that the "90% happy" solution doesn't get implemented in the interim. And since "everyone being happy" is often not possible, we end up waiting a very long time for some things that could be 90% fixed with little effort. Again, I'm talking about the general culture, not this specific case. If any change made to VW was only made once it was a) certain to be good for everyone, and b) has looked at the big picture and made a wonderful coherent world now with no doors shut for the future, then VW would indeed be amazing. As it is. But all the apps that people ship using VW would be held together by chewing gum and string - a myriad of 'fix' packages. As they are. Because our customers demand at least some kind of solution to all the things that are still awaiting the perfect solution from Cincom. For the record, UTF8_STRING was added in 2000 and released in XFree86 4.2 in 2002: http://www.pps.univ-paris-diderot.fr/~jch/software/UTF8_STRING/ Xft was released in 2000 / XFree86 4.0.2 Windows XP (the last Windows look & feel provided by Cincom) was released in 2001. I have no major gripe on the individual cases, it's the culture or policy that I'm hoping will change a little. All the best, Steve _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
Well, let's see here...
> For the record, UTF8_STRING was added in 2000 and released in XFree86 > 4.2 in 2002: > http://www.pps.univ-paris-diderot.fr/~jch/software/UTF8_STRING/ Ok, so the work now would require making sure the relevant platforms officially support it. At first glance, most seem ok except Solaris versions earlier than 10 do not support XA_UTF8_STRING. Solaris 9 was last updated in 2005, and will become EOL in October of 2014. So, any approach taken with XA_UTF8_STRING has to account for at least one major platform not supporting it, so there has to be a fallback mechanism. > Xft was released in 2000 / XFree86 4.0.2 I cannot really speak much to the details of Xft, other than to point out that it became supported by X11 on X11R6.8. Andres. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: New: X11UnicodeClipboard
|
|
In reply to this post by Andres Valloud-6
And what about support for current platforms with current VMs and images?
Is compatibility with older platforms actually *more* important? We're not talking of "Xfoobar or CoreBellsAndWhistles", as you mentioned above. We're talking about a broken core functionality of the GUI, e.g. an unhandled exception when pasting certain ISO-8859-1 characters. And it's broken for years now. UTF-8 locales, UTF8_STRING and friends are not new kids in the town, they've been around for years. Given that there is no single vendor for X11-based platforms, it will be difficult to implement a "perfect" solution, because things *might* change. But I doubt that UTF-8 locales and UTF8_STRING will be replaced in the near future. A suggestion concerning backward compatibility: ObjectMemory registerObject: 'UTF8_STRING' withEngineFor: 'X11SelectionAtom' Screen externalSelectionEncoding: #'utf-8'.
|
«
Return to VisualWorks
|
1 view|%1 views
| Free forum by Nabble | Edit this page |