The Trunk: TrueType-nice.13.mcz

> Nicolas Cellier uploaded a new version of TrueType to project The Trunk:
> http://source.squeak.org/trunk/TrueType-nice.13.mcz
>
> ==================== Summary ====================
>
> Name: TrueType-nice.13
> Author: nice
> Time: 18 January 2010, 6:18:47.592 pm
> UUID: 9c0ae3bf-8eee-434b-bb69-a4cf0bda202a
> Ancestors: TrueType-nice.12
>
> Use ByteArray literals
>

> 2010/1/18  <[hidden email]>:
>> Nicolas Cellier uploaded a new version of TrueType to project The Trunk:
>> http://source.squeak.org/trunk/TrueType-nice.13.mcz
>>
>> ==================== Summary ====================
>>
>> Name: TrueType-nice.13
>> Author: nice
>> Time: 18 January 2010, 6:18:47.592 pm
>> UUID: 9c0ae3bf-8eee-434b-bb69-a4cf0bda202a
>> Ancestors: TrueType-nice.12
>>
>> Use ByteArray literals
>>
>
> Woohooo, ByteArray literals!!!
>

> 2010/1/18 Igor Stasenko <[hidden email]>:
>> 2010/1/18  <[hidden email]>:
>>> Nicolas Cellier uploaded a new version of TrueType to project The Trunk:
>>> http://source.squeak.org/trunk/TrueType-nice.13.mcz
>>>
>>> ==================== Summary ====================
>>>
>>> Name: TrueType-nice.13
>>> Author: nice
>>> Time: 18 January 2010, 6:18:47.592 pm
>>> UUID: 9c0ae3bf-8eee-434b-bb69-a4cf0bda202a
>>> Ancestors: TrueType-nice.12
>>>
>>> Use ByteArray literals
>>>
>>
>> Woohooo, ByteArray literals!!!
>>
>
> BTW, I see much code like:
>   'true' asByteArray
> Not that it costs that much CPU, but I'd like to make it a literal...
> However, I'd like to preserve the initializing code and semantics.
> This solution is not really satisfying:
>  "'true' asByteArray" #[116 114 117 101]
>
> Dolphinners would write some initializer like:
> ##( 'true' asByteArray )
>
> Travis Griggs would use a block and a tricky superpower in VW (can't
> find the blog right now).
>
> How do you see it?
>

> Nicolas
>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>>
>
>

>> Dolphinners would write some initializer like:
>> ##( 'true' asByteArray )
>>
>> ...
>> How do you see it?
>>
> Perhaps:
> #['true']
>
> the check that if first literal is string (not a number), then it
> should be the only literal within []

> Hi,
>
> Am 18.01.2010 um 19:47 schrieb Igor Stasenko <[hidden email]>:
>>>
>>> Dolphinners would write some initializer like:
>>> ##( 'true' asByteArray )
>>>
>>> ...
>>> How do you see it?
>>>
>> Perhaps:
>> #['true']
>>
>> the check that if first literal is string (not a number), then it
>> should be the only literal within []
>
> I don't know Dolphin, but the construct looks like a macro (correct?). Now,
> macros are way cool. At least in my opinion. Who says we have to stick to
> ST-80 forever?
>

>
> On 2010-01-18, at 10:34 AM, Nicolas Cellier wrote:
>
>>
>> BTW, I see much code like:
>>   'true' asByteArray
>> Not that it costs that much CPU, but I'd like to make it a literal...
>> However, I'd like to preserve the initializing code and semantics.
>> This solution is not really satisfying:
>>  "'true' asByteArray" #[116 114 117 101]
>
> I prefer this option, actually. 'true' asByteArray is bogus to begin with. What encoding are those bytes? ASCII? Latin-1? UTF8? Do the bytes that are produced depend on the internal encoding of String? Or maybe the encoding of the changes file. Sure, all these questions have answers, but they are not obvious from the source code. At least #[116 114 117 101] is unambiguous.
>
> Colin
>

> On 2010-01-18, at 10:34 AM, Nicolas Cellier wrote:
>
>> BTW, I see much code like:
>>   'true' asByteArray
>> Not that it costs that much CPU, but I'd like to make it a literal...
>> However, I'd like to preserve the initializing code and semantics.
>> This solution is not really satisfying:
>>  "'true' asByteArray" #[116 114 117 101]
>
> I prefer this option, actually. 'true' asByteArray is bogus to begin with.

> It's not. Check the comment:
>
> ['true' asByteArray] -> ["Version 1.0 TTF file"
> "http://developer.apple.com/textfonts/TTRefMan/RM06/Chap6.html
> The values 'true' (0x74727565) and 0x00010000 are recognized by the Mac OS
> as referring to TrueType fonts."
> ^Array with: 0 "only one font"
> ].
>
> Since this code has no performance implication whatsoever, I find 'true' asByteArray vastly more readable and intention revealing than  #[116 114 117 101].

> I agree, and hence why not:
>
>        [#[ 16r74 "t" 16r72 "r" 16r75 "u" 16r65 "e" ] -> ["Version 1.0
> TTF file"
>              
>  "http://developer.apple.com/textfonts/TTRefMan/RM06/Chap6.html
>                The values 'true' (16r74727565 in ascii) and 16r00010000
> are recognized by the Mac OS
>                as referring to TrueType fonts."
>                ^Array with: 0 "only one font"
>        ].
>
> ?  Use of 16r is to be preferred to 0x because this is a comment to be
> read by Smalltalk programmers.

>
> On 2010-01-19, at 10:04 AM, Andreas Raab wrote:
>
>> Regarding the "oh, this would be a problem if the string encoding changes", let's keep in mind that the string encoding *hasn't* changed for ASCII. So, no, it's specifically incorrect to say that the past change would have affected or invalidated that code. I don't think you understand how fundamental the impact of an encoding change is - if you did that *all* string literals would look like gobbly gook. The only reason we could do it for non-ascii was that we're not using any non-ascii characters and we were only switching the characters > 127 from Mac Roman to Unicode/Latin-1. If you don't believe me, grab your favorite non-ascii piece of text and throw it at "yourText squeakToMac" and have a look at it.
>>
>> So the argument that the dependency on literal string encoding is an issue is bogus. If you change literal string encoding there are so many other places that break it's not even funny. And at least I don't design for the implausible (what reason do we have to expect the encoding to change in the next fifty years?)
>
>
> Here's another way to think about it: #asByteArray violates the abstraction provided by String. A string is a sequence of characters, right? How that sequence is represented in memory is internal to the implementation, and should not be relied upon by users of the string. Historically this abstraction has been very leaky in Squeak, I suppose because of performance issues. (VW does a much better job of this - to convert strings to bytes, you have to specify an encoding, and when you re-encode a string you get a ByteArray and not another string. Immediate characters are a win.)
>
> As a result, Squeak is rife with code that assumes a particular encoding and treats strings as byte sequences rather than character sequences. That doesn't mean this instance is a good idea, though, it just means we have a lot of bad code floating around. Consider: if we *had* a tighter abstraction in String, switching the encoding would be much easier. Not trivial, because of string literals, source code encodings etc, but easier.
>
> BTW, I'm pretty familiar with issues arising from the encoding of Strings. I had code break under Squeak 3.8 because of m17n, and have long wrestled with encoding issues in web apps - strings go over the network in UTF-8, so either they get transcoded to MacRoman/Latin-1 inside the image and transcoded back to UTF-8 on the way out again, or you leave them in UTF-8 and live with the fact that you can't manipulate them inside the image.
>
> This particular case doesn't matter that much, but in general I'd like to see us moving in the direction of cleaner abstractions. Sure, there's lots of legacy code out there that makes assumptions about the internal structure of strings, but that's not an excuse to write more of it.
>
> Colin
>

>
> On 2010-01-19, at 10:04 AM, Andreas Raab wrote:
>
>> Regarding the "oh, this would be a problem if the string encoding changes", let's keep in mind that the string encoding *hasn't* changed for ASCII. So, no, it's specifically incorrect to say that the past change would have affected or invalidated that code. I don't think you understand how fundamental the impact of an encoding change is - if you did that *all* string literals would look like gobbly gook. The only reason we could do it for non-ascii was that we're not using any non-ascii characters and we were only switching the characters > 127 from Mac Roman to Unicode/Latin-1. If you don't believe me, grab your favorite non-ascii piece of text and throw it at "yourText squeakToMac" and have a look at it.
>>
>> So the argument that the dependency on literal string encoding is an issue is bogus. If you change literal string encoding there are so many other places that break it's not even funny. And at least I don't design for the implausible (what reason do we have to expect the encoding to change in the next fifty years?)
>
>
> Here's another way to think about it: #asByteArray violates the abstraction provided by String. A string is a sequence of characters, right? How that sequence is represented in memory is internal to the implementation, and should not be relied upon by users of the string. Historically this abstraction has been very leaky in Squeak, I suppose because of performance issues. (VW does a much better job of this - to convert strings to bytes, you have to specify an encoding, and when you re-encode a string you get a ByteArray and not another string. Immediate characters are a win.)
>
> As a result, Squeak is rife with code that assumes a particular encoding and treats strings as byte sequences rather than character sequences. That doesn't mean this instance is a good idea, though, it just means we have a lot of bad code floating around. Consider: if we *had* a tighter abstraction in String, switching the encoding would be much easier. Not trivial, because of string literals, source code encodings etc, but easier.
>
> BTW, I'm pretty familiar with issues arising from the encoding of Strings. I had code break under Squeak 3.8 because of m17n, and have long wrestled with encoding issues in web apps - strings go over the network in UTF-8, so either they get transcoded to MacRoman/Latin-1 inside the image and transcoded back to UTF-8 on the way out again, or you leave them in UTF-8 and live with the fact that you can't manipulate them inside the image.
>
> This particular case doesn't matter that much, but in general I'd like to see us moving in the direction of cleaner abstractions. Sure, there's lots of legacy code out there that makes assumptions about the internal structure of strings, but that's not an excuse to write more of it.

> 2010/1/21 Colin Putney <[hidden email]>:
>>
>> On 2010-01-19, at 10:04 AM, Andreas Raab wrote:
>>
>>> Regarding the "oh, this would be a problem if the string encoding changes", let's keep in mind that the string encoding *hasn't* changed for ASCII. So, no, it's specifically incorrect to say that the past change would have affected or invalidated that code. I don't think you understand how fundamental the impact of an encoding change is - if you did that *all* string literals would look like gobbly gook. The only reason we could do it for non-ascii was that we're not using any non-ascii characters and we were only switching the characters > 127 from Mac Roman to Unicode/Latin-1. If you don't believe me, grab your favorite non-ascii piece of text and throw it at "yourText squeakToMac" and have a look at it.
>>>
>>> So the argument that the dependency on literal string encoding is an issue is bogus. If you change literal string encoding there are so many other places that break it's not even funny. And at least I don't design for the implausible (what reason do we have to expect the encoding to change in the next fifty years?)
>>
>>
>> Here's another way to think about it: #asByteArray violates the abstraction provided by String. A string is a sequence of characters, right? How that sequence is represented in memory is internal to the implementation, and should not be relied upon by users of the string. Historically this abstraction has been very leaky in Squeak, I suppose because of performance issues. (VW does a much better job of this - to convert strings to bytes, you have to specify an encoding, and when you re-encode a string you get a ByteArray and not another string. Immediate characters are a win.)
>>
>> As a result, Squeak is rife with code that assumes a particular encoding and treats strings as byte sequences rather than character sequences. That doesn't mean this instance is a good idea, though, it just means we have a lot of bad code floating around. Consider: if we *had* a tighter abstraction in String, switching the encoding would be much easier. Not trivial, because of string literals, source code encodings etc, but easier.
>>
>> BTW, I'm pretty familiar with issues arising from the encoding of Strings. I had code break under Squeak 3.8 because of m17n, and have long wrestled with encoding issues in web apps - strings go over the network in UTF-8, so either they get transcoded to MacRoman/Latin-1 inside the image and transcoded back to UTF-8 on the way out again, or you leave them in UTF-8 and live with the fact that you can't manipulate them inside the image.
>>
>> This particular case doesn't matter that much, but in general I'd like to see us moving in the direction of cleaner abstractions. Sure, there's lots of legacy code out there that makes assumptions about the internal structure of strings, but that's not an excuse to write more of it.
>>
>> Colin
>>
>
> I'm not convinced.
> It's like saying that the fact that a true-type file begins with the
> de-facto universal encoding of 'true' is purely coincidental...
> IMO, it's the same expression of 'true' asByteArray and never was a
> series of bits 16r74727565 in the mind of whoever conceived the
> pattern.

>
> 2010/1/21 Colin Putney <[hidden email]>:
>>
>> On 2010-01-19, at 10:04 AM, Andreas Raab wrote:
>>
>>> Regarding the "oh, this would be a problem if the string encoding changes", let's keep in mind that the string encoding *hasn't* changed for ASCII. So, no, it's specifically incorrect to say that the past change would have affected or invalidated that code. I don't think you understand how fundamental the impact of an encoding change is - if you did that *all* string literals would look like gobbly gook. The only reason we could do it for non-ascii was that we're not using any non-ascii characters and we were only switching the characters > 127 from Mac Roman to Unicode/Latin-1. If you don't believe me, grab your favorite non-ascii piece of text and throw it at "yourText squeakToMac" and have a look at it.
>>>
>>> So the argument that the dependency on literal string encoding is an issue is bogus. If you change literal string encoding there are so many other places that break it's not even funny. And at least I don't design for the implausible (what reason do we have to expect the encoding to change in the next fifty years?)
>>
>>
>> Here's another way to think about it: #asByteArray violates the abstraction provided by String. A string is a sequence of characters, right? How that sequence is represented in memory is internal to the implementation, and should not be relied upon by users of the string. Historically this abstraction has been very leaky in Squeak, I suppose because of performance issues. (VW does a much better job of this - to convert strings to bytes, you have to specify an encoding, and when you re-encode a string you get a ByteArray and not another string. Immediate characters are a win.)
>>
>> As a result, Squeak is rife with code that assumes a particular encoding and treats strings as byte sequences rather than character sequences. That doesn't mean this instance is a good idea, though, it just means we have a lot of bad code floating around. Consider: if we *had* a tighter abstraction in String, switching the encoding would be much easier. Not trivial, because of string literals, source code encodings etc, but easier.
>>
>> BTW, I'm pretty familiar with issues arising from the encoding of Strings. I had code break under Squeak 3.8 because of m17n, and have long wrestled with encoding issues in web apps - strings go over the network in UTF-8, so either they get transcoded to MacRoman/Latin-1 inside the image and transcoded back to UTF-8 on the way out again, or you leave them in UTF-8 and live with the fact that you can't manipulate them inside the image.
>>
>> This particular case doesn't matter that much, but in general I'd like to see us moving in the direction of cleaner abstractions. Sure, there's lots of legacy code out there that makes assumptions about the internal structure of strings, but that's not an excuse to write more of it.
>>
>> Colin
>>
>
> I'm not convinced.
> It's like saying that the fact that a true-type file begins with the
> de-facto universal encoding of 'true' is purely coincidental...
> IMO, it's the same expression of 'true' asByteArray and never was a
> series of bits 16r74727565 in the mind of whoever conceived the
> pattern.
>
> Nicolas

> On 21.01.2010, at 08:02, Nicolas Cellier wrote:
>>
>> 2010/1/21 Colin Putney <[hidden email]>:
>>>
>>> On 2010-01-19, at 10:04 AM, Andreas Raab wrote:
>>>
>>>> Regarding the "oh, this would be a problem if the string encoding changes", let's keep in mind that the string encoding *hasn't* changed for ASCII. So, no, it's specifically incorrect to say that the past change would have affected or invalidated that code. I don't think you understand how fundamental the impact of an encoding change is - if you did that *all* string literals would look like gobbly gook. The only reason we could do it for non-ascii was that we're not using any non-ascii characters and we were only switching the characters > 127 from Mac Roman to Unicode/Latin-1. If you don't believe me, grab your favorite non-ascii piece of text and throw it at "yourText squeakToMac" and have a look at it.
>>>>
>>>> So the argument that the dependency on literal string encoding is an issue is bogus. If you change literal string encoding there are so many other places that break it's not even funny. And at least I don't design for the implausible (what reason do we have to expect the encoding to change in the next fifty years?)
>>>
>>>
>>> Here's another way to think about it: #asByteArray violates the abstraction provided by String. A string is a sequence of characters, right? How that sequence is represented in memory is internal to the implementation, and should not be relied upon by users of the string. Historically this abstraction has been very leaky in Squeak, I suppose because of performance issues. (VW does a much better job of this - to convert strings to bytes, you have to specify an encoding, and when you re-encode a string you get a ByteArray and not another string. Immediate characters are a win.)
>>>
>>> As a result, Squeak is rife with code that assumes a particular encoding and treats strings as byte sequences rather than character sequences. That doesn't mean this instance is a good idea, though, it just means we have a lot of bad code floating around. Consider: if we *had* a tighter abstraction in String, switching the encoding would be much easier. Not trivial, because of string literals, source code encodings etc, but easier.
>>>
>>> BTW, I'm pretty familiar with issues arising from the encoding of Strings. I had code break under Squeak 3.8 because of m17n, and have long wrestled with encoding issues in web apps - strings go over the network in UTF-8, so either they get transcoded to MacRoman/Latin-1 inside the image and transcoded back to UTF-8 on the way out again, or you leave them in UTF-8 and live with the fact that you can't manipulate them inside the image.
>>>
>>> This particular case doesn't matter that much, but in general I'd like to see us moving in the direction of cleaner abstractions. Sure, there's lots of legacy code out there that makes assumptions about the internal structure of strings, but that's not an excuse to write more of it.
>>>
>>> Colin
>>>
>>
>> I'm not convinced.
>> It's like saying that the fact that a true-type file begins with the
>> de-facto universal encoding of 'true' is purely coincidental...
>> IMO, it's the same expression of 'true' asByteArray and never was a
>> series of bits 16r74727565 in the mind of whoever conceived the
>> pattern.
>>
>> Nicolas
>
> IMHO in the TrueType case, 'true' asByteArray is just fine, because the behavior of #asByteArray is well-specified if the receiver happens to be an ASCII string. And readability counts.
>
> With that out of the way I have to side with Colin. Here is the problem in a nutshell:
>
> #(127 128 256) collect: [:c | c -> {
> c asCharacter asString asByteArray.
> c asCharacter asString squeakToUtf8 asByteArray}]
>
> ==> { 127->#( #[127]     #[127] ) .
> 128->#( #[128]     #[194 128] ) .
> 256->#( #[0 0 1 0] #[196 128] ) }
>