3.9 and encoding in multipart fields

> 2007/4/26, Norbert Hartl <[hidden email]>:
>> On Thu, 2007-04-26 at 16:48 +0200, Damien Cassou wrote:
>> > - you can't use string literals containing accented characters
>> That is a problem. I'm german and I need to use labels containing
>> Umlauts. What is the way to generate this character. Are strings
>> you get from WAKom different to the internal encoding squeak uses.
>
> What you get from WAKom is what you get from the distant
> user/webbrowser. I think it's always utf-8 because Seaside generates
> utf-8 but I'm not sure.
>
> About string literals, Lukas told me he uses  a #squeakToUtf method,
> but I don't know where to find it.
>

> 2007/4/26, Norbert Hartl <[hidden email]>:
> > On Thu, 2007-04-26 at 16:48 +0200, Damien Cassou wrote:
> > > - you can't use string literals containing accented characters
> > That is a problem. I'm german and I need to use labels containing
> > Umlauts. What is the way to generate this character. Are strings
> > you get from WAKom different to the internal encoding squeak uses.
>
> What you get from WAKom is what you get from the distant
> user/webbrowser. I think it's always utf-8 because Seaside generates
> utf-8 but I'm not sure.
>
> About string literals, Lukas told me he uses  a #squeakToUtf method,
> but I don't know where to find it.

> Hmmm, it is still not working. I have two forms. One is only text fields
> (multipart false) and one is text fields and input type="file"
> (multipart true). I store the values in the database using Glorp.
> I have two modifications to convert strings. One to utf8 when storing
> in the database and one to convert from utf8 when reading from the
> database.
>
> The multipart=false case leads to a proper store of the string in the
> database. The multipart=true does not. Using WAKomEncoded39 the none
> multipart case is displayed correctly. With WAKom the multipart case
> is displayed correct.
>
> Examine the kom requests I can see that on multipart=false the strings
> are displaying correct in the inspector. With multipart=true the
> strings are looking strange (no conversion to the squeak encoding???)
>
> I don't know much about the multipart stuff. Are the single parts from
> a multipart recognizable of which type they are (binary or text). I
> would like to do a conversion if it is text. Do you have any hint where
> in WAKom this encoding should occurr.
>

> On 4/26/07, John Thornborrow <[hidden email]> wrote:
>
> > var _ 'thisisastring' squeakToIso isoToUtf
> >
> > is the best I can find.
>
> Here's what I use on Squeak 3.7:
>
> Integer>>asUTF8
> ^ self <= 16r7F
> ifTrue: [self asCharacter asString]
> ifFalse:
> [self <= 16r7FF
> ifTrue:
> [String
> with: (16rC0 bitOr: (self bitShift: -6)) asCharacter
> with: (16r80 bitOr: (self bitAnd: (16r3F))) asCharacter]
> ifFalse:
> [String
> with: (16rE0 bitOr: (self bitShift: -12)) asCharacter
> with: (16r80 bitOr: ((self bitAnd: 16rFC0) bitShift: -6)) asCharacter
> with: (16r80 bitOr: (self bitAnd: 16r3F)) asCharacter]]
>
>
> Then you can do things like "247 asUTF8" (which is a division symbol,
> for example).

> http://www.lukas-renggli.ch/blog/studenckifestwal for an example.
>
> The only problems are:
> - you won't have a meaningful result if you send #size to your strings.
> - you won't be able to read the data through Squeak inspectors.
> - you can't use string literals containing accented characters
> directly in your Squeak code.
>
> The rest should just work.
>
> --
> Damien Cassou
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

> .
> > Hmmm, it is still not working. I have two forms. One is only text fields
> > (multipart false) and one is text fields and input type="file"
> > (multipart true). I store the values in the database using Glorp.
> > I have two modifications to convert strings. One to utf8 when storing
> > in the database and one to convert from utf8 when reading from the
> > database.
> >
> > The multipart=false case leads to a proper store of the string in the
> > database. The multipart=true does not. Using WAKomEncoded39 the none
> > multipart case is displayed correctly. With WAKom the multipart case
> > is displayed correct.
> >
> > Examine the kom requests I can see that on multipart=false the strings
> > are displaying correct in the inspector. With multipart=true the
> > strings are looking strange (no conversion to the squeak encoding???)
> >
> > I don't know much about the multipart stuff. Are the single parts from
> > a multipart recognizable of which type they are (binary or text). I
> > would like to do a conversion if it is text. Do you have any hint where
> > in WAKom this encoding should occurr.
> >
> Ok, now I tested my theory. I added some code to WAKom to convert
> multipart fields to an internal representation.
>
> processMultipartFields: aRequest
>         aRequest multipartFormFieldsDo:
>                 [:chunk |
>                 chunk explore.
>                 chunk fileName isEmptyOrNil ifFalse:
>                         [|stream file|
>                         stream := WriteStream on: String new.
>                         chunk saveToStream: stream.
>                         file := WAFile new
>                                 fileName: chunk fileName;
>                                 contents: stream contents;
>                                 contentType: chunk contentType;
>                                 yourself.
>                         aRequest postFields at: chunk fieldName put: file]
>                 ifTrue: [
>                         |stream|
>                         stream := WriteStream on: String new.
>                         chunk saveToStream: stream.
>
>                         aRequest postFields at: chunk fieldName put:( (stream contents)
> convertFromEncoding: #utf8)]
>                 ].

> 2007/4/26, Norbert Hartl <[hidden email]>:
> > .
> > > Hmmm, it is still not working. I have two forms. One is only text fields
> > > (multipart false) and one is text fields and input type="file"
> > > (multipart true). I store the values in the database using Glorp.
> > > I have two modifications to convert strings. One to utf8 when storing
> > > in the database and one to convert from utf8 when reading from the
> > > database.
> > >
> > > The multipart=false case leads to a proper store of the string in the
> > > database. The multipart=true does not. Using WAKomEncoded39 the none
> > > multipart case is displayed correctly. With WAKom the multipart case
> > > is displayed correct.
> > >
> > > Examine the kom requests I can see that on multipart=false the strings
> > > are displaying correct in the inspector. With multipart=true the
> > > strings are looking strange (no conversion to the squeak encoding???)
> > >
> > > I don't know much about the multipart stuff. Are the single parts from
> > > a multipart recognizable of which type they are (binary or text). I
> > > would like to do a conversion if it is text. Do you have any hint where
> > > in WAKom this encoding should occurr.
> > >
> > Ok, now I tested my theory. I added some code to WAKom to convert
> > multipart fields to an internal representation.
> >
> > processMultipartFields: aRequest
> >         aRequest multipartFormFieldsDo:
> >                 [:chunk |
> >                 chunk explore.
> >                 chunk fileName isEmptyOrNil ifFalse:
> >                         [|stream file|
> >                         stream := WriteStream on: String new.
> >                         chunk saveToStream: stream.
> >                         file := WAFile new
> >                                 fileName: chunk fileName;
> >                                 contents: stream contents;
> >                                 contentType: chunk contentType;
> >                                 yourself.
> >                         aRequest postFields at: chunk fieldName put: file]
> >                 ifTrue: [
> >                         |stream|
> >                         stream := WriteStream on: String new.
> >                         chunk saveToStream: stream.
> >
> >                         aRequest postFields at: chunk fieldName put:( (stream contents)
> > convertFromEncoding: #utf8)]
> >                 ].
>
> Do I see correctly that the #ifTrue: part ist new?
>

> > ....
> > - converting from utf-8 when reading from the database
>
> See above.
>
> > - Using WAKomEncoded39 which encodes to utf-8 before sending to the
> >   client.
> >
> > It is much encoding but relatively clean (I think). I can use Umlauts
> > in squeak code (being squeak encoding).
>
> Yes, in Squeak >= 3.8 and WAKomEncoded(39)

> On Thu, 2007-04-26 at 23:17 +0200, Philippe Marschall wrote:
> > 2007/4/26, Norbert Hartl <[hidden email]>:
> > > .
> > > > Hmmm, it is still not working. I have two forms. One is only text fields
> > > > (multipart false) and one is text fields and input type="file"
> > > > (multipart true). I store the values in the database using Glorp.
> > > > I have two modifications to convert strings. One to utf8 when storing
> > > > in the database and one to convert from utf8 when reading from the
> > > > database.
> > > >
> > > > The multipart=false case leads to a proper store of the string in the
> > > > database. The multipart=true does not. Using WAKomEncoded39 the none
> > > > multipart case is displayed correctly. With WAKom the multipart case
> > > > is displayed correct.
> > > >
> > > > Examine the kom requests I can see that on multipart=false the strings
> > > > are displaying correct in the inspector. With multipart=true the
> > > > strings are looking strange (no conversion to the squeak encoding???)
> > > >
> > > > I don't know much about the multipart stuff. Are the single parts from
> > > > a multipart recognizable of which type they are (binary or text). I
> > > > would like to do a conversion if it is text. Do you have any hint where
> > > > in WAKom this encoding should occurr.
> > > >
> > > Ok, now I tested my theory. I added some code to WAKom to convert
> > > multipart fields to an internal representation.
> > >
> > > processMultipartFields: aRequest
> > >         aRequest multipartFormFieldsDo:
> > >                 [:chunk |
> > >                 chunk explore.
> > >                 chunk fileName isEmptyOrNil ifFalse:
> > >                         [|stream file|
> > >                         stream := WriteStream on: String new.
> > >                         chunk saveToStream: stream.
> > >                         file := WAFile new
> > >                                 fileName: chunk fileName;
> > >                                 contents: stream contents;
> > >                                 contentType: chunk contentType;
> > >                                 yourself.
> > >                         aRequest postFields at: chunk fieldName put: file]
> > >                 ifTrue: [
> > >                         |stream|
> > >                         stream := WriteStream on: String new.
> > >                         chunk saveToStream: stream.
> > >
> > >                         aRequest postFields at: chunk fieldName put:( (stream contents)
> > > convertFromEncoding: #utf8)]
> > >                 ].
> >
> > Do I see correctly that the #ifTrue: part ist new?
> >
> yes.
> > > That works for me now. It is slower than any other approach while
> > > doing:
> > >
> > > - getting UTF-8 from the client (is that always true or how does WAKom
> > >   decide?)
> >
> > WAKom does _nothing_ whatever the clients sends to you in whatever
> > encoding will be what you get.
> >
> Ok, that isn't very good. So I have to figure out the cases where I
> don't get utf-8 and cope with that.

> > > - converting multipart and none multipart fields from utf-8 to squeak
> > > - converting the string to utf-8 when storing in the database of the
> > >   PostgresQL client encoding utf-8
> >
> > Your job. Seaside does not know or care about databases.
> >
> Seaside doesn't but squeak should :)
> > > ....
> > > - converting from utf-8 when reading from the database
> >
> > See above.
> >
> > > - Using WAKomEncoded39 which encodes to utf-8 before sending to the
> > >   client.
> > >
> > > It is much encoding but relatively clean (I think). I can use Umlauts
> > > in squeak code (being squeak encoding).
> >
> > Yes, in Squeak >= 3.8 and WAKomEncoded(39)
>
> That is what I'm using.
>
> Norbert
>
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

> 2007/4/26, Norbert Hartl <[hidden email]>:
> > On Thu, 2007-04-26 at 23:17 +0200, Philippe Marschall wrote:
> > > 2007/4/26, Norbert Hartl <[hidden email]>:
> > > > .
> > > > > Hmmm, it is still not working. I have two forms. One is only text fields
> > > > > (multipart false) and one is text fields and input type="file"
> > > > > (multipart true). I store the values in the database using Glorp.
> > > > > I have two modifications to convert strings. One to utf8 when storing
> > > > > in the database and one to convert from utf8 when reading from the
> > > > > database.
> > > > >
> > > > > The multipart=false case leads to a proper store of the string in the
> > > > > database. The multipart=true does not. Using WAKomEncoded39 the none
> > > > > multipart case is displayed correctly. With WAKom the multipart case
> > > > > is displayed correct.
> > > > >
> > > > > Examine the kom requests I can see that on multipart=false the strings
> > > > > are displaying correct in the inspector. With multipart=true the
> > > > > strings are looking strange (no conversion to the squeak encoding???)
> > > > >
> > > > > I don't know much about the multipart stuff. Are the single parts from
> > > > > a multipart recognizable of which type they are (binary or text). I
> > > > > would like to do a conversion if it is text. Do you have any hint where
> > > > > in WAKom this encoding should occurr.
> > > > >
> > > > Ok, now I tested my theory. I added some code to WAKom to convert
> > > > multipart fields to an internal representation.
> > > >
> > > > processMultipartFields: aRequest
> > > >         aRequest multipartFormFieldsDo:
> > > >                 [:chunk |
> > > >                 chunk explore.
> > > >                 chunk fileName isEmptyOrNil ifFalse:
> > > >                         [|stream file|
> > > >                         stream := WriteStream on: String new.
> > > >                         chunk saveToStream: stream.
> > > >                         file := WAFile new
> > > >                                 fileName: chunk fileName;
> > > >                                 contents: stream contents;
> > > >                                 contentType: chunk contentType;
> > > >                                 yourself.
> > > >                         aRequest postFields at: chunk fieldName put: file]
> > > >                 ifTrue: [
> > > >                         |stream|
> > > >                         stream := WriteStream on: String new.
> > > >                         chunk saveToStream: stream.
> > > >
> > > >                         aRequest postFields at: chunk fieldName put:( (stream contents)
> > > > convertFromEncoding: #utf8)]
> > > >                 ].
> > >
> > > Do I see correctly that the #ifTrue: part ist new?
> > >
> > yes.
> > > > That works for me now. It is slower than any other approach while
> > > > doing:
> > > >
> > > > - getting UTF-8 from the client (is that always true or how does WAKom
> > > >   decide?)
> > >
> > > WAKom does _nothing_ whatever the clients sends to you in whatever
> > > encoding will be what you get.
> > >
> > Ok, that isn't very good. So I have to figure out the cases where I
> > don't get utf-8 and cope with that.
>
> Then use the WAKomEncoded* series, they will deliver you WideStrings
> and expect WideStrings for you (and use and expect utf8 externally,
> you could make subclasses that do something else, it's just that
> nobody ever needed anything besides utf8).
>

> On Thu, 2007-04-26 at 23:54 +0200, Philippe Marschall wrote:
> > 2007/4/26, Norbert Hartl <[hidden email]>:
> > > On Thu, 2007-04-26 at 23:17 +0200, Philippe Marschall wrote:
> > > > 2007/4/26, Norbert Hartl <[hidden email]>:
> > > > > .
> > > > > > Hmmm, it is still not working. I have two forms. One is only text fields
> > > > > > (multipart false) and one is text fields and input type="file"
> > > > > > (multipart true). I store the values in the database using Glorp.
> > > > > > I have two modifications to convert strings. One to utf8 when storing
> > > > > > in the database and one to convert from utf8 when reading from the
> > > > > > database.
> > > > > >
> > > > > > The multipart=false case leads to a proper store of the string in the
> > > > > > database. The multipart=true does not. Using WAKomEncoded39 the none
> > > > > > multipart case is displayed correctly. With WAKom the multipart case
> > > > > > is displayed correct.
> > > > > >
> > > > > > Examine the kom requests I can see that on multipart=false the strings
> > > > > > are displaying correct in the inspector. With multipart=true the
> > > > > > strings are looking strange (no conversion to the squeak encoding???)
> > > > > >
> > > > > > I don't know much about the multipart stuff. Are the single parts from
> > > > > > a multipart recognizable of which type they are (binary or text). I
> > > > > > would like to do a conversion if it is text. Do you have any hint where
> > > > > > in WAKom this encoding should occurr.
> > > > > >
> > > > > Ok, now I tested my theory. I added some code to WAKom to convert
> > > > > multipart fields to an internal representation.
> > > > >
> > > > > processMultipartFields: aRequest
> > > > >         aRequest multipartFormFieldsDo:
> > > > >                 [:chunk |
> > > > >                 chunk explore.
> > > > >                 chunk fileName isEmptyOrNil ifFalse:
> > > > >                         [|stream file|
> > > > >                         stream := WriteStream on: String new.
> > > > >                         chunk saveToStream: stream.
> > > > >                         file := WAFile new
> > > > >                                 fileName: chunk fileName;
> > > > >                                 contents: stream contents;
> > > > >                                 contentType: chunk contentType;
> > > > >                                 yourself.
> > > > >                         aRequest postFields at: chunk fieldName put: file]
> > > > >                 ifTrue: [
> > > > >                         |stream|
> > > > >                         stream := WriteStream on: String new.
> > > > >                         chunk saveToStream: stream.
> > > > >
> > > > >                         aRequest postFields at: chunk fieldName put:( (stream contents)
> > > > > convertFromEncoding: #utf8)]
> > > > >                 ].
> > > >
> > > > Do I see correctly that the #ifTrue: part ist new?
> > > >
> > > yes.
> > > > > That works for me now. It is slower than any other approach while
> > > > > doing:
> > > > >
> > > > > - getting UTF-8 from the client (is that always true or how does WAKom
> > > > >   decide?)
> > > >
> > > > WAKom does _nothing_ whatever the clients sends to you in whatever
> > > > encoding will be what you get.
> > > >
> > > Ok, that isn't very good. So I have to figure out the cases where I
> > > don't get utf-8 and cope with that.
> >
> > Then use the WAKomEncoded* series, they will deliver you WideStrings
> > and expect WideStrings for you (and use and expect utf8 externally,
> > you could make subclasses that do something else, it's just that
> > nobody ever needed anything besides utf8).
> >
> I can see WAKomEncoded39 is converting to utf-8 before sending the
> respose. WAKomEncoded does some conversion I don't understand. Under
> which circumstances is a field kind of OrderedCollection?