Postgres / Glorp
Locked
12
12
 
|
I was unable to use databases with UTF8 encoding, only SQL_ASCII
encoding worked for me. is it something trivial to configure that I did not see or is it a limitation? Thanks r. |
|
|
Sorry, it does work.
I don´t know what happend. On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > I was unable to use databases with UTF8 encoding, only SQL_ASCII > encoding worked for me. > is it something trivial to configure that I did not see or is it a limitation? > Thanks > > r. > |
|
|
In reply to this post by Ramiro Diaz Trepat
Sorry, it does work.
I don´t know what happend. On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > I was unable to use databases with UTF8 encoding, only SQL_ASCII > encoding worked for me. > is it something trivial to configure that I did not see or is it a limitation? > Thanks > > r. > |
|
|
In reply to this post by Ramiro Diaz Trepat
Well... sorry for building this thread by myself.
The issue now is that conversion does not happen automatically. I can properly store a string on a UTF8 database, by previously converting it using #isoToUtf8. But when strings come back after a query (using Glorp), they are not converted to iso, and hence they show all screwed up. Is there any place in Glorp or the Postgres driver to configure the encoding of the underlying database for these conversions to happen automatically? Thanks ! r. On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > Sorry, it does work. > I don´t know what happend. > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > > I was unable to use databases with UTF8 encoding, only SQL_ASCII > > encoding worked for me. > > is it something trivial to configure that I did not see or is it a limitation? > > Thanks > > > > r. > > > |
|
|
What version of Squeak are you using?
On May 13, 2007, at 2:59 PM, Ramiro Diaz Trepat wrote: > Well... sorry for building this thread by myself. > The issue now is that conversion does not happen automatically. I can > properly store a string on a UTF8 database, by previously converting > it using #isoToUtf8. > But when strings come back after a query (using Glorp), they are not > converted to iso, and hence they show all screwed up. > Is there any place in Glorp or the Postgres driver to configure the > encoding of the underlying database for these conversions to happen > automatically? > Thanks ! > > r. > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: >> Sorry, it does work. >> I don´t know what happend. >> >> >> On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: >> > I was unable to use databases with UTF8 encoding, only SQL_ASCII >> > encoding worked for me. >> > is it something trivial to configure that I did not see or is it >> a limitation? >> > Thanks >> > >> > r. >> > >> > |
|
|
In reply to this post by Ramiro Diaz Trepat
Ramiro
I'm postgres user , i hope help you , postgres encondig can be configurated at database creation, postgres client can configurated using statements SET CLIENT_ENCODING TO 'value'; take a look at http://www.postgresql.org/docs/8.2/static/multibyte.html best regards mdc --- Ramiro Diaz Trepat <[hidden email]> escribió: > Well... sorry for building this thread by myself. > The issue now is that conversion does not happen > automatically. I can > properly store a string on a UTF8 database, by > previously converting > it using #isoToUtf8. > But when strings come back after a query (using > Glorp), they are not > converted to iso, and hence they show all screwed > up. > Is there any place in Glorp or the Postgres driver > to configure the > encoding of the underlying database for these > conversions to happen > automatically? > Thanks ! > > r. > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > wrote: > > Sorry, it does work. > > I don´t know what happend. > > > > > > On 5/13/07, Ramiro Diaz Trepat > <[hidden email]> wrote: > > > I was unable to use databases with UTF8 > encoding, only SQL_ASCII > > > encoding worked for me. > > > is it something trivial to configure that I did > not see or is it a limitation? > > > Thanks > > > > > > r. > > > > > > > __________________________________________________ Preguntá. Respondé. Descubrí. Todo lo que querías saber, y lo que ni imaginabas, está en Yahoo! Respuestas (Beta). ¡Probalo ya! http://www.yahoo.com.ar/respuestas |
|
|
Hello Todd and Marcelo,
I am using Squeak 3.9. I know about how to set the encoding on postgres. Indeed, utf8 is the encoding I need on the database. What I want to know is if there is any place on the Postgres, Glorp or MagritteGlorp packages where I could configure for the strings to be translated to Squeak´s default (I think it´s iso). If I have, for example a Person on Glorp, and I want to set a last name like mine Díaz (with an accented vowel), coming from a Magritte form for instance, on the setter I translate it to UTF8 (#isoToUtf8) before I assign it to my class i.var. And it gets stored on the database with no errors. Later on, when Glorp "hydrates" my instance, of course it brings a weird string like "DÃ≈az", instead of "Díaz". Of course I would love that conversion to happen automatically with a little configuration. Otherwise, what are you guys doing? On 5/13/07, marcelo Cortez <[hidden email]> wrote: > Ramiro > > I'm postgres user , i hope help you , postgres > encondig can be configurated at database creation, > postgres client can configurated using statements > SET CLIENT_ENCODING TO 'value'; > take a look at > http://www.postgresql.org/docs/8.2/static/multibyte.html > best regards > mdc > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > Well... sorry for building this thread by myself. > > The issue now is that conversion does not happen > > automatically. I can > > properly store a string on a UTF8 database, by > > previously converting > > it using #isoToUtf8. > > But when strings come back after a query (using > > Glorp), they are not > > converted to iso, and hence they show all screwed > > up. > > Is there any place in Glorp or the Postgres driver > > to configure the > > encoding of the underlying database for these > > conversions to happen > > automatically? > > Thanks ! > > > > r. > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > wrote: > > > Sorry, it does work. > > > I don´t know what happend. > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > <[hidden email]> wrote: > > > > I was unable to use databases with UTF8 > > encoding, only SQL_ASCII > > > > encoding worked for me. > > > > is it something trivial to configure that I did > > not see or is it a limitation? > > > > Thanks > > > > > > > > r. > > > > > > > > > > > > > > > > > > > __________________________________________________ > Preguntá. Respondé. Descubrí. > Todo lo que querías saber, y lo que ni imaginabas, > está en Yahoo! Respuestas (Beta). > ¡Probalo ya! > http://www.yahoo.com.ar/respuestas > > > |
|
|
Indeed, probably another great thing it would be if I could configure
Seaside to directly send me utf8 strings from the input taken on a form´s entry fields. Is it possible? On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > Hello Todd and Marcelo, > I am using Squeak 3.9. > I know about how to set the encoding on postgres. Indeed, utf8 is > the encoding I need on the database. > What I want to know is if there is any place on the Postgres, Glorp > or MagritteGlorp packages where I could configure for the strings to > be translated to Squeak´s default (I think it´s iso). > If I have, for example a Person on Glorp, and I want to set a last > name like mine Díaz (with an accented vowel), coming from a Magritte > form for instance, on the setter I translate it to UTF8 (#isoToUtf8) > before I assign it to my class i.var. And it gets stored on the > database with no errors. Later on, when Glorp "hydrates" my instance, > of course it brings a weird string like "DÃ≈az", instead of "Díaz". > Of course I would love that conversion to happen automatically with > a little configuration. > Otherwise, what are you guys doing? > > > > On 5/13/07, marcelo Cortez <[hidden email]> wrote: > > Ramiro > > > > I'm postgres user , i hope help you , postgres > > encondig can be configurated at database creation, > > postgres client can configurated using statements > > SET CLIENT_ENCODING TO 'value'; > > take a look at > > http://www.postgresql.org/docs/8.2/static/multibyte.html > > best regards > > mdc > > > > > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > > > Well... sorry for building this thread by myself. > > > The issue now is that conversion does not happen > > > automatically. I can > > > properly store a string on a UTF8 database, by > > > previously converting > > > it using #isoToUtf8. > > > But when strings come back after a query (using > > > Glorp), they are not > > > converted to iso, and hence they show all screwed > > > up. > > > Is there any place in Glorp or the Postgres driver > > > to configure the > > > encoding of the underlying database for these > > > conversions to happen > > > automatically? > > > Thanks ! > > > > > > r. > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > > wrote: > > > > Sorry, it does work. > > > > I don´t know what happend. > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > > <[hidden email]> wrote: > > > > > I was unable to use databases with UTF8 > > > encoding, only SQL_ASCII > > > > > encoding worked for me. > > > > > is it something trivial to configure that I did > > > not see or is it a limitation? > > > > > Thanks > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > Preguntá. Respondé. Descubrí. > > Todo lo que querías saber, y lo que ni imaginabas, > > está en Yahoo! Respuestas (Beta). > > ¡Probalo ya! > > http://www.yahoo.com.ar/respuestas > > > > > > > |
|
|
One more question.
Is there a way to test if a String is valid in some encoding, or to test if a particular string is already in UTF8 ? Because, if I want to force all my string attributes to be in UTF8, I have to know if the String I am getting is already encoded. In my example class Person, it would be great if I could do something like: Person>>lastName: aLastName lastName := aLastName ifNotNil: [ (UTF8Tools isUtf8: aLastName) ifTrue: [ aLastName ] ifFalse: [ aLastName isoToUtf8 ]] On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > Indeed, probably another great thing it would be if I could configure > Seaside to directly send me utf8 strings from the input taken on a > form´s entry fields. > Is it possible? > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > > Hello Todd and Marcelo, > > I am using Squeak 3.9. > > I know about how to set the encoding on postgres. Indeed, utf8 is > > the encoding I need on the database. > > What I want to know is if there is any place on the Postgres, Glorp > > or MagritteGlorp packages where I could configure for the strings to > > be translated to Squeak´s default (I think it´s iso). > > If I have, for example a Person on Glorp, and I want to set a last > > name like mine Díaz (with an accented vowel), coming from a Magritte > > form for instance, on the setter I translate it to UTF8 (#isoToUtf8) > > before I assign it to my class i.var. And it gets stored on the > > database with no errors. Later on, when Glorp "hydrates" my instance, > > of course it brings a weird string like "DÃ≈az", instead of "Díaz". > > Of course I would love that conversion to happen automatically with > > a little configuration. > > Otherwise, what are you guys doing? > > > > > > > > On 5/13/07, marcelo Cortez <[hidden email]> wrote: > > > Ramiro > > > > > > I'm postgres user , i hope help you , postgres > > > encondig can be configurated at database creation, > > > postgres client can configurated using statements > > > SET CLIENT_ENCODING TO 'value'; > > > take a look at > > > http://www.postgresql.org/docs/8.2/static/multibyte.html > > > best regards > > > mdc > > > > > > > > > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > > > > > Well... sorry for building this thread by myself. > > > > The issue now is that conversion does not happen > > > > automatically. I can > > > > properly store a string on a UTF8 database, by > > > > previously converting > > > > it using #isoToUtf8. > > > > But when strings come back after a query (using > > > > Glorp), they are not > > > > converted to iso, and hence they show all screwed > > > > up. > > > > Is there any place in Glorp or the Postgres driver > > > > to configure the > > > > encoding of the underlying database for these > > > > conversions to happen > > > > automatically? > > > > Thanks ! > > > > > > > > r. > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > > > wrote: > > > > > Sorry, it does work. > > > > > I don´t know what happend. > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > > > <[hidden email]> wrote: > > > > > > I was unable to use databases with UTF8 > > > > encoding, only SQL_ASCII > > > > > > encoding worked for me. > > > > > > is it something trivial to configure that I did > > > > not see or is it a limitation? > > > > > > Thanks > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > Preguntá. Respondé. Descubrí. > > > Todo lo que querías saber, y lo que ni imaginabas, > > > está en Yahoo! Respuestas (Beta). > > > ¡Probalo ya! > > > http://www.yahoo.com.ar/respuestas > > > > > > > > > > > > |
|
|
In reply to this post by Ramiro Diaz Trepat
On Sun, 2007-05-13 at 18:28 -0300, Ramiro Diaz Trepat wrote:
> Sorry, it does work. > I don´t know what happend. > > Try more to be sure :) I had luck for a few weeks but then... I post the message I send you in private to the list so it is archived. --- There are four places where encoding plays a role: 1. getting data from the web into squeak 2. putting data from squeak into a database 3. getting data from the database into squeak 4. putting the data into the web. 1. from the web works usually quite well but not for multipart forms (if you have something for upload). I exchanged a WAKom method: WAKom>>processMultipartFields: aRequest aRequest multipartFormFieldsDo: [:chunk | chunk fileName isEmptyOrNil ifFalse: [|stream file| stream := WriteStream on: String new. chunk saveToStream: stream. file := WAFile new fileName: chunk fileName; contents: stream contents; contentType: chunk contentType; yourself. aRequest postFields at: chunk fieldName put: file] ifTrue: [ |stream| stream := WriteStream on: String new. chunk saveToStream: stream. aRequest postFields at: chunk fieldName put:( (stream contents) convertFromEncoding: #utf8)] ]. 2. squeak to database herefor I assume that the client encoding between squeak and postgres is UTF-8. I just encode the complete outgoing SQL string to utf-8. SqueakDatabaseAccessor>>basicExecuteSQLString: aString | result rowCollection | "self log: aString." result := connection execute: (aString convertToEncoding: #utf8). "result := connection execute: aString." result errorResponse notNil ifTrue: [self externalDatabaseErrorSignal signal: result errorResponse value]. rowCollection := OrderedCollection new. result rows do: [:ea | rowCollection add: ea data asArray]. ^ReadStream on: rowCollection asArray 3. database to squeak here the same client encoding applies. To convert these you have to register type converters. I did this for varchar and text fields. Add #(1043 25) do: [:each| converters at: each put: [:s | s convertFromEncoding: #utf8]]. somewhere in PGConnection class>>buildDefaultFieldConverters 4. squeak to web is the easiest as it is solved by using WAKomEncoded39 instead of WAKom. This approach solves all four ends where encoding plays a role. It means there happens a lot of conversion but the approach is clean in a way that you use inside squeak its own encoding. You're able to use special spanisch characters inside squeak and they are displayed properly in squeak and on the web pages. Hope this helps, Norbert |
|
|
In reply to this post by Ramiro Diaz Trepat
2007/5/14, Ramiro Diaz Trepat <[hidden email]>:
> Indeed, probably another great thing it would be if I could configure > Seaside to directly send me utf8 strings from the input taken on a > form´s entry fields. > Is it possible? Yes, this is what WAKom is doing. Just remember that you also have to send utf8 strings to Seaside. Unfortunately WAKom does not work on Squeak 3.9. Cheers Philippe > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > > Hello Todd and Marcelo, > > I am using Squeak 3.9. > > I know about how to set the encoding on postgres. Indeed, utf8 is > > the encoding I need on the database. > > What I want to know is if there is any place on the Postgres, Glorp > > or MagritteGlorp packages where I could configure for the strings to > > be translated to Squeak´s default (I think it´s iso). > > If I have, for example a Person on Glorp, and I want to set a last > > name like mine Díaz (with an accented vowel), coming from a Magritte > > form for instance, on the setter I translate it to UTF8 (#isoToUtf8) > > before I assign it to my class i.var. And it gets stored on the > > database with no errors. Later on, when Glorp "hydrates" my instance, > > of course it brings a weird string like "DÃ≈az", instead of "Díaz". > > Of course I would love that conversion to happen automatically with > > a little configuration. > > Otherwise, what are you guys doing? > > > > > > > > On 5/13/07, marcelo Cortez <[hidden email]> wrote: > > > Ramiro > > > > > > I'm postgres user , i hope help you , postgres > > > encondig can be configurated at database creation, > > > postgres client can configurated using statements > > > SET CLIENT_ENCODING TO 'value'; > > > take a look at > > > http://www.postgresql.org/docs/8.2/static/multibyte.html > > > best regards > > > mdc > > > > > > > > > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > > > > > Well... sorry for building this thread by myself. > > > > The issue now is that conversion does not happen > > > > automatically. I can > > > > properly store a string on a UTF8 database, by > > > > previously converting > > > > it using #isoToUtf8. > > > > But when strings come back after a query (using > > > > Glorp), they are not > > > > converted to iso, and hence they show all screwed > > > > up. > > > > Is there any place in Glorp or the Postgres driver > > > > to configure the > > > > encoding of the underlying database for these > > > > conversions to happen > > > > automatically? > > > > Thanks ! > > > > > > > > r. > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > > > wrote: > > > > > Sorry, it does work. > > > > > I don´t know what happend. > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > > > <[hidden email]> wrote: > > > > > > I was unable to use databases with UTF8 > > > > encoding, only SQL_ASCII > > > > > > encoding worked for me. > > > > > > is it something trivial to configure that I did > > > > not see or is it a limitation? > > > > > > Thanks > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > Preguntá. Respondé. Descubrí. > > > Todo lo que querías saber, y lo que ni imaginabas, > > > está en Yahoo! Respuestas (Beta). > > > ¡Probalo ya! > > > http://www.yahoo.com.ar/respuestas > > > > > > > > > > > > > > > |
|
|
Wow...
So I assume that mostly everyone not in an English speaking country with a Seaside application in production is using Squeak 3.8? Is that correct? On 5/14/07, Philippe Marschall <[hidden email]> wrote: > 2007/5/14, Ramiro Diaz Trepat <[hidden email]>: > > Indeed, probably another great thing it would be if I could configure > > Seaside to directly send me utf8 strings from the input taken on a > > form´s entry fields. > > Is it possible? > > Yes, this is what WAKom is doing. Just remember that you also have to > send utf8 strings to Seaside. Unfortunately WAKom does not work on > Squeak 3.9. > > Cheers > Philippe > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > > > Hello Todd and Marcelo, > > > I am using Squeak 3.9. > > > I know about how to set the encoding on postgres. Indeed, utf8 is > > > the encoding I need on the database. > > > What I want to know is if there is any place on the Postgres, Glorp > > > or MagritteGlorp packages where I could configure for the strings to > > > be translated to Squeak´s default (I think it´s iso). > > > If I have, for example a Person on Glorp, and I want to set a last > > > name like mine Díaz (with an accented vowel), coming from a Magritte > > > form for instance, on the setter I translate it to UTF8 (#isoToUtf8) > > > before I assign it to my class i.var. And it gets stored on the > > > database with no errors. Later on, when Glorp "hydrates" my instance, > > > of course it brings a weird string like "DÃ≈az", instead of "Díaz". > > > Of course I would love that conversion to happen automatically with > > > a little configuration. > > > Otherwise, what are you guys doing? > > > > > > > > > > > > On 5/13/07, marcelo Cortez <[hidden email]> wrote: > > > > Ramiro > > > > > > > > I'm postgres user , i hope help you , postgres > > > > encondig can be configurated at database creation, > > > > postgres client can configurated using statements > > > > SET CLIENT_ENCODING TO 'value'; > > > > take a look at > > > > http://www.postgresql.org/docs/8.2/static/multibyte.html > > > > best regards > > > > mdc > > > > > > > > > > > > > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > > > > > > > Well... sorry for building this thread by myself. > > > > > The issue now is that conversion does not happen > > > > > automatically. I can > > > > > properly store a string on a UTF8 database, by > > > > > previously converting > > > > > it using #isoToUtf8. > > > > > But when strings come back after a query (using > > > > > Glorp), they are not > > > > > converted to iso, and hence they show all screwed > > > > > up. > > > > > Is there any place in Glorp or the Postgres driver > > > > > to configure the > > > > > encoding of the underlying database for these > > > > > conversions to happen > > > > > automatically? > > > > > Thanks ! > > > > > > > > > > r. > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > > > > wrote: > > > > > > Sorry, it does work. > > > > > > I don´t know what happend. > > > > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > > > > <[hidden email]> wrote: > > > > > > > I was unable to use databases with UTF8 > > > > > encoding, only SQL_ASCII > > > > > > > encoding worked for me. > > > > > > > is it something trivial to configure that I did > > > > > not see or is it a limitation? > > > > > > > Thanks > > > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > Preguntá. Respondé. Descubrí. > > > > Todo lo que querías saber, y lo que ni imaginabas, > > > > está en Yahoo! Respuestas (Beta). > > > > ¡Probalo ya! > > > > http://www.yahoo.com.ar/respuestas > > > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
In reply to this post by NorbertHartl
Thank you very much Norbert.
It would be great if someone with an important blog (Ramon/Lukas) posted an article about how to get around the current limitations of Squeak/Seaside to work with UTF8 and the possible work arounds, like this. Cheers, and thanks again. On 5/14/07, Norbert Hartl <[hidden email]> wrote: > On Sun, 2007-05-13 at 18:28 -0300, Ramiro Diaz Trepat wrote: > > Sorry, it does work. > > I don´t know what happend. > > > > > Try more to be sure :) I had luck for a few weeks but then... > I post the message I send you in private to the list so > it is archived. > > --- > > There are four places where encoding plays a role: > > 1. getting data from the web into squeak > 2. putting data from squeak into a database > 3. getting data from the database into squeak > 4. putting the data into the web. > > > 1. from the web > > works usually quite well but not for multipart forms (if you > have something for upload). I exchanged a WAKom method: > > WAKom>>processMultipartFields: aRequest > aRequest multipartFormFieldsDo: > [:chunk | > chunk fileName isEmptyOrNil ifFalse: > [|stream file| > stream := WriteStream on: String new. > chunk saveToStream: stream. > file := WAFile new > fileName: chunk fileName; > contents: stream contents; > contentType: chunk contentType; > yourself. > aRequest postFields at: chunk fieldName put: > file] > ifTrue: [ > |stream| > stream := WriteStream on: String new. > chunk saveToStream: stream. > > aRequest postFields at: chunk fieldName > put:( (stream contents) > convertFromEncoding: #utf8)] > ]. > > 2. squeak to database > > herefor I assume that the client encoding between squeak and postgres > is UTF-8. I just encode the complete outgoing SQL string to utf-8. > > SqueakDatabaseAccessor>>basicExecuteSQLString: aString > | result rowCollection | > "self log: aString." > result := connection execute: (aString convertToEncoding: > #utf8). > "result := connection execute: aString." > result errorResponse notNil > ifTrue: > [self externalDatabaseErrorSignal signal: result > errorResponse > value]. > rowCollection := OrderedCollection new. > result rows do: [:ea | rowCollection add: ea data asArray]. > ^ReadStream on: rowCollection asArray > > 3. database to squeak > > here the same client encoding applies. To convert these you have > to register type converters. I did this for varchar and text fields. > > Add > #(1043 25) > do: [:each| converters at: each put: [:s | s > convertFromEncoding: > #utf8]]. > > somewhere in PGConnection class>>buildDefaultFieldConverters > > > 4. squeak to web > > is the easiest as it is solved by using WAKomEncoded39 instead > of WAKom. > > > This approach solves all four ends where encoding plays a role. It > means there happens a lot of conversion but the approach is clean > in a way that you use inside squeak its own encoding. You're able > to use special spanisch characters inside squeak and they are displayed > properly in squeak and on the web pages. > > Hope this helps, > > Norbert > > > > |
|
|
In reply to this post by Ramiro Diaz Trepat
No, you can either use a custom Kom or get WideStrings.
2007/5/14, Ramiro Diaz Trepat <[hidden email]>: > Wow... > So I assume that mostly everyone not in an English speaking country > with a Seaside application in production is using Squeak 3.8? > Is that correct? > > > > > On 5/14/07, Philippe Marschall <[hidden email]> wrote: > > 2007/5/14, Ramiro Diaz Trepat <[hidden email]>: > > > Indeed, probably another great thing it would be if I could configure > > > Seaside to directly send me utf8 strings from the input taken on a > > > form´s entry fields. > > > Is it possible? > > > > Yes, this is what WAKom is doing. Just remember that you also have to > > send utf8 strings to Seaside. Unfortunately WAKom does not work on > > Squeak 3.9. > > > > Cheers > > Philippe > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > > > > Hello Todd and Marcelo, > > > > I am using Squeak 3.9. > > > > I know about how to set the encoding on postgres. Indeed, utf8 is > > > > the encoding I need on the database. > > > > What I want to know is if there is any place on the Postgres, Glorp > > > > or MagritteGlorp packages where I could configure for the strings to > > > > be translated to Squeak´s default (I think it´s iso). > > > > If I have, for example a Person on Glorp, and I want to set a last > > > > name like mine Díaz (with an accented vowel), coming from a Magritte > > > > form for instance, on the setter I translate it to UTF8 (#isoToUtf8) > > > > before I assign it to my class i.var. And it gets stored on the > > > > database with no errors. Later on, when Glorp "hydrates" my instance, > > > > of course it brings a weird string like "DÃ≈az", instead of "Díaz". > > > > Of course I would love that conversion to happen automatically with > > > > a little configuration. > > > > Otherwise, what are you guys doing? > > > > > > > > > > > > > > > > On 5/13/07, marcelo Cortez <[hidden email]> wrote: > > > > > Ramiro > > > > > > > > > > I'm postgres user , i hope help you , postgres > > > > > encondig can be configurated at database creation, > > > > > postgres client can configurated using statements > > > > > SET CLIENT_ENCODING TO 'value'; > > > > > take a look at > > > > > http://www.postgresql.org/docs/8.2/static/multibyte.html > > > > > best regards > > > > > mdc > > > > > > > > > > > > > > > > > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > > > > > > > > > Well... sorry for building this thread by myself. > > > > > > The issue now is that conversion does not happen > > > > > > automatically. I can > > > > > > properly store a string on a UTF8 database, by > > > > > > previously converting > > > > > > it using #isoToUtf8. > > > > > > But when strings come back after a query (using > > > > > > Glorp), they are not > > > > > > converted to iso, and hence they show all screwed > > > > > > up. > > > > > > Is there any place in Glorp or the Postgres driver > > > > > > to configure the > > > > > > encoding of the underlying database for these > > > > > > conversions to happen > > > > > > automatically? > > > > > > Thanks ! > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > > > > > wrote: > > > > > > > Sorry, it does work. > > > > > > > I don´t know what happend. > > > > > > > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > > > > > <[hidden email]> wrote: > > > > > > > > I was unable to use databases with UTF8 > > > > > > encoding, only SQL_ASCII > > > > > > > > encoding worked for me. > > > > > > > > is it something trivial to configure that I did > > > > > > not see or is it a limitation? > > > > > > > > Thanks > > > > > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > > Preguntá. Respondé. Descubrí. > > > > > Todo lo que querías saber, y lo que ni imaginabas, > > > > > está en Yahoo! Respuestas (Beta). > > > > > ¡Probalo ya! > > > > > http://www.yahoo.com.ar/respuestas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
Sorry Philippe, I am not sure what you mean.
When you say "a custom Kom" I suppose you mean extensions like Norbert's. But then, when you say "or get WideStrings". What do you mean? from where? What happened between 3.8 and 3.9? Why does Kom not work with utf8 in 3.9? There are plenty of Spanish, German, Swedish and French speakers in this list, what do you guys use in production? I suppose that Norbert's solution of adding hooks to translate strings everytime fixes the problem, but it also introduces a major performance penalty, because every string that comes and goes has to get translated (everytime, unless you cache the translations you make), and I guess these translations are not that cheap. On 5/14/07, Philippe Marschall <[hidden email]> wrote: > No, you can either use a custom Kom or get WideStrings. > > 2007/5/14, Ramiro Diaz Trepat <[hidden email]>: > > Wow... > > So I assume that mostly everyone not in an English speaking country > > with a Seaside application in production is using Squeak 3.8? > > Is that correct? > > > > > > > > > > On 5/14/07, Philippe Marschall <[hidden email]> wrote: > > > 2007/5/14, Ramiro Diaz Trepat <[hidden email]>: > > > > Indeed, probably another great thing it would be if I could configure > > > > Seaside to directly send me utf8 strings from the input taken on a > > > > form´s entry fields. > > > > Is it possible? > > > > > > Yes, this is what WAKom is doing. Just remember that you also have to > > > send utf8 strings to Seaside. Unfortunately WAKom does not work on > > > Squeak 3.9. > > > > > > Cheers > > > Philippe > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > > > > > Hello Todd and Marcelo, > > > > > I am using Squeak 3.9. > > > > > I know about how to set the encoding on postgres. Indeed, utf8 is > > > > > the encoding I need on the database. > > > > > What I want to know is if there is any place on the Postgres, Glorp > > > > > or MagritteGlorp packages where I could configure for the strings to > > > > > be translated to Squeak´s default (I think it´s iso). > > > > > If I have, for example a Person on Glorp, and I want to set a last > > > > > name like mine Díaz (with an accented vowel), coming from a Magritte > > > > > form for instance, on the setter I translate it to UTF8 (#isoToUtf8) > > > > > before I assign it to my class i.var. And it gets stored on the > > > > > database with no errors. Later on, when Glorp "hydrates" my instance, > > > > > of course it brings a weird string like "DÃ≈az", instead of "Díaz". > > > > > Of course I would love that conversion to happen automatically with > > > > > a little configuration. > > > > > Otherwise, what are you guys doing? > > > > > > > > > > > > > > > > > > > > On 5/13/07, marcelo Cortez <[hidden email]> wrote: > > > > > > Ramiro > > > > > > > > > > > > I'm postgres user , i hope help you , postgres > > > > > > encondig can be configurated at database creation, > > > > > > postgres client can configurated using statements > > > > > > SET CLIENT_ENCODING TO 'value'; > > > > > > take a look at > > > > > > http://www.postgresql.org/docs/8.2/static/multibyte.html > > > > > > best regards > > > > > > mdc > > > > > > > > > > > > > > > > > > > > > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > > > > > > > > > > > Well... sorry for building this thread by myself. > > > > > > > The issue now is that conversion does not happen > > > > > > > automatically. I can > > > > > > > properly store a string on a UTF8 database, by > > > > > > > previously converting > > > > > > > it using #isoToUtf8. > > > > > > > But when strings come back after a query (using > > > > > > > Glorp), they are not > > > > > > > converted to iso, and hence they show all screwed > > > > > > > up. > > > > > > > Is there any place in Glorp or the Postgres driver > > > > > > > to configure the > > > > > > > encoding of the underlying database for these > > > > > > > conversions to happen > > > > > > > automatically? > > > > > > > Thanks ! > > > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > > > > > > wrote: > > > > > > > > Sorry, it does work. > > > > > > > > I don´t know what happend. > > > > > > > > > > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > > > > > > <[hidden email]> wrote: > > > > > > > > > I was unable to use databases with UTF8 > > > > > > > encoding, only SQL_ASCII > > > > > > > > > encoding worked for me. > > > > > > > > > is it something trivial to configure that I did > > > > > > > not see or is it a limitation? > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > > > Preguntá. Respondé. Descubrí. > > > > > > Todo lo que querías saber, y lo que ni imaginabas, > > > > > > está en Yahoo! Respuestas (Beta). > > > > > > ¡Probalo ya! > > > > > > http://www.yahoo.com.ar/respuestas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
As far as Glorp goes, DatabasePlatform has an attribute
"characterEncoding", which will be used for certain types of
conversions. But I'm not sure what the interaction with the Squeak
postgresql drivers would be there. Setting that would probably work OK as
long as they're not using binding.
--
Alan Knight [|], Cincom Smalltalk Development
"The Static Typing Philosophy: Make it fast. Make it right.
Make it run." - Niall Ross
|
|
|
In reply to this post by Ramiro Diaz Trepat
On Mon, 2007-05-14 at 11:43 -0300, Ramiro Diaz Trepat wrote:
> Sorry Philippe, I am not sure what you mean. > > When you say "a custom Kom" I suppose you mean extensions like > Norbert's. But then, when you say "or get WideStrings". What do you > mean? from where? > > What happened between 3.8 and 3.9? > Why does Kom not work with utf8 in 3.9? > There are plenty of Spanish, German, Swedish and French speakers in > this list, what do you guys use in production? > I suppose that Norbert's solution of adding hooks to translate strings > everytime fixes the problem, but it also introduces a major > performance penalty, because every string that comes and goes has to > get translated (everytime, unless you cache the translations you > make), and I guess these translations are not that cheap. > aren't huge so conversion doesn't cost much. And it is the only way to do it. Every component should be free to choose an encoding it needs or it is configured to use. Take Java as an example. The internal storage is UCS-2. It is very close to UTF-8 ;) but not the same. So there are conversions everytime you need the string in a different encoding. Having a big registration for converted strings will cost more than its benefit if you get large lookup tables. My problem is to do assumptions about encoding. I assume to get UTF-8 from the web (as long as I'm sending UTF-8 this is feasible but not safe). I also assume the client encoding to the database is UTF-8 which in my case is always true but it can differ. As long as there is no negotiation about encoding everything is a hack and will fail sooner or later. And having encoding problems is the worst computer problems have to offer :) Norbert |
|
|
In reply to this post by Ramiro Diaz Trepat
On Mon, 2007-05-14 at 12:32 -0400, Alan Knight wrote:
> As far as Glorp goes, DatabasePlatform has an attribute > "characterEncoding", which will be used for certain types of > conversions. But I'm not sure what the interaction with the Squeak > postgresql drivers would be there. Setting that would probably work OK > as long as they're not using binding. > Oh, I need to look at that. The postgresql driver in squeak has a mapping of database type oids to type converters. My hack was to add type converters for varchar and text columns to convert from utf8. Norbert |
|
|
In reply to this post by Ramiro Diaz Trepat
2007/5/14, Ramiro Diaz Trepat <[hidden email]>:
> Sorry Philippe, I am not sure what you mean. > > When you say "a custom Kom" I suppose you mean extensions like > Norbert's. A special version of the KomHttpServer > But then, when you say "or get WideStrings". What do you > mean? from where? In input fields and the strings you render. WAKom: no encoding conversion, a Character is in fact only a byte, that may or may not correspont to a character depending on your encoding, for utf8 that's not the case WAKomEncoded39: utf8 <-> new Squeak encoding conversion, a Character is more or less a unicode character, this is "better" than Java/C# because it's also true for character outside the BMP. > What happened between 3.8 and 3.9? Semantics of #unescapePercents changed. > Why does Kom not work with utf8 in 3.9? Maintainer is too busy to change two methods. Cheers Philippe > There are plenty of Spanish, German, Swedish and French speakers in > this list, what do you guys use in production? > I suppose that Norbert's solution of adding hooks to translate strings > everytime fixes the problem, but it also introduces a major > performance penalty, because every string that comes and goes has to > get translated (everytime, unless you cache the translations you > make), and I guess these translations are not that cheap. > > > > On 5/14/07, Philippe Marschall <[hidden email]> wrote: > > No, you can either use a custom Kom or get WideStrings. > > > > 2007/5/14, Ramiro Diaz Trepat <[hidden email]>: > > > Wow... > > > So I assume that mostly everyone not in an English speaking country > > > with a Seaside application in production is using Squeak 3.8? > > > Is that correct? > > > > > > > > > > > > > > > On 5/14/07, Philippe Marschall <[hidden email]> wrote: > > > > 2007/5/14, Ramiro Diaz Trepat <[hidden email]>: > > > > > Indeed, probably another great thing it would be if I could configure > > > > > Seaside to directly send me utf8 strings from the input taken on a > > > > > form´s entry fields. > > > > > Is it possible? > > > > > > > > Yes, this is what WAKom is doing. Just remember that you also have to > > > > send utf8 strings to Seaside. Unfortunately WAKom does not work on > > > > Squeak 3.9. > > > > > > > > Cheers > > > > Philippe > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> wrote: > > > > > > Hello Todd and Marcelo, > > > > > > I am using Squeak 3.9. > > > > > > I know about how to set the encoding on postgres. Indeed, utf8 is > > > > > > the encoding I need on the database. > > > > > > What I want to know is if there is any place on the Postgres, Glorp > > > > > > or MagritteGlorp packages where I could configure for the strings to > > > > > > be translated to Squeak´s default (I think it´s iso). > > > > > > If I have, for example a Person on Glorp, and I want to set a last > > > > > > name like mine Díaz (with an accented vowel), coming from a Magritte > > > > > > form for instance, on the setter I translate it to UTF8 (#isoToUtf8) > > > > > > before I assign it to my class i.var. And it gets stored on the > > > > > > database with no errors. Later on, when Glorp "hydrates" my instance, > > > > > > of course it brings a weird string like "DÃ≈az", instead of "Díaz". > > > > > > Of course I would love that conversion to happen automatically with > > > > > > a little configuration. > > > > > > Otherwise, what are you guys doing? > > > > > > > > > > > > > > > > > > > > > > > > On 5/13/07, marcelo Cortez <[hidden email]> wrote: > > > > > > > Ramiro > > > > > > > > > > > > > > I'm postgres user , i hope help you , postgres > > > > > > > encondig can be configurated at database creation, > > > > > > > postgres client can configurated using statements > > > > > > > SET CLIENT_ENCODING TO 'value'; > > > > > > > take a look at > > > > > > > http://www.postgresql.org/docs/8.2/static/multibyte.html > > > > > > > best regards > > > > > > > mdc > > > > > > > > > > > > > > > > > > > > > > > > > > > > --- Ramiro Diaz Trepat <[hidden email]> escribió: > > > > > > > > > > > > > > > Well... sorry for building this thread by myself. > > > > > > > > The issue now is that conversion does not happen > > > > > > > > automatically. I can > > > > > > > > properly store a string on a UTF8 database, by > > > > > > > > previously converting > > > > > > > > it using #isoToUtf8. > > > > > > > > But when strings come back after a query (using > > > > > > > > Glorp), they are not > > > > > > > > converted to iso, and hence they show all screwed > > > > > > > > up. > > > > > > > > Is there any place in Glorp or the Postgres driver > > > > > > > > to configure the > > > > > > > > encoding of the underlying database for these > > > > > > > > conversions to happen > > > > > > > > automatically? > > > > > > > > Thanks ! > > > > > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat <[hidden email]> > > > > > > > > wrote: > > > > > > > > > Sorry, it does work. > > > > > > > > > I don´t know what happend. > > > > > > > > > > > > > > > > > > > > > > > > > > > On 5/13/07, Ramiro Diaz Trepat > > > > > > > > <[hidden email]> wrote: > > > > > > > > > > I was unable to use databases with UTF8 > > > > > > > > encoding, only SQL_ASCII > > > > > > > > > > encoding worked for me. > > > > > > > > > > is it something trivial to configure that I did > > > > > > > > not see or is it a limitation? > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > > > > Preguntá. Respondé. Descubrí. > > > > > > > Todo lo que querías saber, y lo que ni imaginabas, > > > > > > > está en Yahoo! Respuestas (Beta). > > > > > > > ¡Probalo ya! > > > > > > > http://www.yahoo.com.ar/respuestas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
In reply to this post by Ramiro Diaz Trepat
On Mon, 2007-05-14 at 12:32 -0400, Alan Knight wrote:
> As far as Glorp goes, DatabasePlatform has an attribute > "characterEncoding", which will be used for certain types of > conversions. But I'm not sure what the interaction with the Squeak > postgresql drivers would be there. Setting that would probably work OK > as long as they're not using binding. > I can't find the attribute in the squeak port. Is this newer code which hasn't been ported yet? Norbert |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |