Looking for advice on SstByteMessage and UTF-8

Hi Joachim,

I think the problem lies with the fact that SstByteMessage>contents is returning an #SstOSPtr and not a string as your overwritten code does.

contents

^SstOSPtr on: contents size: self contentsSize offset: self headerSize

Maybe you can return an #SstOSPtr that points to converted data?

Here is a completely untested wild ass guess:

 

contents

| utf8 |

utf8 := super contents asString convertFromCodePage: 'utf-8'.

^SstOSPtr on: utf8 size: utf8 size.

Lou

On Friday, September 28, 2018 at 3:42:49 AM UTC-4, Joachim Tuchel wrote:

Dear VAST community,

I am dusting off some old RESTful web services code I once wrote because our Web Application needs a public API.

Most things work out of the box now, but I am having a little problem with UTF-8 (i hear some readers sigh).

I'd like to subclass SstByteMessage in a way so that I can access the contents of an incoming request automatically converted from UTF-8 to the local code page (#convertFromCodePage:) and of course I'D like the setting of content of a response on the server side to automatically convert that to UTF-8 when I set it.

So the simplest thing that could possibly work is to sumply subclass SstByteMessage with my own and have it do the conversion in the #contents and #contents: methods.

My naive implementation lloks like this:

contents

  ^super contents asString convertFromCodePage: 'utf-8'

contents: str

  super contents: (str convertToCodePage: 'utf-8').

In theory, this should work. But it doesn't. Interestingly, the #contents: setter seems to work nicely. I use Postman as a development client and there I get correct utf-8 Umlauts in responses.

The #contents getter, however, must be remoived to make things work. As long as the overridden #contents method is in the image, a GET or POST request is never answered. I also get no Exception (http forwarding is disabled). I wonder why that is?

Am I on the right track anyways? One of the reasons why chose this approach is that this way the calculation of the content's size for the response is correctly taking the extra bytes for Umlauts into account....

If I am on the right track, I am facing another problem:

For now I implemented #messageClass in my SstHttpServer subclass to return my own SstByteMessage subclass. IIUC, this will often not work, because there are some places where the actual messageClass to instantiate is not retrieved from the server, but from the Transport configuration... I am not sure how to register  an individual Transport Configuration, however. The only references in the image are loaded or post-load intializers and seem to be global. I'd like to only set this for one Applicatipon context....

So I would be grateful for pointers and ideas.

Joachim

Lou,

I'll try over the weekend. Thanks for the tip. Sure, SstOSPtr is a completely different beast than a String... How could I overlook that?

Any tips for the registration part of teh SstByteMessage subclass?

Joachim

Am Freitag, 28. September 2018 15:46:32 UTC+2 schrieb Louis LaBrunda:

Hi Joachim,

I think the problem lies with the fact that SstByteMessage>contents is returning an #SstOSPtr and not a string as your overwritten code does.

contents

^SstOSPtr on: contents size: self contentsSize offset: self headerSize

Maybe you can return an #SstOSPtr that points to converted data?

Here is a completely untested wild ass guess:

 

contents

| utf8 |

utf8 := super contents asString convertFromCodePage: 'utf-8'.

^SstOSPtr on: utf8 size: utf8 size.

Lou

On Friday, September 28, 2018 at 3:42:49 AM UTC-4, Joachim Tuchel wrote:

Dear VAST community,

I am dusting off some old RESTful web services code I once wrote because our Web Application needs a public API.

Most things work out of the box now, but I am having a little problem with UTF-8 (i hear some readers sigh).

I'd like to subclass SstByteMessage in a way so that I can access the contents of an incoming request automatically converted from UTF-8 to the local code page (#convertFromCodePage:) and of course I'D like the setting of content of a response on the server side to automatically convert that to UTF-8 when I set it.

So the simplest thing that could possibly work is to sumply subclass SstByteMessage with my own and have it do the conversion in the #contents and #contents: methods.

My naive implementation lloks like this:

contents

  ^super contents asString convertFromCodePage: 'utf-8'

contents: str

  super contents: (str convertToCodePage: 'utf-8').

In theory, this should work. But it doesn't. Interestingly, the #contents: setter seems to work nicely. I use Postman as a development client and there I get correct utf-8 Umlauts in responses.

The #contents getter, however, must be remoived to make things work. As long as the overridden #contents method is in the image, a GET or POST request is never answered. I also get no Exception (http forwarding is disabled). I wonder why that is?

Am I on the right track anyways? One of the reasons why chose this approach is that this way the calculation of the content's size for the response is correctly taking the extra bytes for Umlauts into account....

If I am on the right track, I am facing another problem:

For now I implemented #messageClass in my SstHttpServer subclass to return my own SstByteMessage subclass. IIUC, this will often not work, because there are some places where the actual messageClass to instantiate is not retrieved from the server, but from the Transport configuration... I am not sure how to register  an individual Transport Configuration, however. The only references in the image are loaded or post-load intializers and seem to be global. I'd like to only set this for one Applicatipon context....

So I would be grateful for pointers and ideas.

Joachim

HI Joachim,

SstTransportConfiguration accepts the #messageClass: message, so, if you can find the right spot to set it to your subclass of SstByteMessage, you won't need a subclass of SstTransportConfiguration.  Unfortunately, I don't any idea where that spot might be.

Lou


On Saturday, September 29, 2018 at 1:49:47 AM UTC-4, Joachim Tuchel wrote:

Lou,

I'll try over the weekend. Thanks for the tip. Sure, SstOSPtr is a completely different beast than a String... How could I overlook that?

Any tips for the registration part of teh SstByteMessage subclass?

Joachim

Am Freitag, 28. September 2018 15:46:32 UTC+2 schrieb Louis LaBrunda:

Hi Joachim,

I think the problem lies with the fact that SstByteMessage>contents is returning an #SstOSPtr and not a string as your overwritten code does.

contents

^SstOSPtr on: contents size: self contentsSize offset: self headerSize

Maybe you can return an #SstOSPtr that points to converted data?

Here is a completely untested wild ass guess:

 

contents

| utf8 |

utf8 := super contents asString convertFromCodePage: 'utf-8'.

^SstOSPtr on: utf8 size: utf8 size.

Lou

On Friday, September 28, 2018 at 3:42:49 AM UTC-4, Joachim Tuchel wrote:

Dear VAST community,

I am dusting off some old RESTful web services code I once wrote because our Web Application needs a public API.

Most things work out of the box now, but I am having a little problem with UTF-8 (i hear some readers sigh).

I'd like to subclass SstByteMessage in a way so that I can access the contents of an incoming request automatically converted from UTF-8 to the local code page (#convertFromCodePage:) and of course I'D like the setting of content of a response on the server side to automatically convert that to UTF-8 when I set it.

So the simplest thing that could possibly work is to sumply subclass SstByteMessage with my own and have it do the conversion in the #contents and #contents: methods.

My naive implementation lloks like this:

contents

  ^super contents asString convertFromCodePage: 'utf-8'

contents: str

  super contents: (str convertToCodePage: 'utf-8').

In theory, this should work. But it doesn't. Interestingly, the #contents: setter seems to work nicely. I use Postman as a development client and there I get correct utf-8 Umlauts in responses.

The #contents getter, however, must be remoived to make things work. As long as the overridden #contents method is in the image, a GET or POST request is never answered. I also get no Exception (http forwarding is disabled). I wonder why that is?

Am I on the right track anyways? One of the reasons why chose this approach is that this way the calculation of the content's size for the response is correctly taking the extra bytes for Umlauts into account....

If I am on the right track, I am facing another problem:

For now I implemented #messageClass in my SstHttpServer subclass to return my own SstByteMessage subclass. IIUC, this will often not work, because there are some places where the actual messageClass to instantiate is not retrieved from the server, but from the Transport configuration... I am not sure how to register  an individual Transport Configuration, however. The only references in the image are loaded or post-load intializers and seem to be global. I'd like to only set this for one Applicatipon context....

So I would be grateful for pointers and ideas.

Joachim

Hi Joachim,

I think the problem lies with the fact that SstByteMessage>contents is returning an #SstOSPtr and not a string as your overwritten code does.

contents

^SstOSPtr on: contents size: self contentsSize offset: self headerSize

Maybe you can return an #SstOSPtr that points to converted data?

Here is a completely untested wild ass guess:

 

contents

| utf8 |

utf8 := super contents asString convertFromCodePage: 'utf-8'.

^SstOSPtr on: utf8 size: utf8 size.

Lou

On Friday, September 28, 2018 at 3:42:49 AM UTC-4, Joachim Tuchel wrote:

Dear VAST community,

I am dusting off some old RESTful web services code I once wrote because our Web Application needs a public API.

Most things work out of the box now, but I am having a little problem with UTF-8 (i hear some readers sigh).

I'd like to subclass SstByteMessage in a way so that I can access the contents of an incoming request automatically converted from UTF-8 to the local code page (#convertFromCodePage:) and of course I'D like the setting of content of a response on the server side to automatically convert that to UTF-8 when I set it.

So the simplest thing that could possibly work is to sumply subclass SstByteMessage with my own and have it do the conversion in the #contents and #contents: methods.

My naive implementation lloks like this:

contents

  ^super contents asString convertFromCodePage: 'utf-8'

contents: str

  super contents: (str convertToCodePage: 'utf-8').

In theory, this should work. But it doesn't. Interestingly, the #contents: setter seems to work nicely. I use Postman as a development client and there I get correct utf-8 Umlauts in responses.

The #contents getter, however, must be remoived to make things work. As long as the overridden #contents method is in the image, a GET or POST request is never answered. I also get no Exception (http forwarding is disabled). I wonder why that is?

Am I on the right track anyways? One of the reasons why chose this approach is that this way the calculation of the content's size for the response is correctly taking the extra bytes for Umlauts into account....

If I am on the right track, I am facing another problem:

For now I implemented #messageClass in my SstHttpServer subclass to return my own SstByteMessage subclass. IIUC, this will often not work, because there are some places where the actual messageClass to instantiate is not retrieved from the server, but from the Transport configuration... I am not sure how to register  an individual Transport Configuration, however. The only references in the image are loaded or post-load intializers and seem to be global. I'd like to only set this for one Applicatipon context....

So I would be grateful for pointers and ideas.

Joachim