Sockets, WebClient, HttpClient and friends
Locked
Sockets, WebClient, HttpClient and friends
 
|
Looking at the code I had a minor issue with the other day when loading from SqueakMap, I see what looks like a good opportunity to cleanup and simplify some classes. Since I'm most definitely not an expert in sockets and web interfaces etc I think some input from those of you more knowledgeable would be very helpful.
SqueakMap uses an HttpSocket directly to load; this has a problem in that the only error detection is to check the response for being a String. Which isn't the most robust or helpful technique. RBInternalSpellCheck does much the same, as do a few other users of #httpGet: (as an example). InstallerInternetBased uses an http socket and checks the response for responding to #reset (I imagine as a proxy for stream-like behaviour?) which doesn't seem long-term robust. Worst of all quite a few users of #httpGet: don't do any sort of checking and so will just dump users into a debugger; not our best look. I know that WebClient is a somewhat heavier class but it does seem to do a lot more helpful things like handling redirects and so on as well as some better error testing. Would it be sensible to convert as much as practical to use WebClient instead of a plain HttpSocket? Might it be better to improve HttpSocket? Some related stuff - there is an HttpClient class as well, unconnected to WebClient but used in several important looking places. Can that be improved or merged? At the very least it would be good to be able to document and comment code to explain why so many classes exist. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Strange OpCodes: TOAC: Turn Off Air Conditioner |
Re: Sockets, WebClient, HttpClient and friends
|
|
Hi Tim
> On 18.01.2019, at 02:29, tim Rowledge <[hidden email]> wrote: > > Looking at the code I had a minor issue with the other day when loading from SqueakMap, I see what looks like a good opportunity to cleanup and simplify some classes. Since I'm most definitely not an expert in sockets and web interfaces etc I think some input from those of you more knowledgeable would be very helpful. > > SqueakMap uses an HttpSocket directly to load; this has a problem in that the only error detection is to check the response for being a String. Which isn't the most robust or helpful technique. RBInternalSpellCheck does much the same, as do a few other users of #httpGet: (as an example). > > InstallerInternetBased uses an http socket and checks the response for responding to #reset (I imagine as a proxy for stream-like behaviour?) which doesn't seem long-term robust. > > Worst of all quite a few users of #httpGet: don't do any sort of checking and so will just dump users into a debugger; not our best look. > > I know that WebClient is a somewhat heavier class but it does seem to do a lot more helpful things like handling redirects and so on as well as some better error testing. Would it be sensible to convert as much as practical to use WebClient instead of a plain HttpSocket? Might it be better to improve HttpSocket? > > Some related stuff - there is an HttpClient class as well, unconnected to WebClient but used in several important looking places. Can that be improved or merged? At the very least it would be good to be able to document and comment code to explain why so many classes exist. > To puff the fog away a bit: Essentially everything uses WebClient. SqueakMap uses WebClient: See SMClient>>client and SMClient class>>assureWebClient. (there's a direct use of Socket and HTTPSocket in SMSqueakMap class>>pingServer: and #loadFullFrom: but onto that later) InstallerInternetBased also relies on HTTPSocket. Same for any user of HTTPClient. BUT: HTTPSocket now just "redirects" to WebClient. See: HTTPSocket class>>getDocument:args:accept:request: "…" "Some raw extra headers which historically have been added" xhdrs := HTTPProxyCredentials, HTTPBlabEmail, "may be empty" requestString. "extra user request. Authorization" client := WebClient new. ^[resp := client httpGet: urlString do:[:req| "Add ACCEPT header, accept plain text by default." req headerAt: 'Accept' put: (mimeType ifNil: ['text/html']). "Add the additional headers" (WebUtils readHeadersFrom: xhdrs readStream) do:[:assoc| req addHeader: assoc key value: assoc value]]. "…" TL;DR: It's just WebClient with differing layers of icing on top. But, yes, it could all be a bit more clear cut. Best regards -Tobias > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Strange OpCodes: TOAC: Turn Off Air Conditioner > > > |
Re: Sockets, WebClient, HttpClient and friends
|
|
> On 2019-01-18, at 1:30 AM, Tobias Pape <[hidden email]> wrote: > > TL;DR: It's just WebClient with differing layers of icing on top. Hah; so it is. That's good, I guess. Evidently diving a dozen or so layers down wasn't enough to discover that. ;-) > > But, yes, it could all be a bit more clear cut. It certainly could; but more importantly it could do much better for raising and handling errors helpfully. I note with wry amusement how the last part of HTTPSocket class>>#httpGetDocument:args:accept:request: carefully takes the helpful error info of WebClient and ensures that the less helpful HttpSocket response is generated. And of course none of this alters the fact that a good few methods could do with changing to handle network errors, whichever flavour of WebClient icing they use! tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Useful random insult:- Proof that evolution CAN go in reverse. |
Re: Sockets, WebClient, HttpClient and friends
|
|
Hello
This thread is related to my question how to get a picture from the web. I understand that I probably have to use WebClient. How would one of the convenience methods of WebClient be used to get a picture from the web? --Hannes On 1/18/19, tim Rowledge <[hidden email]> wrote: > > >> On 2019-01-18, at 1:30 AM, Tobias Pape <[hidden email]> wrote: >> >> TL;DR: It's just WebClient with differing layers of icing on top. > > Hah; so it is. That's good, I guess. Evidently diving a dozen or so layers > down wasn't enough to discover that. ;-) > >> >> But, yes, it could all be a bit more clear cut. > > It certainly could; but more importantly it could do much better for raising > and handling errors helpfully. I note with wry amusement how the last part > of HTTPSocket class>>#httpGetDocument:args:accept:request: carefully takes > the helpful error info of WebClient and ensures that the less helpful > HttpSocket response is generated. > > And of course none of this alters the fact that a good few methods could do > with changing to handle network errors, whichever flavour of WebClient icing > they use! > > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Useful random insult:- Proof that evolution CAN go in reverse. > > > > |
Re: Sockets, WebClient, HttpClient and friends
|
|
> On 21.03.2019, at 13:14, H. Hirzel <[hidden email]> wrote: > > Hello > > This thread is related to my question how to get a picture from the web. > > I understand that I probably have to use WebClient. > > How would one of the convenience methods of WebClient be used to get a > picture from the web? maybe: ImageReadWriter formFromStream: ((WebClient httpGet: 'the-url') contents readStream) -t > > --Hannes > > On 1/18/19, tim Rowledge <[hidden email]> wrote: >> >> >>> On 2019-01-18, at 1:30 AM, Tobias Pape <[hidden email]> wrote: >>> >>> TL;DR: It's just WebClient with differing layers of icing on top. >> >> Hah; so it is. That's good, I guess. Evidently diving a dozen or so layers >> down wasn't enough to discover that. ;-) >> >>> >>> But, yes, it could all be a bit more clear cut. >> >> It certainly could; but more importantly it could do much better for raising >> and handling errors helpfully. I note with wry amusement how the last part >> of HTTPSocket class>>#httpGetDocument:args:accept:request: carefully takes >> the helpful error info of WebClient and ensures that the less helpful >> HttpSocket response is generated. >> >> And of course none of this alters the fact that a good few methods could do >> with changing to handle network errors, whichever flavour of WebClient icing >> they use! >> >> >> tim >> -- >> tim Rowledge; [hidden email]; http://www.rowledge.org/tim >> Useful random insult:- Proof that evolution CAN go in reverse. >> >> >> >> > |
Re: Sockets, WebClient, HttpClient and friends
|
|
The following (slighlty adapted) comes up with an error
'image format not recognized' I put it in another process because today I crashed the image trying out variants of this (infinitive loop somewhere, could not analyze why). | theUrl | [ theUrl := 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Leopard_%2837072888293%29.jpg/640px-Leopard_%2837072888293%29.jpg'. (ImageReadWriter formFromStream: ((WebClient httpGet: theUrl) content readStream)) inspect] fork. HH. On 3/21/19, Tobias Pape <[hidden email]> wrote: > >> On 21.03.2019, at 13:14, H. Hirzel <[hidden email]> wrote: >> >> Hello >> >> This thread is related to my question how to get a picture from the web. >> >> I understand that I probably have to use WebClient. >> >> How would one of the convenience methods of WebClient be used to get a >> picture from the web? > > maybe: > > ImageReadWriter formFromStream: ((WebClient httpGet: 'the-url') contents > readStream) > > -t >> >> --Hannes >> >> On 1/18/19, tim Rowledge <[hidden email]> wrote: >>> >>> >>>> On 2019-01-18, at 1:30 AM, Tobias Pape <[hidden email]> wrote: >>>> >>>> TL;DR: It's just WebClient with differing layers of icing on top. >>> >>> Hah; so it is. That's good, I guess. Evidently diving a dozen or so >>> layers >>> down wasn't enough to discover that. ;-) >>> >>>> >>>> But, yes, it could all be a bit more clear cut. >>> >>> It certainly could; but more importantly it could do much better for >>> raising >>> and handling errors helpfully. I note with wry amusement how the last >>> part >>> of HTTPSocket class>>#httpGetDocument:args:accept:request: carefully >>> takes >>> the helpful error info of WebClient and ensures that the less helpful >>> HttpSocket response is generated. >>> >>> And of course none of this alters the fact that a good few methods could >>> do >>> with changing to handle network errors, whichever flavour of WebClient >>> icing >>> they use! >>> >>> >>> tim >>> -- >>> tim Rowledge; [hidden email]; http://www.rowledge.org/tim >>> Useful random insult:- Proof that evolution CAN go in reverse. >>> >>> >>> >>> >> > > > > |
Re: Sockets, WebClient, HttpClient and friends
|
|
I get a form object if I convert the ByteString object I obtain from
the webResponse object by sending #content to a ByteArray object #asByteArray | theUrl webResponse | [ theUrl := 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Leopard_%2837072888293%29.jpg/640px-Leopard_%2837072888293%29.jpg'. webResponse := WebClient httpGet: theUrl. (ImageReadWriter formFromStream: (webResponse content asByteArray readStream)) inspect] fork. On 3/21/19, H. Hirzel <[hidden email]> wrote: > The following (slighlty adapted) comes up with an error > > 'image format not recognized' > > I put it in another process because today I crashed the image trying > out variants of this (infinitive loop somewhere, could not analyze > why). > > | theUrl | > [ theUrl := > 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Leopard_%2837072888293%29.jpg/640px-Leopard_%2837072888293%29.jpg'. > > (ImageReadWriter formFromStream: ((WebClient httpGet: theUrl) content > readStream)) inspect] fork. > > HH. > > On 3/21/19, Tobias Pape <[hidden email]> wrote: >> >>> On 21.03.2019, at 13:14, H. Hirzel <[hidden email]> wrote: >>> >>> Hello >>> >>> This thread is related to my question how to get a picture from the web. >>> >>> I understand that I probably have to use WebClient. >>> >>> How would one of the convenience methods of WebClient be used to get a >>> picture from the web? >> >> maybe: >> >> ImageReadWriter formFromStream: ((WebClient httpGet: 'the-url') contents >> readStream) >> >> -t >>> >>> --Hannes >>> >>> On 1/18/19, tim Rowledge <[hidden email]> wrote: >>>> >>>> >>>>> On 2019-01-18, at 1:30 AM, Tobias Pape <[hidden email]> wrote: >>>>> >>>>> TL;DR: It's just WebClient with differing layers of icing on top. >>>> >>>> Hah; so it is. That's good, I guess. Evidently diving a dozen or so >>>> layers >>>> down wasn't enough to discover that. ;-) >>>> >>>>> >>>>> But, yes, it could all be a bit more clear cut. >>>> >>>> It certainly could; but more importantly it could do much better for >>>> raising >>>> and handling errors helpfully. I note with wry amusement how the last >>>> part >>>> of HTTPSocket class>>#httpGetDocument:args:accept:request: carefully >>>> takes >>>> the helpful error info of WebClient and ensures that the less helpful >>>> HttpSocket response is generated. >>>> >>>> And of course none of this alters the fact that a good few methods >>>> could >>>> do >>>> with changing to handle network errors, whichever flavour of WebClient >>>> icing >>>> they use! >>>> >>>> >>>> tim >>>> -- >>>> tim Rowledge; [hidden email]; http://www.rowledge.org/tim >>>> Useful random insult:- Proof that evolution CAN go in reverse. >>>> >>>> >>>> >>>> >>> >> >> >> >> > |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |