The Inbox: WebClient-Core-mt.125.mcz

A new version of WebClient-Core was added to project The Inbox:
http://source.squeak.org/inbox/WebClient-Core-mt.125.mcz

==================== Summary ====================

Name: WebClient-Core-mt.125
Author: mt
Time: 1 October 2020, 3:39:01.65348 pm
UUID: 8a9c023e-5992-534e-ba7d-2a824197d979
Ancestors: WebClient-Core-mt.124

Re-commit proposal from WebClient-Core-monty.114 (-> treated) to use more recent codebase. Original commit message follows.

(30 October 2017, 2:59 am)

Added support for disabling the automatic character decoding of text responses by adding #getContentWithProgress:decodesText:, making #getContentWithProgress: delegate to it, adding #contentWithProgress:decodesText: as a caching front-end and then making #contentWithProgress: delegate to it and #content delegate to #contentWithProgress:.

This is needed because some text formats allow encodings to be inferred from the document itself, like from a leading BOM or an XML-style 'encoding="..."?>' declaration, and to give an alternative for situations where the charset is incorrectly inferred and WebClient mangles the response. This is different from HTTP encoding (like with GZIP) which is part of HTTP and is meant to be applied before transit and removed after entirely transparant to the user and so does not permanently alter the original payload, unlike automatic decoding from UTF-8 or another character encoding.

=============== Diff against WebClient-Core-mt.124 ===============

Item was changed:
  ----- Method: WebMessage>>getContentWithProgress: (in category 'private') -----
  getContentWithProgress: progressBlockOrNil
- "Reads available content and returns it."
 
+ ^ self getContentWithProgress: progressBlockOrNil decodesText: true!
- | length result |
- length := self contentLength.
- result := (stream isBinary ifTrue:[ ByteArray ] ifFalse: [ ByteString ])
- new: (length ifNil: [ 1000 ])
- streamContents: [ :outputStream |
- self
- streamFrom: stream
- to: outputStream
- size: length
- progress: progressBlockOrNil ].
- self decoderForContentEncoding ifNotNil: [:decoder |
- result := (decoder on: result) upToEnd].
- self textConverterForContentType ifNotNil: [:converter |
- [result := result convertFromWithConverter: converter]
- on: InvalidUTF8 "some servers lie"
- do: [^ result]].
- ^ result
- !

Item was added:
+ ----- Method: WebMessage>>getContentWithProgress:decodesText: (in category 'private') -----
+ getContentWithProgress: progressBlockOrNil decodesText: aBoolean
+ "Reads available content and returns it, periodically evaluating progressBlockOrNil
+ with the total size and the total bytes read so far, optionally decoding text content
+ based on the Content-Type charset."
+
+ | length result |
+ length := self contentLength.
+ result := (stream isBinary ifTrue:[ ByteArray ] ifFalse: [ ByteString ])
+ new: (length ifNil: [ 1000 ])
+ streamContents: [ :outputStream |
+ self
+ streamFrom: stream
+ to: outputStream
+ size: length
+ progress: progressBlockOrNil ].
+ self decoderForContentEncoding ifNotNil: [:decoder |
+ result := (decoder on: result) upToEnd].
+ aBoolean ifTrue: [
+ self textConverterForContentType ifNotNil: [:converter |
+ [result := result convertFromWithConverter: converter]
+ on: InvalidUTF8 "some servers lie"
+ do: [^ result]]].
+ ^ result!

Item was removed:
- ----- Method: WebResponse>>getContentWithProgress: (in category 'private') -----
- getContentWithProgress: progressBlockOrNil
- "Any response to a HEAD request and any response with a 1xx (Informational), 204 (No Content), or 304 (Not Modified) status code is always terminated by the first empty line after the header fields, regardless of the header fields present in the message, and thus cannot contain a message body. See https://tools.ietf.org/html/rfc7230#section-3.3.3 "
-
- [
- (request method = 'HEAD'
- or: [(code between: 100 and: 199)
- or: [code = 204
- or: [code = 304]]]) ifTrue: [^ ''].
-
- ^ super getContentWithProgress: progressBlockOrNil
-
- ] ensure: [self closeIfTransient]!

Item was added:
+ ----- Method: WebResponse>>getContentWithProgress:decodesText: (in category 'private') -----
+ getContentWithProgress: progressBlockOrNil decodesText: aBoolean
+ "Any response to a HEAD request and any response with a 1xx (Informational), 204 (No Content), or 304 (Not Modified) status code is always terminated by the first empty line after the header fields, regardless of the header fields present in the message, and thus cannot contain a message body. See https://tools.ietf.org/html/rfc7230#section-3.3.3 "
+
+ [
+ (request method = 'HEAD'
+ or: [(code between: 100 and: 199)
+ or: [code = 204
+ or: [code = 304]]]) ifTrue: [^ ''].
+
+ ^ super getContentWithProgress: progressBlockOrNil decodesText: aBoolean
+
+ ] ensure: [self closeIfTransient]!