Smalltalk › Squeak › Squeak - Dev

Problems with Browse revisions in System package

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

Eliot Miranda-2

Problems with Browse revisions in System package

Hi Chris, Hi All,

I tried browse revisions for SmalltalkImage>>recreateSpecialObjectsArray and this was the reply:

HTTP/1.1 414 Request-URI Too Large

server: nginx/1.16.0

date: Sat, 29 Jun 2019 00:18:02 GMT

content-type: text/html

content-length: 177

connection: close

<html>

<head><title>414 Request-URI Too Large</title></head>

<body>

<center><h1>414 Request-URI Too Large</h1></center>

<hr><center>nginx/1.16.0</center>

</body>

</html>

Just an FYI. I can live without this but it is really nice to have.

_,,,^..^,,,_

best, Eliot

David T. Lewis

Re: Problems with Browse revisions in System package

On Fri, Jun 28, 2019 at 05:20:09PM -0700, Eliot Miranda wrote:

> Hi Chris, Hi All,
>
> I tried browse revisions for
> SmalltalkImage>>recreateSpecialObjectsArray and this was the reply:
>
> HTTP/1.1 414 Request-URI Too Large
> server: nginx/1.16.0
> date: Sat, 29 Jun 2019 00:18:02 GMT
> content-type: text/html
> content-length: 177
> connection: close
>
> <html>
> <head><title>414 Request-URI Too Large</title></head>
> <body>
> <center><h1>414 Request-URI Too Large</h1></center>
> <hr><center>nginx/1.16.0</center>
> </body>
> </html>
>
> Just an FYI. I can live without this but it is really nice to have.
> _,,,^..^,,,_
> best, Eliot

This might be a hard thing to fix. The issue is that the http request contains
a serialized version of the MCMethodDefinition for the method being checked.

In the case of SmalltalkImage>>recreateSpecialObjectsArray, the method contains
references to a lot of classes, and the serialized version of this is rather
huge (9324 bytes after serializing and encoding it for HTTP).

The result is an unreasonablly large HTTP get request, which nginx (I think) is
complaining about.

A workaround might be to tell nginx to accept unreasonably large requests (I
don't know how to do that, but I am sure it must be possible), or to find some
way to reformulate the request to use a simpler representation of the method
(possibly STON or perhaps just fileout format) rather than a full ReferenceStream
serialization.

Dave

Levente Uzonyi

Re: Problems with Browse revisions in System package

On Sat, 29 Jun 2019, David T. Lewis wrote:

> On Fri, Jun 28, 2019 at 05:20:09PM -0700, Eliot Miranda wrote:
>> Hi Chris, Hi All,
>>
>> I tried browse revisions for
>> SmalltalkImage>>recreateSpecialObjectsArray and this was the reply:
>>
>> HTTP/1.1 414 Request-URI Too Large
>> server: nginx/1.16.0
>> date: Sat, 29 Jun 2019 00:18:02 GMT
>> content-type: text/html
>> content-length: 177
>> connection: close
>>
>> <html>
>> <head><title>414 Request-URI Too Large</title></head>
>> <body>
>> <center><h1>414 Request-URI Too Large</h1></center>
>> <hr><center>nginx/1.16.0</center>
>> </body>
>> </html>
>>
>> Just an FYI. I can live without this but it is really nice to have.
>> _,,,^..^,,,_
>> best, Eliot
>
> This might be a hard thing to fix. The issue is that the http request contains
> a serialized version of the MCMethodDefinition for the method being checked.
>
> In the case of SmalltalkImage>>recreateSpecialObjectsArray, the method contains
> references to a lot of classes, and the serialized version of this is rather
> huge (9324 bytes after serializing and encoding it for HTTP).
>
> The result is an unreasonablly large HTTP get request, which nginx (I think) is
> complaining about.
>
> A workaround might be to tell nginx to accept unreasonably large requests (I
> don't know how to do that, but I am sure it must be possible), or to find some
> way to reformulate the request to use a simpler representation of the method
> (possibly STON or perhaps just fileout format) rather than a full ReferenceStream
> serialization.

It's possible to change nginx's 8kB default (this method produces a 12kB
url), but I'd rather change the server (squeaksource image) to accept POST
requests as well. Then the only change needed in the image would be to
replace httpGet: with httpPost:.

Levente

>
> Dave

Chris Muller-3

Re: Problems with Browse revisions in System package

Hi guys,

url), but I'd rather change the server (squeaksource image) to accept POST
requests as well. Then the only change needed in the image would be to
replace httpGet: with httpPost:.

Yes, me too.

Then the only change needed in the image would be to
replace httpGet: with httpPost:.

SqueakSource was designed since the beginning to treat every PUT request as a file upload (.mcm or .mcz). This is method which handles them, in SSSession:

putRequest: aProject
| author data version response blessed |
self expire.
(self isAllowed: SSAccessPolicy write in: aProject)
ifFalse: [ self authentificateProject: aProject ].
author := self user ifNil: [ SSMember anonymousMember ].
SSRepository current log: 'PUT ', self request url, ' (', author initials, ')'.
data := self request at: 'PUTData'.

(self request url endsWith: '.mcm')
ifTrue: [aProject addConfig: data author: author url: self request url]
ifFalse: [version := aProject addVersion: data author: author].

(version notNil and: [self request accessPath second = 'blessed'])
ifTrue: [blessed := true.
aProject
blessVersion: version
ifForbidden: [ blessed := false] ].
"Only filesystems actually need to save here, databases already saved, above, in addConfig:... | addVersion:"
self saveIf: [ : repository | repository isDatabase not ].
response := SSCreatedResponse new.
blessed ifNotNil: [
blessed
ifTrue: [ response nextPutAll: 'Version blessed' ]
ifFalse: [ response nextPutAll: 'Version NOT blessed' ] ].
self returnResponse: response

The challenge is not just reconfiguring the code, of course, but the protocol. The client is constrained to use of core image code only (e.g., not STON code) and the server needs to remain backward compatible with older client images. It looks like it checks its url to see if its a '.mcm' suffix, otherwise assumes its an .mcz. Perhaps it could do something else if it was neither of those...?

The code running our server loads cleanly and easily into a modern image:

Scanner allowUnderscoreAsAssignment: true.
Installer new merge: #squeaksource.

I'd love if someone with HTTP expertise would like to take a crack at it. While in there, don't forget Levente has another good suggestion to increase the number of semaphores for better performance, too.

- Chris

Levente

>
> Dave