Hi all,
I'm calling for people who have got the nginx file upload module to work in their seaside apps... We are trying to use the Nginx file upload module but are stuck when the request needs to get passed on to seaside. We are using a configuration with a separate '/upload' location that gets handled by the file upload module. The request then needs to get passed to the upstream backend (seaside) but that request still has the URI '/upload' and we do not seem to get it rewritten to the '/seasideapp' URI. Is there anyone who has this working completely and could give us some advice on how solve this? The relevant parts of our nginx conf look like this: We also tried with a named location, but according to the docs, that makes the request arguments be dropped (and we need the _s & _k args for seaside...) Johan ---- location / { include fastcgi_params; fastcgi_pass seaside; } # Upload form should be submitted to this location location /upload { # Pass altered request body to this location upload_pass /backend; upload_pass_args on; #Pass all fields of the form upload_pass_form_field ".*"; # Store files to this directory upload_store /var/www/documents/; # Set specified fields in request body upload_set_form_field $upload_field_name.name "$upload_file_name"; upload_set_form_field $upload_field_name.content_type "$upload_content_type"; upload_set_form_field $upload_field_name.path "$upload_tmp_path"; upload_cleanup 400 404 499 500-505; } location /backend { rewrite ^/upload/(.*) /seasideapp/$1 last; }_______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Hi Johan,
Sorry for the delay. I have the Nginx file upload module working. In case it's still useful here's the relevant portions of my nginx configuration: upstream seaside {
server 127.0.0.1:9001; server 127.0.0.1:9002; server 127.0.0.1:9003;
fair; } server { listen 80 default; server_name www.getitmade.com; # server_name _;
root /var/www; location / { try_files $uri @seaside; } # ensure that files are searched for locally and not passed to the
# Gems if the file isn't found on the disk location ~* \.(gif|jpg|jpeg|png|css|html|htm|js|zip)$ { } location ~ fileupload {
# Pass altered request body to this location upload_pass @seaside; # if there's no upload file in this request, nginx generates a 405 - we use this
# to pass the request onto the 'normal' seaside processing # The way this is achieved in the other location directives is through a "try_files" method # however using "try_files" results in the other directives never being processed
error_page 405 415 = @seaside; # Store files to this directory # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
upload_store /var/nginx/temp 1; # Set the file attributes for uploaded files upload_store_access user:rw group:rw all:rw; # Set specified fields in request body
upload_set_form_field $upload_field_name.name "$upload_file_name"; upload_set_form_field $upload_field_name.content_type "$upload_content_type";
upload_set_form_field $upload_field_name.path "$upload_tmp_path"; # Inform backend about hash and size of a file # upload_aggregate_form_field "$upload_field_name.md5" "$upload_file_md5";
upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size"; # seaside automatically assigns sequential integers to fields with callbacks
# we want to pass those fields to the backend upload_pass_form_field "^\d+$"; # we don't want files hanging around if the server failed to process them.
upload_cleanup 500-505; # file upload progress tracking - 30s is the timeout (progress tracking is # available 30s after the upload has finished)
# this must be the last directive in the location block. track_uploads proxied 30s; } # used to report upload progress - defined by the Nginx Upload Progress Module
location /progress { report_uploads proxied; }
location @seaside { include /etc/nginx/fastcgi_params; fastcgi_intercept_errors on; fastcgi_pass seaside; } location /nginx_status {
stub_status on; access_log off; allow 127.0.0.1; deny all; } error_page 404 /errors/404.html;
error_page 403 /errors/403.html; error_page 500 502 503 504 /errors/50x.html; } On 12 June 2011 10:28, Johan Brichau <[hidden email]> wrote: Hi all, _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Hi Nick,
Thanks for this. I wonder: did you register a 'fileupload' handler in the seaside backend? We can now get it to work when we put the file upload module on the / location, but I wonder if that is a good solution. Your nginx config still catches only a location that matches 'fileupload'. On 15 Jun 2011, at 13:14, Nick Ager wrote: > Hi Johan, > > Sorry for the delay. I have the Nginx file upload module working. In case it's still useful here's the relevant portions of my nginx configuration: > > upstream seaside { > server 127.0.0.1:9001; > server 127.0.0.1:9002; > server 127.0.0.1:9003; > fair; > } > > server { > listen 80 default; > server_name www.getitmade.com; > # server_name _; > root /var/www; > > location / { > try_files $uri @seaside; > } > > # ensure that files are searched for locally and not passed to the > # Gems if the file isn't found on the disk > location ~* \.(gif|jpg|jpeg|png|css|html|htm|js|zip)$ { > > } > > location ~ fileupload { > # Pass altered request body to this location > upload_pass @seaside; > > # if there's no upload file in this request, nginx generates a 405 - we use this > # to pass the request onto the 'normal' seaside processing > # The way this is achieved in the other location directives is through a "try_files" method > # however using "try_files" results in the other directives never being processed > error_page 405 415 = @seaside; > > # Store files to this directory > # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist > upload_store /var/nginx/temp 1; > > # Set the file attributes for uploaded files > upload_store_access user:rw group:rw all:rw; > > # Set specified fields in request body > upload_set_form_field $upload_field_name.name "$upload_file_name"; > upload_set_form_field $upload_field_name.content_type "$upload_content_type"; > upload_set_form_field $upload_field_name.path "$upload_tmp_path"; > > # Inform backend about hash and size of a file > # upload_aggregate_form_field "$upload_field_name.md5" "$upload_file_md5"; > upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size"; > > # seaside automatically assigns sequential integers to fields with callbacks > # we want to pass those fields to the backend > upload_pass_form_field "^\d+$"; > > # we don't want files hanging around if the server failed to process them. > upload_cleanup 500-505; > > # file upload progress tracking - 30s is the timeout (progress tracking is > # available 30s after the upload has finished) > # this must be the last directive in the location block. > track_uploads proxied 30s; > } > > # used to report upload progress - defined by the Nginx Upload Progress Module > # see http://wiki.nginx.org/HttpUploadProgressModule > location /progress { > report_uploads proxied; > } > > location @seaside { > include /etc/nginx/fastcgi_params; > fastcgi_intercept_errors on; > fastcgi_pass seaside; > } > > location /nginx_status { > # copied from http://blog.kovyrin.net/2006/04/29/monitoring-nginx-with-rrdtool/ > stub_status on; > access_log off; > allow 127.0.0.1; > deny all; > } > > error_page 404 /errors/404.html; > error_page 403 /errors/403.html; > error_page 500 502 503 504 /errors/50x.html; > } > > > > > On 12 June 2011 10:28, Johan Brichau <[hidden email]> wrote: > Hi all, > > I'm calling for people who have got the nginx file upload module to work in their seaside apps... > > We are trying to use the Nginx file upload module but are stuck when the request needs to get passed on to seaside. > We are using a configuration with a separate '/upload' location that gets handled by the file upload module. The request then needs to get passed to the upstream backend (seaside) but that request still has the URI '/upload' and we do not seem to get it rewritten to the '/seasideapp' URI. > > > Is there anyone who has this working completely and could give us some advice on how solve this? > > The relevant parts of our nginx conf look like this: > We also tried with a named location, but according to the docs, that makes the request arguments be dropped (and we need the _s & _k args for seaside...) > > Johan > > ---- > > location / { > include fastcgi_params; > fastcgi_pass seaside; > } > > # Upload form should be submitted to this location > location /upload { > > # Pass altered request body to this location > upload_pass /backend; > upload_pass_args on; > > > #Pass all fields of the form > upload_pass_form_field ".*"; > > # Store files to this directory > upload_store /var/www/documents/; > > # Set specified fields in request body > upload_set_form_field $upload_field_name.name "$upload_file_name"; > upload_set_form_field $upload_field_name.content_type "$upload_content_type"; > upload_set_form_field $upload_field_name.path "$upload_tmp_path"; > > upload_cleanup 400 404 499 500-505; > } > > location /backend > { > rewrite ^/upload/(.*) /seasideapp/$1 last; > }_______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Hi Johan,
I wonder: did you register a 'fileupload' handler in the seaside backend? Exactly. I think my rationale was that I wanted a way for the backend to communicate with Nginx that it should treat file upload request differently so I append 'fileupload' to the url. I tried adding a url parameter (eg ?fileupload=1&_s=...) but couldn't find a way to trap that within Nginx configuration. Originally I used #updateUrl: to add the 'fileupload' to the url, now I use javascript as I also add an 'X-Progress-ID=' for the upload tracking.
I keep meaning to write this up, I'll try to find time this weekend. Hope this helps Nick _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
On 15 Jun 2011, at 22:53, Nick Ager wrote: > Exactly. I think my rationale was that I wanted a way for the backend to communicate with Nginx that it should treat file upload request differently so I append 'fileupload' to the url. I tried adding a url parameter (eg ?fileupload=1&_s=...) but couldn't find a way to trap that within Nginx configuration. Originally I used #updateUrl: to add the 'fileupload' to the url, now I use javascript as I also add an 'X-Progress-ID=' for the upload tracking. Okay, so it's all a bit trickier than I thought. I was trying to rewrite the request to '/upload' after nginx has processed the file, such that the normal callback invocation of the seaside application can do its work with the request (except that a hidden field is required in this case). Unfortunately, that does not seem to work and you thus need to handle the '/upload' request in the seaside backend as well. I'm still not entirely convinced that a rewrite of the request in nginx after the file processing should not work though ;-) I will give it another try. But I'm looking forward to reading your write-up too! thanks again Johan_______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Nick,
After writing my last response, I got the picture... Because Seaside ignores whatever comes in the path after the application name, the request will be processed by the seaside app anyway. This is what I failed to see. In our attempts, we _replaced_ the action url on the form by something like '/fileupload?_s=...&k=...'. Instead, you merely _append_ to the action url such that it has the form: '/<app>/fileupload?_s=...&k=...' That last url will be correctly handled as an incoming request for <app> by seaside, ignoring whatever comes after in the path. Great 'aha!' moment ;-) Thanks! Johan On 16 Jun 2011, at 08:57, Johan Brichau wrote: > > On 15 Jun 2011, at 22:53, Nick Ager wrote: > >> Exactly. I think my rationale was that I wanted a way for the backend to communicate with Nginx that it should treat file upload request differently so I append 'fileupload' to the url. I tried adding a url parameter (eg ?fileupload=1&_s=...) but couldn't find a way to trap that within Nginx configuration. Originally I used #updateUrl: to add the 'fileupload' to the url, now I use javascript as I also add an 'X-Progress-ID=' for the upload tracking. > > Okay, so it's all a bit trickier than I thought. > I was trying to rewrite the request to '/upload' after nginx has processed the file, such that the normal callback invocation of the seaside application can do its work with the request (except that a hidden field is required in this case). > Unfortunately, that does not seem to work and you thus need to handle the '/upload' request in the seaside backend as well. > > I'm still not entirely convinced that a rewrite of the request in nginx after the file processing should not work though ;-) I will give it another try. > > But I'm looking forward to reading your write-up too! > > thanks again > Johan_______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Hi Johan,
For what it's worth I started writing it up here: http://www.nickager.com/blog/file-upload-using-Nginx-and-Seaside - though it's little more than notes at the moment..
In the meantime here is a raw dump of some code. Hope it might help, though I'm afraid it's more complex than it needs to be as it includes upload tracking, Iframe submission, additional user feedback.
I set the target of the form to a hidden Iframe: form := html form. form id: #fileUploadForm; style: 'margin:0px'; attributeAt: 'target' put: 'hiddenImageUploader'; onSubmit: 'startUploadProgressBar(); return true;'; I manipulate the path using javascript: fileuploadSpecificJavascript ^ String streamContents: [:aStream | aStream nextPutAll: 'var xProgressId="'; nextPutAll: self xProgressId; nextPutAll: '"; function checkFileAndSubmitIfImage(actionUrl) { var filename = $("#fileUploadId").val(); var extensionIndex=filename.lastIndexOf(''.'') + 1; var extension=filename.substring(extensionIndex).toLowerCase(); if (extension && /^(jpg|png|jpeg|gif)$/.test(extension)) { var newUrl = "/fileupload" + actionUrl+"&X-Progress-ID=" + xProgressId; $("#fileUploadForm").attr("action", newUrl); $("#fileUploadForm").submit(); } else { $("#uploadNotification").append("<ul class=\"errors\"><li>''" + filename + "'' is not a valid image file (jpg, jpeg, gif, png), try another file</li></ul>"); } $("#fileUploadId").val(""); return false; } ' ] You trigger an update with the iframe as a target. The image is uploaded and a minimal response is returned to the iframe. That response triggers the onload. The onload then calls a method in the iframes parent - the main page. I've called it imageUploadedCallback() then performs the following:
1) clears the upload progress 2) renders any upload errors 3) renders thumb nails of the images uploaded 4) sets a new xProgressId for the next upload. The imageUploadedCallback() code looks like:
imageUploadedCallbackScriptOn: html ^ html jQuery ajax script: [ :s | s << (JSStream on: 'clearUploadProgress()').
s << (s jQuery: #uploadNotification) html: [ :renderer |
errorText notNil ifTrue: [ renderer div
class: 'error'; with: errorText ] ifFalse: [
renderer div] ]. s << (s jQuery: #projectUploadedFilesContainer) html: [ :renderer |
renderer render: self imageRenderer ]. s << (JSStream on: 'xProgressId="', self xProgressId, '"') ]
"The hiddenInput callback will contain the file upload fields if we've loading via nginx file upload module" html hiddenInput value: 'hidden'; callback: [:val | | request postFields fileName uploadFieldName | request := self requestContext request. postFields := request postFields. uploadFieldName := fileUploadField attributeAt: 'name'. fileName := (postFields at: (uploadFieldName, '.name') ifAbsent: [nil]). "has nginx file upload module inserted it's post fields in the request?" fileName notNil ifTrue: [ fileMoveCallback value: postFields value: uploadFieldName ] ] I use the OSes 'mv' to move the files: moveFrom: fromString to: toString | shellMoveFileCommand | shellMoveFileCommand := String streamContents: [:stream | stream nextPutAll: 'mv '; nextPutAll: fromString; nextPutAll: ' '''; nextPutAll: toString; nextPutAll: '''' ]. IZLogger logInfo: 'IZShellScripts>>#moveFrom:to:' with: shellMoveFileCommand. SpEnvironment runShellCommandString: shellMoveFileCommand. some more javascript: imageUploadedCallbackScriptOn: html ^ html jQuery ajax script: [ :s | s << (JSStream on: 'clearUploadProgress()'). s << (s jQuery: #uploadNotification) html: [ :renderer | errorText notNil ifTrue: [ renderer div class: 'error'; with: errorText ] ifFalse: [ renderer div] ]. s << (s jQuery: #projectUploadedFilesContainer) html: [ :renderer | renderer render: self imageRenderer ]. s << (JSStream on: 'xProgressId="', self xProgressId, '"') ] and one more javascript method: startUploadProgressBarJavascript ^' var interval = null; function startUploadProgressBar() { $("#progressBar").progressbar({ value: 0 }); $("#progressBar").css("display", "block"); $("#fileUploadContainer").css("display", "none"); interval = window.setInterval( function () { fetch(xProgressId); }, 1000); } function clearUploadProgress() { window.clearTimeout(interval); $("#progressBar").css("display", "none"); $("#fileUploadContainer").css("display", "block"); } function setUploadStatus (statusText) { $("#uploadNotification").html("<div class=''status''>" + statusText + "</div>"); } function fetch(uuid) { $.ajax({ url: "/progress", beforeSend: function (xhr) { xhr.setRequestHeader("X-Progress-ID", uuid); }, success: function (data, textStatus, xhr) { var upload = eval(data); if (upload.state == "done" || upload.state == "uploading") { var percentageComplete = Math.floor(upload.received * 100 / upload.size); $("#progressBar").progressbar({ value: percentageComplete }); if (upload.received != upload.size) { setUploadStatus("Uploading: " + percentageComplete + "%"); } else { setUploadStatus("Uploaded: processing"); } } if (upload.state == "starting") { setUploadStatus("Starting upload"); } if (upload.state == "done") { clearUploadProgress(); } }, error: function(xhr, textError) { setUploadStatus("Error: " + textError); } }); } ' _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
In reply to this post by Johan Brichau-2
Johan,
In our attempts, we _replaced_ the action url on the form by something like '/fileupload?_s=...&k=...'. Exactly though in my case I cut out the dispatcher and serve the requests directly from my application so I don't need to bother rewriting the incoming request. More info here: http://nickager.com/blog/serving-seaside-requests-without-the-application-name-in-the-URL/
HTH Nick _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Free forum by Nabble | Edit this page |