Smalltalk › Pharo › Pharo Smalltalk Users

ZnClient problem with extensions

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

kilon.alios

ZnClient problem with extensions

I am on Yosemite, latest 5 image, lastest VM and using this code

ZnClient new url: 'https://github.com/kilon/ChronosManager/archive/master.zip'; setIfModifiedSince: (Date year: 2015 month: 12 day: 1); downloadTo: FileLocator imageDirectory.

it downloads the file but does not append the extension ".zip" as it should.

Unfortunately setIfModifiedSince: does not seem to be supported by Github :(

Sven Van Caekenberghe-2

Re: ZnClient problem with extensions

Hi Dimitris,

Thanks for the feedback.

> On 04 Dec 2015, at 08:01, Dimitris Chloupis <[hidden email]> wrote:
>
> I am on Yosemite, latest 5 image, lastest VM and using this code
>
> ZnClient new url: 'https://github.com/kilon/ChronosManager/archive/master.zip'; setIfModifiedSince: (Date year: 2015 month: 12 day: 1); downloadTo: FileLocator imageDirectory.
>
> it downloads the file but does not append the extension ".zip" as it should.

Hmm, that is tricky. If you inspect ZnLogEvent announcer during the execution you will notice that a redirect is involved. Your original request for https://github.com/kilon/ChronosManager/archive/master.zip gets redirected to a request for https://codeload.github.com/kilon/ChronosManager/zip/master for which the name is effectively 'master' without the extension. Maybe the original name should be used, I have to think a bit about that. It seems that curl does it like that.

In any case, you can also specify a file to #downloadTo: as a workaround. Try:

ZnClient new url: 'https://github.com/kilon/ChronosManager/archive/master.zip'; setIfModifiedSince: (Date year: 2015 month: 12 day: 1); downloadTo: FileLocator imageDirectory / 'master.zip'; yourself.

> Unfortunately setIfModifiedSince: does not seem to be supported by Github :(

Yeah, not much you can do about that.

Sven

Skip Lentz-2

Re: ZnClient problem with extensions

In reply to this post by kilon.alios

On Dec 4, 2015, at 8:01 AM, Dimitris Chloupis <[hidden email]> wrote:

I am on Yosemite, latest 5 image, lastest VM and using this code

ZnClient new url: 'https://github.com/kilon/ChronosManager/archive/master.zip'; setIfModifiedSince: (Date year: 2015 month: 12 day: 1); downloadTo: FileLocator imageDirectory.

it downloads the file but does not append the extension ".zip" as it should.

Unfortunately setIfModifiedSince: does not seem to be supported by Github :(

The API supports conditional requests (If-Modified-Since and If-None-Match), so perhaps you should use the API endpoints:

https://developer.github.com/v3/#conditional-requests

I am planning to implement caching and conditional requests in my API bindings:

https://github.com/Balletie/GitHub

Skip

kilon.alios

Re: ZnClient problem with extensions

In reply to this post by Sven Van Caekenberghe-2

"Thanks for the feedback."

Thanks for this very useful library :)

"Hmm, that is tricky. If you inspect ZnLogEvent announcer during the execution you will notice that a redirect is involved. Your original request for https://github.com/kilon/ChronosManager/archive/master.zip gets redirected to a request for https://codeload.github.com/kilon/ChronosManager/zip/master for which the name is effectively 'master' without the extension. Maybe the original name should be used, I have to think a bit about that. It seems that curl does it like that."

naughty Github

but I am curious is it not normal for a dowload link to redirect to something else anyway ? I have seen such redirections frequently with browser.

"ZnClient new url: 'https://github.com/kilon/ChronosManager/archive/master.zip'; setIfModifiedSince: (Date year: 2015 month: 12 day: 1); downloadTo: FileLocator imageDirectory / 'master.zip'; yourself."

yes that though crossed my mind too.

How I tell ZnClient to overwrite an existing file , because by default it complains and pops up a dialog.

"Yeah, not much you can do about that."

Well I could have gone with the Github api in the first place , as Skip notedm but I am clueless when it comes to web dev.

"

The API supports conditional requests (If-Modified-Since and If-None-Match), so perhaps you should use the API endpoints:

https://developer.github.com/v3/#conditional-requests"

because I am clueless with web dev can you help me understand how to do this with ZnClient ?

Is this what people are referring to as a REST API ?

I am planning to implement caching and conditional requests in my API bindings:

https://github.com/Balletie/GitHub

your tool is great but i am trying to create as few dependencies for my project as possible or else it will take forever to load in a pharo image but I am definitely interesting into learning from your code . Thanks :)

Sven Van Caekenberghe-2

Re: ZnClient problem with extensions

I decided to add the behaviour "use filename from original request, which could make a difference when redirects are followed" to #downloadTo: while #downloadEntityTo: remains unchanged (not that it could change because the original request is gone by then).

In Zn #bleedingEdge:

===
Name: Zinc-HTTP-SvenVanCaekenberghe.443
Author: SvenVanCaekenberghe
Time: 4 December 2015, 3:26:34.596347 pm
UUID: 967114b0-e87f-4c5e-a511-d31efc27acae
Ancestors: Zinc-HTTP-SvenVanCaekenberghe.442

Change ZnClient>>#downloadTo: to use the original request's URI for the basename to download to (this could make a difference when redirects are followed)
Add comments to make this difference between #downloadTo: and #downloadEntityTo: clear
===

As for the overwriting of an existing file, if you load

===
Name: Zinc-FileSystem-SvenVanCaekenberghe.12
Author: SvenVanCaekenberghe
Time: 4 December 2015, 3:23:41.83904 pm
UUID: 66adef8d-edfb-49e1-aa33-f464ae2ca28f
Ancestors: Zinc-FileSystem-SvenVanCaekenberghe.11

Use proper resumable exceptions, FileExits and FileDoes NotExit, in ZnFileSystemUtils
===

You can now do as follows:

[ ZnClient new
url: 'https://github.com/kilon/ChronosManager/archive/master.zip';
downloadTo: FileLocator imageDirectory; yourself ]
on: FileExists do: [ :exception | exception resume ]

HTH,

Sven

> On 04 Dec 2015, at 11:56, Dimitris Chloupis <[hidden email]> wrote:
>
> "Thanks for the feedback."
>
> Thanks for this very useful library :)
>
> "Hmm, that is tricky. If you inspect ZnLogEvent announcer during the execution you will notice that a redirect is involved. Your original request for https://github.com/kilon/ChronosManager/archive/master.zip gets redirected to a request for https://codeload.github.com/kilon/ChronosManager/zip/master for which the name is effectively 'master' without the extension. Maybe the original name should be used, I have to think a bit about that. It seems that curl does it like that."
>
> naughty Github
>
> but I am curious is it not normal for a dowload link to redirect to something else anyway ? I have seen such redirections frequently with browser.
>
> "ZnClient new url: 'https://github.com/kilon/ChronosManager/archive/master.zip'; setIfModifiedSince: (Date year: 2015 month: 12 day: 1); downloadTo: FileLocator imageDirectory / 'master.zip'; yourself."
>
> yes that though crossed my mind too.
>
> How I tell ZnClient to overwrite an existing file , because by default it complains and pops up a dialog.
>
> "Yeah, not much you can do about that."
>
> Well I could have gone with the Github api in the first place , as Skip notedm but I am clueless when it comes to web dev.
>
> "
> The API supports conditional requests (If-Modified-Since and If-None-Match), so perhaps you should use the API endpoints:
>
> https://developer.github.com/v3/#conditional-requests"
>
> because I am clueless with web dev can you help me understand how to do this with ZnClient ?
>
> Is this what people are referring to as a REST API ?
>
>
> "
> I am planning to implement caching and conditional requests in my API bindings:
>
> https://github.com/Balletie/GitHub
> "
>
> your tool is great but i am trying to create as few dependencies for my project as possible or else it will take forever to load in a pharo image but I am definitely interesting into learning from your code . Thanks :)

Skip Lentz-2

Re: ZnClient problem with extensions

In reply to this post by kilon.alios

Hi,

On Dec 4, 2015, at 11:56 AM, Dimitris Chloupis <[hidden email]> wrote:

because I am clueless with web dev can you help me understand how to do this with ZnClient ?

I tried some things, such as this snippet:

ZnClient new

setIfModifiedSince: DateAndTime now;

contentReader: [ :entity | ZipArchive new readFrom: entity readStream ];

get: 'https://api.github.com/repos/kilon/ChronosManager/zipball/master'

But the response does not have a Last-Modified date.

So what you probably want is the date of the latest commit to master (or whatever branch). Considering you do want to minimize dependencies and therefore not use the API bindings, the following could be done:

client := ZnClient new.

" Get latest commit list, we only care about the response in this case "

client

setIfModifiedSince: (Date year: 2015 month: 12 day: 1);

get: 'https://api.github.com/repos/kilon/ChronosManager/commits?sha=master'.

client response isNotModified

ifTrue: [ nil ]

ifFalse: [

client

contentReader: [ :entity | ZipArchive new readFrom: entity readStream ];

get: 'https://api.github.com/repos/kilon/ChronosManager/zipball/master']

So you need in total three GET requests, since the second request responds with a redirect. The first request does not count to the API rate limit if it returns a 304 Not Modified.

kilon.alios

Re: ZnClient problem with extensions

Thank you both I will give both a try and be back with more questions :)

On Fri, Dec 4, 2015 at 6:57 PM Skip Lentz <[hidden email]> wrote:

Hi,

On Dec 4, 2015, at 11:56 AM, Dimitris Chloupis <[hidden email]> wrote:

because I am clueless with web dev can you help me understand how to do this with ZnClient ?

I tried some things, such as this snippet:

ZnClient new
setIfModifiedSince: DateAndTime now;
contentReader: [ :entity | ZipArchive new readFrom: entity readStream ];
get: 'https://api.github.com/repos/kilon/ChronosManager/zipball/master'

But the response does not have a Last-Modified date.

So what you probably want is the date of the latest commit to master (or whatever branch). Considering you do want to minimize dependencies and therefore not use the API bindings, the following could be done:

client := ZnClient new.
" Get latest commit list, we only care about the response in this case "
client
setIfModifiedSince: (Date year: 2015 month: 12 day: 1);
get: 'https://api.github.com/repos/kilon/ChronosManager/commits?sha=master'.
client response isNotModified
ifTrue: [ nil ]
ifFalse: [
client
contentReader: [ :entity | ZipArchive new readFrom: entity readStream ];
get: 'https://api.github.com/repos/kilon/ChronosManager/zipball/master']

So you need in total three GET requests, since the second request responds with a redirect. The first request does not count to the API rate limit if it returns a 304 Not Modified.

kilon.alios

Re: ZnClient problem with extensions

ok with your help I found the code to do this with every new release. I take the info from my README

list := OrderedCollection new. client := ZnClient new. readme := client get: 'https://raw.githubusercontent.com/kilon/ChronosManager/master/README.md'. readme regex: 'v.\..' matchesDo: [ :each | list add: each ]. list inspect.

list at:1 gives the top / latest version. So all will have to do is just add the new version info in the README which I would have done anyway.

Its not using the GITHUB API but it get the job done :)

Actually now that I am thinking about it the readme can contain all sort of information that my code can use and all in user readable format :) Beats Json and XML/Html syntax :)

On Fri, Dec 4, 2015 at 11:04 PM Dimitris Chloupis <[hidden email]> wrote:

Thank you both I will give both a try and be back with more questions :)

On Fri, Dec 4, 2015 at 6:57 PM Skip Lentz <[hidden email]> wrote:
Hi,

On Dec 4, 2015, at 11:56 AM, Dimitris Chloupis <[hidden email]> wrote:

because I am clueless with web dev can you help me understand how to do this with ZnClient ?

I tried some things, such as this snippet:

ZnClient new
setIfModifiedSince: DateAndTime now;
contentReader: [ :entity | ZipArchive new readFrom: entity readStream ];
get: 'https://api.github.com/repos/kilon/ChronosManager/zipball/master'

But the response does not have a Last-Modified date.

So what you probably want is the date of the latest commit to master (or whatever branch). Considering you do want to minimize dependencies and therefore not use the API bindings, the following could be done:

client := ZnClient new.
" Get latest commit list, we only care about the response in this case "
client
setIfModifiedSince: (Date year: 2015 month: 12 day: 1);
get: 'https://api.github.com/repos/kilon/ChronosManager/commits?sha=master'.
client response isNotModified
ifTrue: [ nil ]
ifFalse: [
client
contentReader: [ :entity | ZipArchive new readFrom: entity readStream ];
get: 'https://api.github.com/repos/kilon/ChronosManager/zipball/master']

So you need in total three GET requests, since the second request responds with a redirect. The first request does not count to the API rate limit if it returns a 304 Not Modified.