Squot and Github

>
> Hi Eric:
>
> On Sat, Jul 11, 2020 at 2:07 AM Eric Gade <[hidden email]> wrote:
> >
> > Hi all,
> >
> > I have a project I've been working on using Squot and the new Git tools and pushing to a repository I have on Github. The repository is private but works with my username and password. I have been successfully pushing commits to it for a couple of weeks from the same image. This includes commits that I made yesterday. But today, suddenly, I am no longer able to push to my remote. Instead, I get the error in the attached image.
> >
>
> Sorry to hear you're having issues pushing to remote from your image.
> I have no idea what's going on, maybe renew your access token and try
> again? What OS/VM/image are you using?
>
> > Note that I have not changed any Github settings or anything like that. So I'm not sure what has changed or what's going on. Any ideas?  Is anyone else having this issue? If I wish to find a way to "back out" of using Squot, is there some way to convert everything to Monticello (filetree) and just push it manually from my terminal?
> >
>
> No need to back out of using Squot just yet: when you added your
> project, the GitBrowser asked you for a local directory. Although
> Squot does not touch the working copy on the filesystem (the files
> that end up on GitHub), it does keep the repository metadata
> (everything in .git) in sync. So that means you can just push from
> your terminal from your filesystem repository. This use case is one of
> many reasons why Squot asks for a local directory.
>
> Hope this helps,
> Fabio
>
> > --
> > Eric
> >
>

>
> Hello Fabio and Jakob, thanks for responding.
>
>> Sorry to hear you're having issues pushing to remote from your image.
>> I have no idea what's going on, maybe renew your access token and try
>> again? What OS/VM/image are you using?
>
>
> I'm running macOS 10.15.1. Here's my about info:
> Squeak5.3
> latest update: #19448
> Current Change Set: HomeProject
> Image format 68021 (64 bit)
>
> Virtual Machine
> ---------------
> /Applications/Squeak5.3-19431-64bit.app/Contents/MacOS/Squeak
> Open Smalltalk Cog[Spur] VM [CoInterpreterPrimitives VMMaker.oscog-nice.2715] 64 bit
> Mac OS X built on Mar  3 2020 08:28:30 GMT Compiler: 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)
> platform sources revision VM: 202003021730 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Mon Mar 2 18:30:55 2020 CommitHash: 6a0bc96 Plugins: 202003021730 https://github.com/OpenSmalltalk/opensmalltalk-vm.git
> CoInterpreter VMMaker.oscog-nice.2715 uuid: 78e2f556-9829-42fe-963d-e19dfc43c0e9 Mar  3 2020
> StackToRegisterMappingCogit VMMaker.oscog-eem.2719 uuid: e40f3e94-3a54-411b-9613-5d19114ea131 Mar  3 2020
>
>> Sorry for the issue. Can you please press the Debug button in the
>> dialog and tell me some more about the `response` variable you should
>> see in the top context? For example what is the `response code` (401
>> or 403) or the `response content`?
>
>
> The response code is `401 Authorization Required`.
>
>> Could you please inspect whether the `response request` actually
>> contains authentication headers? I have seen in the past that
>> sometimes the WebClient does not send them when it should.
>
>
> It looks to me like the WebClient's response request (the original request sent I assume) headers *don't* have any Auth information: ` 'an OrderedCollection(''Content-Type''->''application/x-git-receive-pack-request'' ''Content-Length''->5343 ''User-Agent''->''git/2.0.5'' ''Host''->''github.com'')'`
>
>> If you restart the context in the debugger (should be either
>> GitSmartHTTPProtocol>>#discoverReferencesForService: or
>> GitSmartHTTPProtocol>>#invokeReceivePackWithRefs:andPack:deleting:) to
>> repeat the request, does the same error occur again or does it work on
>> second try?
>
>
> Ok, so this seems to have worked! Restarting #invokeReceivePackWithRefs:andPack:deleting: and proceeding seems to have done the trick! I have no idea why, of course. Any hints there? Either way, thanks for the tip. I've checked github and my latest changes are pushed.
>
>>
>> when you added your
>> project, the GitBrowser asked you for a local directory. Although
>> Squot does not touch the working copy on the filesystem (the files
>> that end up on GitHub), it does keep the repository metadata
>> (everything in .git) in sync. So that means you can just push from
>> your terminal from your filesystem repository.
>
>
> I was hoping a local file copy was there (as it has been when I manually pushed Monticello stuff in the past), but when I went snooping I didn't see anything. Also one thing I've noticed that might be of interest here is that when I first cloned my existing repo, and it asked for a folder to clone into, it seems to have put the .git file directly in that folder. This happens to be a folder that I keep all of my working images in, so that's not a good place. But the reason I did it that way is because I expected the normal git behavior: when you clone a repository on the command line in git, it *creates a new directory with the repo name* and puts the .git and other files inside of that directory. So when I selected ~/my_images/ or whatever as the clone directory with Squot, my git experience led me to believe I would find something at ~/my_images/<repo-name>/.git but instead it's just at ~/my_images/.git
>
> One last semi offtopic thing: is the Squeak community aware that Github will be deprecating password based authentication come November?
>
> Anyway the debugging/solutions you've all provided here seem to have done the trick for now  -- thanks!
>
>
>
> On Sat, Jul 11, 2020 at 4:02 AM Jakob Reschke <[hidden email]> wrote:
>>
>> Hi Eric,
>>
>> Sorry for the issue. Can you please press the Debug button in the
>> dialog and tell me some more about the `response` variable you should
>> see in the top context? For example what is the `response code` (401
>> or 403) or the `response content`? I would ask for the whole object,
>> but it also contains the request and that should contain your
>> credentials in the headers, which I don't want to see obviously. :-)
>> Could you please inspect whether the `response request` actually
>> contains authentication headers? I have seen in the past that
>> sometimes the WebClient does not send them when it should.
>>
>> If you restart the context in the debugger (should be either
>> GitSmartHTTPProtocol>>#discoverReferencesForService: or
>> GitSmartHTTPProtocol>>#invokeReceivePackWithRefs:andPack:deleting:) to
>> repeat the request, does the same error occur again or does it work on
>> second try? If the WebClient shows further problems, you can also try
>> to restart the topmost SquitBrowser context that you find on the
>> stack.
>>
>> About backing out: like Fabio said, the Git history is all on your
>> disk and you can still pull/push/... with the Git command line. To see
>> the files, you just have to checkout a commit (or reset --hard to see
>> the latest files on the branch). Then you will find everything in
>> filetree format (or Tonel if you configured it so). A converter exists
>> for Monticello->Squot (see
>> https://github.com/hpi-swa/Squot/#converting-monticello-history), but
>> not for the other way around, since Squot repositories are just
>> metadata-less filetree repositories. If someone really wants to build
>> a converter for the other direction, they could probably learn from
>> the existing converter and turn the read/write logic around. However,
>> I would rather like to find and fix your push issue.
>>
>> Kind regards,
>> Jakob
>>
>> Am Sa., 11. Juli 2020 um 08:43 Uhr schrieb Fabio Niephaus <[hidden email]>:
>> >
>> > Hi Eric:
>> >
>> > On Sat, Jul 11, 2020 at 2:07 AM Eric Gade <[hidden email]> wrote:
>> > >
>> > > Hi all,
>> > >
>> > > I have a project I've been working on using Squot and the new Git tools and pushing to a repository I have on Github. The repository is private but works with my username and password. I have been successfully pushing commits to it for a couple of weeks from the same image. This includes commits that I made yesterday. But today, suddenly, I am no longer able to push to my remote. Instead, I get the error in the attached image.
>> > >
>> >
>> > Sorry to hear you're having issues pushing to remote from your image.
>> > I have no idea what's going on, maybe renew your access token and try
>> > again? What OS/VM/image are you using?
>> >
>> > > Note that I have not changed any Github settings or anything like that. So I'm not sure what has changed or what's going on. Any ideas?  Is anyone else having this issue? If I wish to find a way to "back out" of using Squot, is there some way to convert everything to Monticello (filetree) and just push it manually from my terminal?
>> > >
>> >
>> > No need to back out of using Squot just yet: when you added your
>> > project, the GitBrowser asked you for a local directory. Although
>> > Squot does not touch the working copy on the filesystem (the files
>> > that end up on GitHub), it does keep the repository metadata
>> > (everything in .git) in sync. So that means you can just push from
>> > your terminal from your filesystem repository. This use case is one of
>> > many reasons why Squot asks for a local directory.
>> >
>> > Hope this helps,
>> > Fabio
>> >
>> > > --
>> > > Eric
>> > >
>> >
>>
>
>
> --
> Eric
>

> Hi all,
>
> Am Sa., 11. Juli 2020 um 13:53 Uhr schrieb Eric Gade <[hidden email]>:
>>
>>> Could you please inspect whether the `response request` actually
>>> contains authentication headers? I have seen in the past that
>>> sometimes the WebClient does not send them when it should.
>>
>>
>> It looks to me like the WebClient's response request (the original request sent I assume) headers *don't* have any Auth information: ` 'an OrderedCollection(''Content-Type''->''application/x-git-receive-pack-request'' ''Content-Length''->5343 ''User-Agent''->''git/2.0.5'' ''Host''->''github.com'')'`
>>
>
> To me these missing auth headers look like a WebClient bug because the
> GitSmartHTTPProtocol class (comes with the Git Browser) clearly
> supplies the credentials to the client (see
> discoverReferencesForService:, requestCredentials, applyCredentials)
> and uses the same WebClient instance for the whole push operation,
> which entails two requests (discover references is the first, invoke
> receive pack is the second).
>
> If the misbehavior triggers, the second request is usually the one
> that is missing the auth headers. I have already seen this
> sporadically myself, but not lately. Repeating the request (by
> restarting the method) usually resolves the problem.
>
> Does anyone have a hint? It is possible that I have wrong expectations
> about the WebClient interface and how it remembers the credentials...

>
> On Sat, 11 Jul 2020, Jakob Reschke wrote:
>
> > Hi all,
> >
> > Am Sa., 11. Juli 2020 um 13:53 Uhr schrieb Eric Gade <[hidden email]>:
> >>
> >>> Could you please inspect whether the `response request` actually
> >>> contains authentication headers? I have seen in the past that
> >>> sometimes the WebClient does not send them when it should.
> >>
> >>
> >> It looks to me like the WebClient's response request (the original request sent I assume) headers *don't* have any Auth information: ` 'an OrderedCollection(''Content-Type''->''application/x-git-receive-pack-request'' ''Content-Length''->5343 ''User-Agent''->''git/2.0.5'' ''Host''->''github.com'')'`
> >>
> >
> > To me these missing auth headers look like a WebClient bug because the
> > GitSmartHTTPProtocol class (comes with the Git Browser) clearly
> > supplies the credentials to the client (see
> > discoverReferencesForService:, requestCredentials, applyCredentials)
> > and uses the same WebClient instance for the whole push operation,
> > which entails two requests (discover references is the first, invoke
> > receive pack is the second).
> >
> > If the misbehavior triggers, the second request is usually the one
> > that is missing the auth headers. I have already seen this
> > sporadically myself, but not lately. Repeating the request (by
> > restarting the method) usually resolves the problem.
> >
> > Does anyone have a hint? It is possible that I have wrong expectations
> > about the WebClient interface and how it remembers the credentials...
>
> Perhaps this part of WebClient >> #sendRequest:contentBlock: is
> responsible:
>
>         "Flush previous authState.
>         XXXX: Fixme. authState must be preserved for pre-authentication of requests."
>         self flushAuthState.
>
> #flushAuthState removes the passed credentials.
>
> Also, any kind of redirect will remove the credentials.
>
>
> Levente
>
> >
> > Kind regards,
> > Jakob
>