SqueakMap server
Locked
 
|
>> ...
>> 3. I'm interested in hearing what you have to say about the design of a >> new SMServer, even if you're hysterical about. But I would appreciate it if >> you would allow me to respond in my own time. As you may know, I put a lot of time and thought and work into SqueakMap. The goal has always been for the community to have one single source for information about all or most Squeak-based software, to the extent that, anyone who wants published software to be noticed by Squeaker's will want it to be "in Squeak's app store" too. When you first said replace the server, I thought you meant replace the implementation and look-and-feel of the web-UI per your discretion. That would be great. Not only would the community end up with a new SM server running on 4.5/Cog, it'd also be a signature-example of a web-app to learn from that has browse, search, user-accounts and a callout API. Then when I saw your revamp of the requirements and that something about the "closet" and, remembering past events, I wondered while I was reading it whether all my work had already gone "poof". Had to check. Whew! Downloaded a backup of entire SqueakMap. We don't have to be friends to do good for the SqueakMap server (though it would be preferable, of course). I've had this on my own to-do list to replace because I want to learn web and it's a good example-app. I don't think it'll be a "quick-and-done" at all, because all the requirements serve a needed purpose and integrate into a overall process. If you're willing to retain "SqueakMap" as we know it today, with the domain model and all the critical functionality intact, I'd welcome that. |
|
|
On 14 April 2014 20:46, Chris Muller <[hidden email]> wrote:
>>> ... >>> 3. I'm interested in hearing what you have to say about the design of a >>> new SMServer, even if you're hysterical about. But I would appreciate it if >>> you would allow me to respond in my own time. > > As you may know, I put a lot of time and thought and work into > SqueakMap. The goal has always been for the community to have one > single source for information about all or most Squeak-based software, > to the extent that, anyone who wants published software to be noticed > by Squeaker's will want it to be "in Squeak's app store" too. > > When you first said replace the server, I thought you meant replace > the implementation and look-and-feel of the web-UI per your > discretion. That would be great. Not only would the community end up > with a new SM server running on 4.5/Cog, it'd also be a > signature-example of a web-app to learn from that has browse, search, > user-accounts and a callout API. > > Then when I saw your revamp of the requirements and that something > about the "closet" and, remembering past events, I wondered while I > was reading it whether all my work had already gone "poof". Had to > check. Whew! Downloaded a backup of entire SqueakMap. > > We don't have to be friends to do good for the SqueakMap server > (though it would be preferable, of course). I've had this on my own > to-do list to replace because I want to learn web and it's a good > example-app. I don't think it'll be a "quick-and-done" at all, > because all the requirements serve a needed purpose and integrate into > a overall process. If you're willing to retain "SqueakMap" as we know > it today, with the domain model and all the critical functionality > intact, I'd welcome that. I think it's safe to say that the _purpose_ of SqueakMap is well-understood, and hopefully everyone knows that it's (a) underutilised and (b) very, very important. (Datum: Pharo are re-inventing/have re-invented the concept.) But having said that, if someone came along and wrote a new server that provided the same functionality as SM, and provided some means of letting people migrate with a minimum of fuss between the old service and the new, I'm sure that noone would complain. In fact, the opposite: I think that several people would be very happy at seeing some revitalisation of SqueakMap. Especially if the client/server protocol was well-documented. frank |
|
|
"Migration" means we'd have two catalogs. One with migrated stuff,
the other with unmigrated stuff. So, when one goes looking for MorphicWrappers they would need to first check the "new-SqueakMap" to see if it had been migrated and, if not, go and look at the "old" SqueakMap. The whole purpose of a single, uniform, accepted Catalog is to avoid this kind of confusion. The entire legacy SM domain model is currently up to 722k (up from 700k more than 3 years ago). It's an elegant domain model that resides entirely in memory and doesn't need changed at all to replace the server implementation. Part of the model refers to a place in a directory structure on the SM server for the .st or .sar files. So there's nothing to "migrate". We just need to replace the server implementation and keep this same domain model and storage directory structure. Sure, it they can be "refined" on a whim but that's not necessary to replace the server. Some people want to have a new, empty SqueakMap so we can keep track of what's "new and shiny" apart from what's "old and dusty". That's understandable and SM already delineates new from old, what's been uploaded recently, what works for new versions of Squeak's and what for old versions. On Mon, Apr 14, 2014 at 3:22 PM, Frank Shearar <[hidden email]> wrote: > On 14 April 2014 20:46, Chris Muller <[hidden email]> wrote: >>>> ... >>>> 3. I'm interested in hearing what you have to say about the design of a >>>> new SMServer, even if you're hysterical about. But I would appreciate it if >>>> you would allow me to respond in my own time. >> >> As you may know, I put a lot of time and thought and work into >> SqueakMap. The goal has always been for the community to have one >> single source for information about all or most Squeak-based software, >> to the extent that, anyone who wants published software to be noticed >> by Squeaker's will want it to be "in Squeak's app store" too. >> >> When you first said replace the server, I thought you meant replace >> the implementation and look-and-feel of the web-UI per your >> discretion. That would be great. Not only would the community end up >> with a new SM server running on 4.5/Cog, it'd also be a >> signature-example of a web-app to learn from that has browse, search, >> user-accounts and a callout API. >> >> Then when I saw your revamp of the requirements and that something >> about the "closet" and, remembering past events, I wondered while I >> was reading it whether all my work had already gone "poof". Had to >> check. Whew! Downloaded a backup of entire SqueakMap. >> >> We don't have to be friends to do good for the SqueakMap server >> (though it would be preferable, of course). I've had this on my own >> to-do list to replace because I want to learn web and it's a good >> example-app. I don't think it'll be a "quick-and-done" at all, >> because all the requirements serve a needed purpose and integrate into >> a overall process. If you're willing to retain "SqueakMap" as we know >> it today, with the domain model and all the critical functionality >> intact, I'd welcome that. > > I think it's safe to say that the _purpose_ of SqueakMap is > well-understood, and hopefully everyone knows that it's (a) > underutilised and (b) very, very important. (Datum: Pharo are > re-inventing/have re-invented the concept.) But having said that, if > someone came along and wrote a new server that provided the same > functionality as SM, and provided some means of letting people migrate > with a minimum of fuss between the old service and the new, I'm sure > that noone would complain. In fact, the opposite: I think that several > people would be very happy at seeing some revitalisation of SqueakMap. > Especially if the client/server protocol was well-documented. > > frank > |
|
|
Hi guys!
Just FYI, I don't track all of squeak-dev (no time), but I can promise to read and reply (to questions) on all threads mentioning SqueakMap in the subject. Chris Muller knows SM in-and-out I would say and yes, the web UI is separated from the domain model. In fact, the *exact same* domain model is used in your image when you use SMLoader (or its cousins), SMLoader is just a "different UI" on top of it. That was indeed one of the *key design points* that unfortunately was never really understood by large parts of the community - my guess at least! The idea was always that we would maintain a map together, the domain model, and we could operate on it *in our own images*. This means that say, iterating over all packages and selecting a subset of them and installing them - that is trivially done in a few lines of code. Frank: There is "no" client-server protocol to speak of. The client just does a HTTP GET to check the transaction counter of the domain model at the SqueakMap server - if its higher than the local counter - we do another GET to fetch a gzipped ImageSegment of the domain model. ...anyway, if anyone has any questions, feel free to ask and I will do my best to answer. SIDENOTE: The web UI really sucks code wise. It was written using HttpView, my own web layer which actually inspired Avi in some ways when he did Seaside. Today I would personally just clone off the UI of SmalltalkHub (written all in Amber) or write a new one in Seaside or some other proper web layer. Both those approaches need some attention to "bookmarkable URLs though", SM is very restful which is IMHO very important to make things googlable. And oh, for any restful API I would use Seaside-REST, its really nice and I use it now in a project at 3DICC. regards, Göran |
|
|
In reply to this post by Chris Muller-3
On 14 April 2014 23:21, Chris Muller <[hidden email]> wrote:
> "Migration" means we'd have two catalogs. One with migrated stuff, > the other with unmigrated stuff. So, when one goes looking for > MorphicWrappers they would need to first check the "new-SqueakMap" to > see if it had been migrated and, if not, go and look at the "old" > SqueakMap. The whole purpose of a single, uniform, accepted Catalog > is to avoid this kind of confusion. That is not what migration means. Migration means replacing the existing thing with a new thing, with a short transition period so you know the new one's good, _and destroying the old one_. > The entire legacy SM domain model is currently up to 722k (up from > 700k more than 3 years ago). It's an elegant domain model that > resides entirely in memory and doesn't need changed at all to replace > the server implementation. Part of the model refers to a place in a > directory structure on the SM server for the .st or .sar files. So > there's nothing to "migrate". We just need to replace the server > implementation and keep this same domain model and storage directory > structure. Sure, it they can be "refined" on a whim but that's not > necessary to replace the server. I tried to add a feature (it's from two years ago, and I can't be bothered to trawl the list to find out the details - perhaps it was so you could publish to multiple Squeak versions from within the in-image client?) to SM that required understanding the client/server protocol. I gave up after a handful of hours because I couldn't reverse engineer the protocol. > Some people want to have a new, empty SqueakMap so we can keep track > of what's "new and shiny" apart from what's "old and dusty". That's > understandable and SM already delineates new from old, what's been > uploaded recently, what works for new versions of Squeak's and what > for old versions. Some people are clearly silly. SM tags everything, so if you want "new, shiny" you just use a new tag. frank > On Mon, Apr 14, 2014 at 3:22 PM, Frank Shearar <[hidden email]> wrote: >> On 14 April 2014 20:46, Chris Muller <[hidden email]> wrote: >>>>> ... >>>>> 3. I'm interested in hearing what you have to say about the design of a >>>>> new SMServer, even if you're hysterical about. But I would appreciate it if >>>>> you would allow me to respond in my own time. >>> >>> As you may know, I put a lot of time and thought and work into >>> SqueakMap. The goal has always been for the community to have one >>> single source for information about all or most Squeak-based software, >>> to the extent that, anyone who wants published software to be noticed >>> by Squeaker's will want it to be "in Squeak's app store" too. >>> >>> When you first said replace the server, I thought you meant replace >>> the implementation and look-and-feel of the web-UI per your >>> discretion. That would be great. Not only would the community end up >>> with a new SM server running on 4.5/Cog, it'd also be a >>> signature-example of a web-app to learn from that has browse, search, >>> user-accounts and a callout API. >>> >>> Then when I saw your revamp of the requirements and that something >>> about the "closet" and, remembering past events, I wondered while I >>> was reading it whether all my work had already gone "poof". Had to >>> check. Whew! Downloaded a backup of entire SqueakMap. >>> >>> We don't have to be friends to do good for the SqueakMap server >>> (though it would be preferable, of course). I've had this on my own >>> to-do list to replace because I want to learn web and it's a good >>> example-app. I don't think it'll be a "quick-and-done" at all, >>> because all the requirements serve a needed purpose and integrate into >>> a overall process. If you're willing to retain "SqueakMap" as we know >>> it today, with the domain model and all the critical functionality >>> intact, I'd welcome that. >> >> I think it's safe to say that the _purpose_ of SqueakMap is >> well-understood, and hopefully everyone knows that it's (a) >> underutilised and (b) very, very important. (Datum: Pharo are >> re-inventing/have re-invented the concept.) But having said that, if >> someone came along and wrote a new server that provided the same >> functionality as SM, and provided some means of letting people migrate >> with a minimum of fuss between the old service and the new, I'm sure >> that noone would complain. In fact, the opposite: I think that several >> people would be very happy at seeing some revitalisation of SqueakMap. >> Especially if the client/server protocol was well-documented. >> >> frank >> > |
|
|
In reply to this post by Göran Krampe
On 15 April 2014 08:34, Göran Krampe <[hidden email]> wrote:
> Hi guys! > > Just FYI, I don't track all of squeak-dev (no time), but I can promise to > read and reply (to questions) on all threads mentioning SqueakMap in the > subject. > > Chris Muller knows SM in-and-out I would say and yes, the web UI is > separated from the domain model. In fact, the *exact same* domain model is > used in your image when you use SMLoader (or its cousins), SMLoader is just > a "different UI" on top of it. > > That was indeed one of the *key design points* that unfortunately was never > really understood by large parts of the community - my guess at least! > > The idea was always that we would maintain a map together, the domain model, > and we could operate on it *in our own images*. This means that say, > iterating over all packages and selecting a subset of them and installing > them - that is trivially done in a few lines of code. > > Frank: There is "no" client-server protocol to speak of. The client just > does a HTTP GET to check the transaction counter of the domain model at the > SqueakMap server - if its higher than the local counter - we do another GET > to fetch a gzipped ImageSegment of the domain model. HTTP GET _is_ a client/server API. SM's protocol isn't documented, and I didn't have the time to figure out the http framework, which you mention below so I'll just continue there... > ...anyway, if anyone has any questions, feel free to ask and I will do my > best to answer. > > SIDENOTE: The web UI really sucks code wise. It was written using HttpView, > my own web layer which actually inspired Avi in some ways when he did > Seaside. Today I would personally just clone off the UI of SmalltalkHub > (written all in Amber) or write a new one in Seaside or some other proper > web layer. Both those approaches need some attention to "bookmarkable URLs > though", SM is very restful which is IMHO very important to make things > googlable. And oh, for any restful API I would use Seaside-REST, its really > nice and I use it now in a project at 3DICC. I guess maybe I should have been more precise in what I was complaining about. I don't know much about SM's domain model, and I don't care: that's an implementation detail of the service. What I care about is the service's API. So if it's possible to take SM's existing domain model, and SM's existing data, and just replace the HttpView stuff with some new API (that's actually documented!), then great! Less work for Chris Cunnington to do! The main point to my reply was that I'd like to see Chris Cunnington supported in his experiments around SM, and I'd like to see Chris Muller sleeping easily at night knowing that our core infrastructure isn't going to be ripped apart during said experiments. frank > regards, Göran |
|
|
Hi!
On 04/15/2014 09:47 AM, Frank Shearar wrote: > On 15 April 2014 08:34, Göran Krampe <[hidden email]> wrote: >> Frank: There is "no" client-server protocol to speak of. The client just >> does a HTTP GET to check the transaction counter of the domain model at the >> SqueakMap server - if its higher than the local counter - we do another GET >> to fetch a gzipped ImageSegment of the domain model. > > HTTP GET _is_ a client/server API. SM's protocol isn't documented, and > I didn't have the time to figure out the http framework, which you > mention below so I'll just continue there... Do note I wrote "no" using quotes, of course its a protocol - but its extremely simple. And I could have helped explain it and even document it but I don't recall anyone emailing me and asking about it :) > I guess maybe I should have been more precise in what I was > complaining about. I don't know much about SM's domain model, and I > don't care: that's an implementation detail of the service. What I > care about is the service's API. So if it's possible to take SM's > existing domain model, and SM's existing data, and just replace the > HttpView stuff with some new API (that's actually documented!), then > great! Less work for Chris Cunnington to do! Yes, that would be very possible IMHO. I actually thought that was what Chris Cunnington was doing (replacing the web UI but keeping the domain model). Also, "implementation detail" is... well, no, its a fair bit more than that I would say. Most web services just offer an API and keep the domain model on the server. SM doesn't - it offers a very simple API to actually download the whole model and then perform all readonly work on the client. And right, documentation is probably scarce, sorry about that, but you don't have to be snotty about it. And I think many of the classes have class comments and the code is probably fairly well commented too (granted it was a long time ago so I don't really remember). > The main point to my reply was that I'd like to see Chris Cunnington > supported in his experiments around SM, and I'd like to see Chris > Muller sleeping easily at night knowing that our core infrastructure > isn't going to be ripped apart during said experiments. Personally I applaud all efforts in any and all directions :) regards, Göran PS. Just for fun, given you have the domain model in *your* image the code snippet in this pdf is stuff you can easily do: http://swiki.krampe.se/gohu/uploads/11/squeakmap.pdf |
|
|
On 15 April 2014 09:13, Göran Krampe <[hidden email]> wrote:
> Hi! > > > On 04/15/2014 09:47 AM, Frank Shearar wrote: >> >> On 15 April 2014 08:34, Göran Krampe <[hidden email]> wrote: >>> >>> Frank: There is "no" client-server protocol to speak of. The client just >>> does a HTTP GET to check the transaction counter of the domain model at >>> the >>> SqueakMap server - if its higher than the local counter - we do another >>> GET >>> to fetch a gzipped ImageSegment of the domain model. >> >> >> HTTP GET _is_ a client/server API. SM's protocol isn't documented, and >> I didn't have the time to figure out the http framework, which you >> mention below so I'll just continue there... > > > Do note I wrote "no" using quotes, of course its a protocol - but its > extremely simple. And I could have helped explain it and even document it > but I don't recall anyone emailing me and asking about it :) > > >> I guess maybe I should have been more precise in what I was >> complaining about. I don't know much about SM's domain model, and I >> don't care: that's an implementation detail of the service. What I >> care about is the service's API. So if it's possible to take SM's >> existing domain model, and SM's existing data, and just replace the >> HttpView stuff with some new API (that's actually documented!), then >> great! Less work for Chris Cunnington to do! > > > Yes, that would be very possible IMHO. I actually thought that was what > Chris Cunnington was doing (replacing the web UI but keeping the domain > model). > > Also, "implementation detail" is... well, no, its a fair bit more than that > I would say. Most web services just offer an API and keep the domain model > on the server. SM doesn't - it offers a very simple API to actually download > the whole model and then perform all readonly work on the client. Ah, but that assumes that the downloaded model is isomorphic to the internal model. That's not necessarily the case. > And right, documentation is probably scarce, sorry about that, but you don't > have to be snotty about it. And I think many of the classes have class > comments and the code is probably fairly well commented too (granted it was > a long time ago so I don't really remember). My apologies: I didn't mean to sound snotty. Frustrated, yes, but not snotty. (The frustration came from having to understand an entire HTTP framework before I could start doing what I actually wanted to do.) Good quality documentation is very expensive to produce. >> The main point to my reply was that I'd like to see Chris Cunnington >> supported in his experiments around SM, and I'd like to see Chris >> Muller sleeping easily at night knowing that our core infrastructure >> isn't going to be ripped apart during said experiments. > > > Personally I applaud all efforts in any and all directions :) > > regards, Göran > > PS. Just for fun, given you have the domain model in *your* image the code > snippet in this pdf is stuff you can easily do: > http://swiki.krampe.se/gohu/uploads/11/squeakmap.pdf Nice! frank |
|
|
In reply to this post by Göran Krampe
On 04/15/2014 10:13 AM, Göran Krampe wrote:
> And right, documentation is probably scarce, sorry about that, but you > don't have to be snotty about it. Sorry about that Frank, I should have used a less offending word. Just a bit tired, sorry. regards, Göran |
|
|
In reply to this post by Frank Shearar-3
On 04/15/2014 10:36 AM, Frank Shearar wrote:
>> Also, "implementation detail" is... well, no, its a fair bit more than that >> I would say. Most web services just offer an API and keep the domain model >> on the server. SM doesn't - it offers a very simple API to actually download >> the whole model and then perform all readonly work on the client. > > Ah, but that assumes that the downloaded model is isomorphic to the > internal model. That's not necessarily the case. Can you elaborate? I don't understand. >> And right, documentation is probably scarce, sorry about that, but you don't >> have to be snotty about it. And I think many of the classes have class >> comments and the code is probably fairly well commented too (granted it was >> a long time ago so I don't really remember). > > My apologies: I didn't mean to sound snotty. Frustrated, yes, but not > snotty. (The frustration came from having to understand an entire HTTP > framework before I could start doing what I actually wanted to do.) > Good quality documentation is very expensive to produce. I should have picked a better word ;) What was it you actually wanted to do, and why did you need to understand the whole web framework? Just curious. ...and did you email me and ask? Perhaps you did, but I don't recall it. regards, Göran |
|
|
On 15 April 2014 11:37, Göran Krampe <[hidden email]> wrote:
> On 04/15/2014 10:36 AM, Frank Shearar wrote: >>> >>> Also, "implementation detail" is... well, no, its a fair bit more than >>> that >>> I would say. Most web services just offer an API and keep the domain >>> model >>> on the server. SM doesn't - it offers a very simple API to actually >>> download >>> the whole model and then perform all readonly work on the client. >> >> >> Ah, but that assumes that the downloaded model is isomorphic to the >> internal model. That's not necessarily the case. > > > Can you elaborate? I don't understand. The internal model might have extra data. For instance, it could store how many times someone's downloaded the package metadata or something. You might not want to push all these extra bits to the client. >>> And right, documentation is probably scarce, sorry about that, but you >>> don't >>> have to be snotty about it. And I think many of the classes have class >>> comments and the code is probably fairly well commented too (granted it >>> was >>> a long time ago so I don't really remember). >> >> >> My apologies: I didn't mean to sound snotty. Frustrated, yes, but not >> snotty. (The frustration came from having to understand an entire HTTP >> framework before I could start doing what I actually wanted to do.) >> Good quality documentation is very expensive to produce. > > > I should have picked a better word ;) > > What was it you actually wanted to do, and why did you need to understand > the whole web framework? Just curious. > > ...and did you email me and ask? Perhaps you did, but I don't recall it. No, I didn't :) I asked on the list a bit, pored over the code a lot (from a copy of the production image). frank > regards, Göran |
|
|
In reply to this post by Göran Krampe
On Tue, Apr 15, 2014 at 09:34:46AM +0200, G??ran Krampe wrote:
> Hi guys! > > Just FYI, I don't track all of squeak-dev (no time), but I can promise > to read and reply (to questions) on all threads mentioning SqueakMap in > the subject. > Hi G??ran, Thanks for your guidance on this :-) Dave |
|
|
In reply to this post by Frank Shearar-3
> I guess maybe I should have been more precise in what I was
> complaining about. I don't know much about SM's domain model, and I > don't care: that's an implementation detail of the service. What I > care about is the service's API. So if it's possible to take SM's > existing domain model, and SM's existing data, and just replace the > HttpView stuff with some new API (that's actually documented!), then > great! Less work for Chris Cunnington to do! Yes, the HttpView is the part that needs replaced. THAT is what provides the service API and the implementation details we don't care about whereas the domain model is the "gold" of the system -- the nougat that needs wrapped in a new server. Not only is it possible to keep the existing domain model whole, it is _essential_. There may be some tweaks we can make to it in the future (like maybe removal of the "published" flag) but we should just focus on the server implementation details right now. The other thing I think is important is to not _lose_ too much functionally just for the sake of a "new" server. The current server, for all of its faults, supports quite a bit of interesting capabilities such as the replication of the model to the client for offline use, and backup of external sites download links in case they go away, among others.. > The main point to my reply was that I'd like to see Chris Cunnington > supported in his experiments around SM, and I'd like to see Chris > Muller sleeping easily at night knowing that our core infrastructure > isn't going to be ripped apart during said experiments. As a dedicated volunteer, my primary goal orients around the end-results for Squeak. Certainly I like my beauty sleep and I want CC to be supported if his goal's are the same. But doing the work is no guarantee of acceptance, which is why it's good to engage community apriori. |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |