Is there an API for SqueakSource? I would like to retrieve all projects
from www.squeaksource.com and add all repositories to the Monticello Browser. And I'd hate to use Web scraping. ;-) If not, I could probably do it myself. It would probably be implemented much like the rss feed feature. What do you think? Before I forget, SqueakSource is really great! Thanks a lot! Cheers, - Bernhard |
2006/2/19, Bernhard Pieber <[hidden email]>:
> Is there an API for SqueakSource? No, there isn't yet. > I would like to retrieve all projects > from www.squeaksource.com and add all repositories to the Monticello > Browser. And I'd hate to use Web scraping. ;-) This sounds very interesting, it basically means client-side support for SqS in Squeak. This would make SqS more like SqueakMap. An other intersting option this opens is it allows us to run expensive jobs like activity analysis on the second cpu. > If not, I could probably do it myself. It would probably be implemented > much like the rss feed feature. What do you think? I don't know if we need that much overhead. You wan't a list of projects and maybe their title, so why not use CSV? Much simpler, much more efficient, no XML-parser required. Cheers, Philippe |
"Philippe Marschall" <[hidden email]> wrote:
> An other intersting option this opens is it allows us to run expensive > jobs like activity analysis on the second cpu. I must admit that I did not understand this. Can you elaborate? > > If not, I could probably do it myself. It would probably be implemented > > much like the rss feed feature. What do you think? > I don't know if we need that much overhead. You wan't a list of > projects and maybe their title, so why not use CSV? Much simpler, much > more efficient, no XML-parser required. You are absolutely right, CSV would be much simpler for my use case. Cheers, Bernhard |
In reply to this post by Philippe Marschall
2006/2/19, Bernhard Pieber <[hidden email]>:
> "Philippe Marschall" <[hidden email]> wrote: > > An other intersting option this opens is it allows us to run expensive > > jobs like activity analysis on the second cpu. > I must admit that I did not understand this. Can you elaborate? A long time ago, in a galaxy far far away, we had a realtime statistics graphic for SqS (number of projects, ...). You can see this in the SqS Paper submitted ESUG. As SqS grew to it's current size, this caused all kinds of problems one of them was that it is too cpu intensive. The code is still there, just commented out. SqS now runs on a dual CPU machine but we can use only one CPU, because we don't need no stinkin' Java threads. If we can access the data from outside the image, we could set up a cron job, that starts a script (or image) that fetches the data, computes the graphic, writes it to the filesystem and then terminates. We can then serve the graphic with apache. Cheers Philippe |
On Sun, Feb 19, 2006 at 12:24:20PM +0100, Philippe Marschall wrote:
> 2006/2/19, Bernhard Pieber <[hidden email]>: > > "Philippe Marschall" <[hidden email]> wrote: > > > An other intersting option this opens is it allows us to run expensive > > > jobs like activity analysis on the second cpu. > > I must admit that I did not understand this. Can you elaborate? > > A long time ago, in a galaxy far far away, we had a realtime > statistics graphic for SqS (number of projects, ...). You can see this > in the SqS Paper submitted ESUG. As SqS grew to it's current size, > this caused all kinds of problems one of them was that it is too cpu > intensive. The code is still there, just commented out. > SqS now runs on a dual CPU machine but we can use only one CPU, > because we don't need no stinkin' Java threads. If we can access the > data from outside the image, we could set up a cron job, that starts a > script (or image) that fetches the data, computes the graphic, writes > it to the filesystem and then terminates. We can then serve the > graphic with apache. cron job?!? we don't need no stinkin' cron jobs neither ;) [[ | pipe backgroundJob ws | pipe := OSPipe new. backgroundJob := OSProcess thisOSProcess forkHeadlessSqueakAndDoThenQuit: [OSProcess accessor nice: 1. "lower priority of background OS process" (99 to: 1 by: -1) do: [:e | pipe nextPutAll: e asString; nextPutAll:' bottle'; nextPutAll: ((e = 1) ifTrue: [''] ifFalse: ['s']); nextPutAll: ' of beer on the wall, '; nextPutAll: e asString; nextPutAll:' bottle'; nextPutAll: ((e = 1) ifTrue: [''] ifFalse: ['s']); nextPutAll: ' of beer'; cr; nextPutAll: 'take one down and pass it around, '; nextPutAll: ((e = 1) ifTrue: ['no more'] ifFalse: [e - 1]) asString; nextPutAll:' bottle'; nextPutAll: ((e = 2) ifTrue: [''] ifFalse: ['s']); nextPutAll: ' of beer on the wall'; cr]. pipe close]. Transcript show: backgroundJob asString, ' started'; cr. pipe closeWriter. "don't need writer end, close it before the #upToEnd" "pipe writer blocks when pipe full, so we need to loop while reading to end" ws := WriteStream on: ''. [backgroundJob isComplete] whileFalse: [(Delay forMilliseconds: 200) wait. pipe upToEnd ifNotNilDo: [:s | ws nextPutAll: s]]. pipe close. Transcript show: backgroundJob asString, ' completed, display results in 2 seconds'; cr. (Delay forSeconds: 2) wait. Transcript show: ws contents. Transcript cr; show: 'delay 5 seconds before forking next Squeak job'; cr. (Delay forSeconds: 5) wait] repeat] forkAt: Processor userBackgroundPriority. It's usually more efficient to do it this way, because you don't need to start up a new image every time you run the job. It also avoids the files and cron jobs, which IMHO are even uglier than the aforementioned Java threads. Dave |
> cron job?!? we don't need no stinkin' cron jobs neither ;)
Yeah, but we have made a lot of bad experiences with long running delays especially in exactly such cases. Cron jobs work. And IIRC SqS runs on a Mac VM, so no forking either. Philippe |
On 19-Feb-06, at 9:16 AM, Philippe Marschall wrote: >> cron job?!? we don't need no stinkin' cron jobs neither ;) > > Yeah, but we have made a lot of bad experiences with long running > delays especially in exactly such cases. There have been recent-ish improvements in long delays so if you have any way to test for the problem it would be very helpful. See mantis 1840 - http://bugs.impara.de/view.php?id=1840 > Cron jobs work. And IIRC SqS > runs on a Mac VM, so no forking either. Really? I'm pretty sure John will say OSX would do forking. Has dave lewis done an osprocess plugin for osx yet? tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Oxymorons: Exact estimate |
Hi tim
I remember harvesting some changes of David. Should these ones get harvested too? Or are you "simply" asking for feedback :)... Stef On 19 févr. 06, at 19:15, tim Rowledge wrote: > > On 19-Feb-06, at 9:16 AM, Philippe Marschall wrote: > >>> cron job?!? we don't need no stinkin' cron jobs neither ;) >> >> Yeah, but we have made a lot of bad experiences with long running >> delays especially in exactly such cases. > There have been recent-ish improvements in long delays so if you > have any way to test for the problem it would be very helpful. See > mantis 1840 - http://bugs.impara.de/view.php?id=1840 > >> Cron jobs work. And IIRC SqS >> runs on a Mac VM, so no forking either. > Really? I'm pretty sure John will say OSX would do forking. Has > dave lewis done an osprocess plugin for osx yet? > > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Oxymorons: Exact estimate > > > |
I read the mantis entry....
so shout for feedback :) Stef On 19 févr. 06, at 19:33, stéphane ducasse wrote: > Hi tim > > I remember harvesting some changes of David. > Should these ones get harvested too? > Or are you "simply" asking for feedback :)... > > Stef > > On 19 févr. 06, at 19:15, tim Rowledge wrote: > >> >> On 19-Feb-06, at 9:16 AM, Philippe Marschall wrote: >> >>>> cron job?!? we don't need no stinkin' cron jobs neither ;) >>> >>> Yeah, but we have made a lot of bad experiences with long running >>> delays especially in exactly such cases. >> There have been recent-ish improvements in long delays so if you >> have any way to test for the problem it would be very helpful. >> See mantis 1840 - http://bugs.impara.de/view.php?id=1840 >> >>> Cron jobs work. And IIRC SqS >>> runs on a Mac VM, so no forking either. >> Really? I'm pretty sure John will say OSX would do forking. Has >> dave lewis done an osprocess plugin for osx yet? >> >> >> tim >> -- >> tim Rowledge; [hidden email]; http://www.rowledge.org/tim >> Oxymorons: Exact estimate >> >> >> > > > |
In reply to this post by stéphane ducasse-2
On 19-Feb-06, at 10:33 AM, stéphane ducasse wrote: > Hi tim > > I remember harvesting some changes of David. > Should these ones get harvested too? > Or are you "simply" asking for feedback :)... The only change I can directly comment on is the 1840 change to Delays. It's in your mantis pile right now so that you can consider what to do with it; my advice would be that we should try to get some opinion from a few more people with an interest in log delays. Dave Lewis, Lex and laza all took part in the mantis 854 discussion so they would be logical interested parties. Philippe seems a good idea too since he claims experience in problems with long delays. Messing with delay is something to be done slowly, carefully and after much standing around pontificating with a bunch of grey bearded oldsters. My suggested change in 1840 is merely an opening suggestion. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim I'm so skeptical that I'm not sure I'm really a skeptic |
In reply to this post by timrowledge
Well the OSProcess plugin wouldn't quite work with the carbon vm
because of: a) x/11 (that was solvable by just commenting out a few things) b) unix path names. (that was harder). c) bundle versus unix library. I'm working on (c) at the moment, (b) is solved by using a 3.8.10 VM At this time I need some folks to help in recompiling and testing the OSProcess plugin to support the mac carbon vm. I'll note the carbon VM actually runs a number of OS threads at the moment. 1) For updating the low resolution clock every 1/60 of a second 2) For managing the mouse/keyboard/event input via carbon event manager. 3) The interpreter loop 4) OS thread for input/output (created by carbon logic) 5) OS thread for sound (created by carbon logic) 6) Lots of other threads if you use quicktime via FFI. However since the carbon VM doesn't run headless forking the VM I'll guess is a problem... Mind starting up multiple headless unix squeak VM managed by a headful squeak VM could be doable. On 19-Feb-06, at 10:15 AM, tim Rowledge wrote: > > On 19-Feb-06, at 9:16 AM, Philippe Marschall wrote: > >>> cron job?!? we don't need no stinkin' cron jobs neither ;) >> >> Yeah, but we have made a lot of bad experiences with long running >> delays especially in exactly such cases. > There have been recent-ish improvements in long delays so if you > have any way to test for the problem it would be very helpful. See > mantis 1840 - http://bugs.impara.de/view.php?id=1840 > >> Cron jobs work. And IIRC SqS >> runs on a Mac VM, so no forking either. > Really? I'm pretty sure John will say OSX would do forking. Has > dave lewis done an osprocess plugin for osx yet? > > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Oxymorons: Exact estimate > > > -- ======================================================================== === John M. McIntosh <[hidden email]> 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== === |
On Sun, Feb 19, 2006 at 12:23:26PM -0800, John M McIntosh wrote:
> Well the OSProcess plugin wouldn't quite work with the carbon vm > because of: > > a) x/11 (that was solvable by just commenting out a few things) > b) unix path names. (that was harder). > c) bundle versus unix library. > > > I'm working on (c) at the moment, (b) is solved by using a 3.8.10 VM Hi John, Problem a) should be fixed in recent OSProcess also. Martin Snelgrove sent me some OSProcess test results a while back, running on the following: wms running OSProcess unit tests 19 October 2005 11:18:36 pm OSProcess platformName => unix OSProcess platformSubtype => powerpc OSProcess osVersion => darwin7.8.0 OSProcess vmVersion => Squeak3.7 of '4 September 2004' [latest update: #5989] The #forkSqueak methods were all working in the unit tests, so I think that forking a headless Squeak will work on OS X: #testForkHeadlessSqueakAndDo->1 run, 1 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes #testForkHeadlessSqueakAndDoThenQuit->1 run, 1 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes #testForkSqueak->1 run, 1 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes #testForkSqueakAndDo->1 run, 1 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes #testForkSqueakAndDoThenQuit->1 run, 1 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes Quite honestly I don't know the difference between a darwin and a carbon (I thought carbon was for the brushes on my harley's generator) but I was pleasantly surprised that #forkSqueak was working on at least some Mac flavors, so I think this is a doable thing. > At this time I need some folks to help in recompiling and testing the > OSProcess plugin to support the mac carbon vm. If you find a volunteer for this, please CC me and I'll provide whatever remote support I can. I don't have a Mac to work with, but OSProcess class>>allTestResults produces some useful output that may allow me to help. > I'll note the carbon VM actually runs a number of OS threads at the > moment. > > 1) For updating the low resolution clock every 1/60 of a second > 2) For managing the mouse/keyboard/event input via carbon event manager. > 3) The interpreter loop > 4) OS thread for input/output (created by carbon logic) > 5) OS thread for sound (created by carbon logic) > 6) Lots of other threads if you use quicktime via FFI. Here I suspect I have some work to do. OSPP does not try to account for pthreads, which means that my signal handling is probably not going to work right. There may be other things that need to be pthread-aware also. I don't know if pthreads are supported on all unix platforms (I guess I am a little suspicious of anything called "posix standard"), but I suppose it's pretty generic stuff nowadays. Which means I probably aught to RTMF and add the required support in OSPP. > However since the carbon VM doesn't run headless forking the VM I'll > guess is a problem... Mind starting up multiple headless unix squeak > VM managed by a headful squeak VM > could be doable. Definitely doable, as it seems to already work on some OS X flavors. Dave |
In reply to this post by Philippe Marschall
My pet use would be to have a sort of mini-squeak source running in
my production seaside image. IOW, my production image would just look like a SS repository but saves would actually update the running image. Then I could push fixes directly from my development laptop to my deployed application on my production server. So I'd also like a SS api because I think we could use it to build automated deployment systems similar to ruby's signal tower. On Feb 19, 2006, at 1:59 AM, Philippe Marschall wrote: > 2006/2/19, Bernhard Pieber <[hidden email]>: >> Is there an API for SqueakSource? > No, there isn't yet. |
Hi Todd,
Todd Blanchard <[hidden email]> wrote: > My pet use would be to have a sort of mini-squeak source running in > my production seaside image. IOW, my production image would just > look like a SS repository but saves would actually update the running > image. Then I could push fixes directly from my development laptop > to my deployed application on my production server. This sounds like a useful use case. However, I don't understand why you need full SqueakSource, i.e. the Web UI, for that. Wouldn't just Monticello and a file upload be enough? Cheers, Bernhard |
In reply to this post by tblanchard
Am 20.02.2006 um 23:50 schrieb Bernhard Pieber: > Hi Todd, > > Todd Blanchard <[hidden email]> wrote: >> My pet use would be to have a sort of mini-squeak source running in >> my production seaside image. IOW, my production image would just >> look like a SS repository but saves would actually update the running >> image. Then I could push fixes directly from my development laptop >> to my deployed application on my production server. > This sounds like a useful use case. However, I don't understand why > you > need full SqueakSource, i.e. the Web UI, for that. Wouldn't just > Monticello and a file upload be enough? Sure. But it would be *way cool* if you just open a regular web browser to update your image, including a nice UI :) - Bert - |
Free forum by Nabble | Edit this page |