Re: Creating the smallest server runtime footprint
Locked
12
12
Re: Creating the smallest server runtime footprint
 
|
Re: Creating the smallest server runtime footprint
|
|
Re: Creating the smallest server runtime footprint
|
|
Re: Creating the smallest server runtime footprint
|
|
Yes, you should be able to load an mcz in that image by doing:
(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load. Guille On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <[hidden email]> wrote:
|
Re: Creating the smallest server runtime footprint
|
|
Ah, I think I’m starting to get closer to what I need…
So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below? Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up? I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects. Tim
|
Re: Creating the smallest server runtime footprint
|
|
Re: Creating the smallest server runtime footprint
|
|
In reply to this post by Tim Mackinnon
The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all): Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package and create baseline to load them. I already tried it so you can test it: Then you will do something like: 2017-08-02 15:18 GMT+02:00 Tim Mackinnon <[hidden email]>:
|
Re: Creating the smallest server runtime footprint
|
|
2017-08-03 10:14 GMT+02:00 Pavel Krivanek <[hidden email]>:
(accidental message send ;-) ) BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src'; load." BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src'; load." As I tried that. The baseline of Lambda is then able to be loaded. -- Pavel
|
Re: Creating the smallest server runtime footprint
|
|
I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…
When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src. ? Just so I know for the future. Tim
|
Re: Creating the smallest server runtime footprint
|
|
2017-08-03 13:02 GMT+02:00 Tim Mackinnon <[hidden email]>:
yes -- Pavel
|
Re: Creating the smallest server runtime footprint
|
|
I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?
I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)
Tim
|
Re: Creating the smallest server runtime footprint
|
|
2017-08-03 13:58 GMT+02:00 Tim Mackinnon <[hidden email]>:
you probably need: spec package: 'Fuel'. spec package: 'FuelPlatform-Core'. spec package: 'FuelPlatform-Pharo-Core'. spec package: 'FuelPlatform-Pharo-06'. It has some dependencies on other packages that you do not load so it should cause some warnings during loading
If you will copy them to your repository then add them to your baseline. For loading of packages without baseline you may use Gofer. - Pavel
|
Re: Creating the smallest server runtime footprint
|
|
In reply to this post by Tim Mackinnon
On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <[hidden email]> wrote:
The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello. Sometimes you can print the result of #record: Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core'). -> "linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06" ------ Anyway.... I guess I would try to load these packages (in this order): FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel And when you are down loading, then execute: (Smalltalk at: #FLPlatform) current addHacks
|
Re: Creating the smallest server runtime footprint
|
|
Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames
I can see that this is a trait - is trait support missing in the minimal image? (I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more? Tim "errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"
|
Re: Creating the smallest server runtime footprint
|
|
On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <[hidden email]> wrote:
Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)
TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.
|
Re: Creating the smallest server runtime footprint
|
|
Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method",
"\u001b[0mLambda class(Object)>>error:", "explicitRequirement”, Any ideas? Tim "--- RUNNING ERROR HANDLER ---",
"Error: Explicitly required method",
"",
"\u001b[0m\u001b[31mError: Explicitly required method",
"\u001b[0mLambda class(Object)>>error:",
"explicitRequirement",
" self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...",
"False>>ifFalse:",
"Lambda class(Object)>>explicitRequirement",
"Lambda class(Class)>>fuelAccept:",
"FLLightGeneralMapper>>mapAndTrace:",
"FLLightGlobalMapper>>mapAndTrace:",
"FLPluggableSubstitutionMapper>>mapAndTrace:",
"FLAnalysis>>mapAndTrace:",
"FLAnalysis>>run",
"setDefaultAnalysis",
" self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...",
"FLAnalyzer>>analysisFor:",
"FLSerialization>>analysisStep",
"FLSerialization>>run",
"setDefaultSerialization",
" self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...",
"serialize: t1 on: t2",
" self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...",
"on: t1 globalEnvironment: t2 do: t3",
" self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...",
"BlockClosure>>ensure:",
"FLEncoder class>>on:globalEnvironment:do:",
"FLSerializer>>serialize:on:",
"NonInteractiveUIManager>>writeFuelContext:",
"Lambda class>>processDebug:",
"Lambda class>>processJSON:",
"Lambda class>>process:",
"activate",
|
Re: Creating the smallest server runtime footprint
|
|
On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <[hidden email]> wrote:
mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass: fuelAccept: aGeneralMapper ^self explicitRequirement. Rather than the correct one: fuelAccept: aGeneralMapper ^aGeneralMapper visitClass: self Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods. If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ? Thoughts?
|
Re: Creating the smallest server runtime footprint
|
|
In reply to this post by Tim Mackinnon
2017-08-03 18:17 GMT+02:00 Tim Mackinnon <[hidden email]>:
No, this image supports them
|
Re: Creating the smallest server runtime footprint
|
|
If Traits are supported then I must be doing something else wrong - maybe load order? (To be honest, I don’t fully understand how traits work - but I will keep digging).
It could be that my latest error (having just done Class compile: ‘…. Missing method code … ‘) is something deeper related to these traits methods not applying somehow… I’ll try a few things this morning. It’s amazing how easily the image size jumps up when you add things though. With the original minimal v6 64bit image you sorted out for me I had a footprint of 21.6mb (image was 12mb). Now with the v7 image - and adding fuel and some S3 handling - it jumps up to 32.4mb (image is 15mb). I know that the S3 support has a lot of XML crud - but it seems quite astounding that this takes almost 8mb of code/objects. AWS loading (and hence response time) seems quite sensitive to the size of the package it has to load - although equally I wonder if there is also more #startUp/initalisation cost that is going on as well as when you get a warm lambda instance (so its not just loading your package) I’m still not seeing the 400-500ms times but more 800-1000ms times. Its all quite interesting - and shows why the work behind a minimal image is very useful. Of course there is tiny-minimal (which I know you guys are after) and then there is small-server- minimal (which is realistically what I’m after, so I can just focus on creating my app and just carefully load code and dependencies - but still have core server services like Fuel, and Git loading). Thanks for doing this work - as it brings a lot of useful things together. Tim
|
Re: Creating the smallest server runtime footprint
|
|
In reply to this post by Mariano Martinez Peck
Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :
fuelAccept: aGeneralMapper ^self explicitRequirement. Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?). Thanks, for helping by the way - this is proving very instructional. Tim
|
| Free forum by Nabble | Edit this page |