Going meta (VM development)

Hello,

spent another day building VM, i found that our setup is still far
from being simple..
the jenkins jobs right now using a page-long scripts to configure &
build vm, and grown with many bells and whistless..

one of my goals when i created a configurations was to avoid that..
and keep these things 1-liners..
one of the reasons for doing that is that you can reproduce the whole
build on local machine from command line
by typing a simple commands like make , as well as simple doits in
VMMaker image.

So, here some of my suggestions:
 - create a single PharoVMBuilder class, which will encompass all the
details of platform-specificness,
and will pick the right configuration & default plugins based on our
preference and target platform.
By issuing:

PharoVMBuilder generate.

- it should generate sources
- it should generate cmake files, based on environment variable set
(btw, can we access an environment variables in our VMs now?)
if not, then based on platform that image runs now , which is not good
for cross-platform builds, but since now we don't have cross-builds,
it will work fine.
The approach , again is the same: lets encapsulate the information we
need in edible form, not like this one:

#!bash -lex
#
# build-vm.sh -- Builds Cog Virtual Machine. Have to be used together
with Jenkins.
#
# Copyright (c) 2012 Christophe Demarey
#


export PHARO_VM=`$WORKSPACE/pharo-shell-scripts/fetchLatestVM.sh`
if [ -z "$PHARO_VM" ] ; then
   echo "PHARO_VM environment variable is not set."
   exit 1
fi

# Set environment ============================================================
PROCESS_PLUGIN="UnixOSProcessPlugin"
EXTERNAL_PLUGINS="FT2Plugin"
ZIP_FILTER='*'

if [ "$OS" == "win" ]; then
    set -e # Stop the execution if a command fails!
    CONFIGNAME="CogWindowsConfig"
    PROCESS_PLUGIN="Win32OSProcessPlugin"
    ZIP_FILTER=' *.exe *.dll'
elif [ "$OS" == "mac" ]; then
    CONFIGNAME="CogCocoaIOSConfig"
    export MACOSX_DEPLOYMENT_TARGET=10.5
    export CC=/usr/bin/gcc-4.2
    export CXX=/usr/bin/g++-4.2
else # [ "$OS" == "linux" ]; then
    CONFIGNAME="CogUnixConfig"
    export EXTERNAL_PLUGINS="SqueakSSLPlugin $EXTERNAL_PLUGINS"
fi

# Unpack sources =============================================================
cd "$WORKSPACE"

rm -rf cog
tar -xzf cog.tar.gz || echo # echo needed for Windows build (error on
symlinks creation)

mkdir -p cog/build
mkdir -p cog/image

cd cog/image
unzip -o ../../vmmaker-image.zip


# Generate sources ===========================================================
echo "$CONFIGNAME new
  addExternalPlugins: #( $EXTERNAL_PLUGINS );
  addInternalPlugins: #( $PROCESS_PLUGIN );
  generateSources; generate.

Smalltalk snapshot: false andQuit: true." > ./script.st
"$PHARO_VM" -headless generator.image script.st -headless

cd ../../
tar -czf ${CONFIGNAME}-sources.tar.gz -C cog .


# Compile ====================================================================
cd cog/build
if [ "$OS" == "win" ]; then
  cmake -G "MSYS Makefiles" .
else
  cmake .
fi
make
if [ "$OS" == "mac" ]; then
    make install
fi
cd ..

# Install SSL Plugin =========================================================
SSL_PLUGIN_URL='http://squeakssl.googlecode.com/files/SqueakSSL-bin-0.1.5.zip'
SSL_PLUGIN_ZIP='SqueakSSL-bin-0.1.5.zip'
SSL_PLUGIN_OUT='SqueakSSL-bin'
test -e 'SqueakSSL-bin-0.1.5.zip' || \
    ($WORKSPACE/download.sh $SSL_PLUGIN_ZIP $SSL_PLUGIN_URL && unzip
-o $SSL_PLUGIN_ZIP)
if [ "$OS" == "win" ]; then
    cp $SSL_PLUGIN_OUT/win/* results/
#elif [ "$OS" == "linux" ]; then
#    cp $SSL_PLUGIN_OUT/unix/so.SqueakSSL results/libSqueakSSL.so
fi
rm -rf $SSL_PLUGIN_OUT

# Archive ====================================================================
cd results
COG_ZIP_FILE="Cog-${OS}.zip"

zip -r $COG_ZIP_FILE $ZIP_FILTER
mv -f $COG_ZIP_FILE ../../

#if [ "$OS" == "mac" ]; then
#    zip -r ../build/CogVM.zip CogVM.app
#    mv -f ../build/CogVM.zip ../../
#else
#    zip -r Cog.zip $ZIP_FILTER
#    mv Cog.zip ../../
#fi

# success
exit 0


Another thing, which i discussed locally , i thinking about creating a
dedicated blog for VM developers, where we can share the news and
describe
details about changes and stuff..
In a way, that it will serve as a source of information about VM ,
jenkins , configurations, settings , building etc etc..
Because all of the bits of knowledge , which we collected till now is
kind of floating in the air, and like any energy it tends to dissipate
(things we remember and forget, the more things we should keep in
head, the faster it happens).
Also, blog is preferable to just simple CHANGELOG file (or similar),
because it is two-way communication , and people can comment and
discuss things.

--
Best regards,
Igor Stasenko.

Good. And adding an iOS entry for a StackVM version (or CogARM one of these days) would be great.

----- Original Message -----
| From: "Igor Stasenko" <[hidden email]>
| Another thing, which i discussed locally , i thinking about creating
| a
| dedicated blog for VM developers, where we can share the news and
| describe
| details about changes and stuff..
| In a way, that it will serve as a source of information about VM ,
| jenkins , configurations, settings , building etc etc..
| Because all of the bits of knowledge , which we collected till now is
| kind of floating in the air, and like any energy it tends to
| dissipate
| (things we remember and forget, the more things we should keep in
| head, the faster it happens).
| Also, blog is preferable to just simple CHANGELOG file (or similar),
| because it is two-way communication , and people can comment and
| discuss things.

Igor,

You might find GitHub as a very good place for this. GitHub has a number of collaboration features that play very well together:

  source code control with git
  issues where you can report bugs, but also host conversations
  wiki for documentation although I like markdown for documentation
    since the docs are then versioned with the code

When you view code diffs you can make line by line comments/questions that are visible to other developers who can in turn comment on your comments ... talk about real two-way communication:)

The real advantage is that all of these things are versioned together ... code/documentation/articles ...

Anyway, it's worth considering:)

Dale

On 30 July 2012 17:59, Dale Henrichs <[hidden email]> wrote:
Hi, Dale, do you have any links, nicely demonstrating the features you
described above?
For me, as an utterly lazy guy, who also doesn't loves writing much,
the simpler the better..
that's why i thought about just setting up a blog and posting notes
there, which is just few minutes..
Throwing git into the cycle would extend this time.. and as result, a
willingness to post about updates.. :)

> Dale
>
>
>



--
Best regards,
Igor Stasenko.

go for a blog like that this is visible to non geek too.

Stef

On Jul 30, 2012, at 7:55 PM, Igor Stasenko wrote:

Igor,

The Zinc project[1] is a good example.

First there is the README.md which introduces you to the project[2]. The README.md is a document (under version control) that can be updated by any member of the development team[3] in the GlassDB organization. The document uses markdown[6], so you get nice formatting and syntax highlighting for your Smalltalk code. You can see the history of the document[5] and easily access diffs (click around on the page). If you have permissions you can even edit the document via your web-browser (useful for minor tweaks).

The same set of capabilities also exist for any other text files that you might like shells scripts, makefiles, etc. Smalltalk code can be stored in the repository[7], via FileTree[8].

The README is not the only way to share information. Sven's writeup on Zinc is an additional example of shared documentation.

With regards to interacting with real-live code, take a look at the interactions of Paul and I on this pull request[10]. If you click on one of the files[11] in the comment boxes associated with the pull request you can see and example where Paul and I had a small discussion about changes to a particular Smalltalk method....

When Paul submitted the pull request, I made a pass through the changes and commented on each change that I had questions/comments about. Paul was notified of my comments and responded to each one in his own time ... and in some cases I came back and re-responded ... In this particular instance Paul and I realized that because we were seeing to many differences that "didn't make sense" that we were probably integrating the wrong code and ended up deciding to got with a different approach, that resulted in a separate pull request[12] being submitted and merged.

Here's the issues list[13]...

Some folks use github to host their own blogs as well, although I haven't looked into the details for that.

If you are looking for true interaction and collaboration, I think that github is hard to beat....

You do have to learn git to use github and I would not claim that git is easy to learn, but there are lots of resources on the web for teaching you git....if you don't have the time to learn git then that should be a factor, but you should weigh that against the amount of time spent trying to collaborate and the cost of learning git may just be worth it and then again it may not.

Dale

[1] https://github.com/glassdb/zinc
[2] https://github.com/glassdb/zinc/blob/gemstone2.4/README.md
[3] https://github.com/organizations/glassdb/teams/
[5] https://github.com/glassdb/zinc/commits/gemstone2.4/README.md
[6] http://github.github.com/github-flavored-markdown/
[7] https://github.com/glassdb/zinc/blob/gemstone2.4/repository/ConfigurationOfZincHTTPComponents.package/ConfigurationOfZincHTTPComponents.class/instance/version14..st
[8] https://github.com/dalehenrich/filetree
[9] https://github.com/glassdb/zinc/blob/gemstone2.4/zinc-http-components-paper.md
[10] https://github.com/glassdb/zinc/pull/1
[11] https://github.com/pdebruic/zinc/commit/5f09f70ba9abab24314ac8ed5e6cba668ac7c01e#repository-zinc-http-package-zndigestauthenticator-class-class-md5hash-st-P18
[12] https://github.com/glassdb/zinc/pull/1#issue-ref-16577627
[13] https://github.com/glassdb/zinc/issues?page=1&sort=updated&state=open
----- Original Message -----
| From: "Igor Stasenko" <[hidden email]>
| To: [hidden email]
| Sent: Monday, July 30, 2012 10:55:13 AM
| Subject: Re: [Pharo-project] Going meta (VM development)
|
| On 30 July 2012 17:59, Dale Henrichs <[hidden email]> wrote:
| >
| >
| > ----- Original Message -----
| > | From: "Igor Stasenko" <[hidden email]>
| > | Another thing, which i discussed locally , i thinking about
| > | creating
| > | a
| > | dedicated blog for VM developers, where we can share the news and
| > | describe
| > | details about changes and stuff..
| > | In a way, that it will serve as a source of information about VM
| > | ,
| > | jenkins , configurations, settings , building etc etc..
| > | Because all of the bits of knowledge , which we collected till
| > | now is
| > | kind of floating in the air, and like any energy it tends to
| > | dissipate
| > | (things we remember and forget, the more things we should keep in
| > | head, the faster it happens).
| > | Also, blog is preferable to just simple CHANGELOG file (or
| > | similar),
| > | because it is two-way communication , and people can comment and
| > | discuss things.
| >
| > Igor,
| >
| > You might find GitHub as a very good place for this. GitHub has a
| > number of collaboration features that play very well together:
| >
| >   source code control with git
| >   issues where you can report bugs, but also host conversations
| >   wiki for documentation although I like markdown for documentation
| >     since the docs are then versioned with the code
| >
| > When you view code diffs you can make line by line
| > comments/questions that are visible to other developers who can in
| > turn comment on your comments ... talk about real two-way
| > communication:)
| >
| > The real advantage is that all of these things are versioned
| > together ... code/documentation/articles ...
| >
| > Anyway, it's worth considering:)
| >
|
| Hi, Dale, do you have any links, nicely demonstrating the features
| you
| described above?
| For me, as an utterly lazy guy, who also doesn't loves writing much,
| the simpler the better..
| that's why i thought about just setting up a blog and posting notes
| there, which is just few minutes..
| Throwing git into the cycle would extend this time.. and as result, a
| willingness to post about updates.. :)
|
| > Dale
| >
| >
| >
|
|
|
| --
| Best regards,
| Igor Stasenko.
|
|

Ah, I see, the docs in markdown. I like it.
I tried to use markdown for NativeBoost documentation, and it quite nice..
But as Stef says, we need something non-geeky and easily accessible.

I agree that documentation , which is well written and with good
coverage is better to keep & organize together with sources and use of
SCM is completely reasonable for it.
But what i want to have is more like news posts, which not necessary
need to have too much details
and just serving to keep every developer(s) (and interested users) to
stay updated and informed about changes.
Because usually we post things into mailing list, and in a couple of
days, the information is buried under tons of other mails.
For documenting things we can use Cog project wiki, or files with
markdown .. and of course the useful information can be distilled from
blog posts, if needed.

That's my impression.

--
Best regards,
Igor Stasenko.

http://digwp.com/2010/06/blogging-in-markdown/ 

Phil

On 1 August 2012 12:54, [hidden email] <[hidden email]> wrote:
> Wouldn't this fulfill both requirements ?
>
> http://digwp.com/2010/06/blogging-in-markdown/
>
> Phil
>
yeeah.. unfortunately, wordpress.com seems like don't supports that
plugin.. or maybe it does, but i don't know where to look for (if you
can help, it will be nice)


--
Best regards,
Igor Stasenko.

Damn wordpress.com and let's self host this one.

Other than that, I'll look at how we can solve this one.



--
Philippe Back
Dramatic Performance Improvements
Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail: [hidden email] | Web: http://philippeback.eu | Blog: http://philippeback.be

High Octane SPRL
rue cour Boisacq 101
1301 Bierges
Belgium

On 1 August 2012 14:54, [hidden email] <[hidden email]> wrote:
> Other than that, I'll look at how we can solve this one.
>
ah.. don't waste your time on something which is not significant.

a usual posts is going to be like:
 - okay i installed/built VM like that, that and that..
 or
 i added this feature/fixed this, now it works and you can download it from ...
or
if you using plugin X/platform Y, to build VM , do not forget about Z.. etc etc

so, it don't needs to be in highly malleable format such as markdown


--
Best regards,
Igor Stasenko.

Igor Stasenko wrote

yeeah.. unfortunately, wordpress.com seems like don't supports that
plugin.. or maybe it does, but i don't know where to look for (if you
can help, it will be nice)

Per http://en.support.wordpress.com/plugins/ : "Plugins... are only applicable to self-hosted blogs and web sites using the WordPress.org software. Plugins are not permitted here at WordPress.com for various security reasons."

If you can provide me with a login, I am going to write my iOS VM StackVM story :-)

And someone should nag Nick Ager to share his experience telling the
UKSTUG folk about the Pharo VM. (With the help of the audience he got
Pharo running in the iPhone simulator.)

Nick, write it up!

frank

On 1 August 2012 15:13, [hidden email] <[hidden email]> wrote:

On 1 August 2012 16:13, [hidden email] <[hidden email]> wrote:
> If you can provide me with a login, I am going to write my iOS VM StackVM
> story :-)
>
done



--
Best regards,
Igor Stasenko.

If you're not going to be intimately collaborating on code, then a general blog will work just fine ...

Dale

----- Original Message -----
| From: "Igor Stasenko" <[hidden email]>
| To: [hidden email]
| Sent: Wednesday, August 1, 2012 2:10:47 AM
| Subject: Re: [Pharo-project] Going meta (VM development)
|
| Ah, I see, the docs in markdown. I like it.
| I tried to use markdown for NativeBoost documentation, and it quite
| nice..
| But as Stef says, we need something non-geeky and easily accessible.
|
| I agree that documentation , which is well written and with good
| coverage is better to keep & organize together with sources and use
| of
| SCM is completely reasonable for it.
| But what i want to have is more like news posts, which not necessary
| need to have too much details
| and just serving to keep every developer(s) (and interested users) to
| stay updated and informed about changes.
| Because usually we post things into mailing list, and in a couple of
| days, the information is buried under tons of other mails.
| For documenting things we can use Cog project wiki, or files with
| markdown .. and of course the useful information can be distilled
| from
| blog posts, if needed.
|
| That's my impression.
|
| --
| Best regards,
| Igor Stasenko.
|
|

I think Nick Ager has a great tool for this purpose. He showed us that in PharoConf.