Beyond Live Coding

My apologies if I wasn't clear. I wasn't stating that Pharo was crash prone. I was stating the exact opposite and that other languages are crash prone. That in other languages that a program crashing or misbehaving in some way is a normal part of the development process. It is one of the ways of discovering problems in the developing app. I am no way saying that deployed apps are crash prone  What I was saying is that in any language, I the programmer can do something stupid. In Pharo, I can recover from my errors without my object environment crashing. Other languages my errors crash my program and I start the process all over with external editor, debugger, repl, or compiler. I will readily admit I am not expert or good in that world and probably lack knowledge.

What I seemingly failed to express is that I believe Pharo (Smalltalk) is far more robust than most any other language I have experienced. Because programmer errors are most frequently caught by the debugger and not crash the environment. Then you the programmer have the ability to correct your errors in the same environment you develop, debug and execute the application. Where else does this happen? Maybe some Lisp systems?

The fact that the Pharo (Smalltalk) live object environment is basically an operating system. So it provides certain robustness that other live coding doesn't. And unlike any current operating system I have used or am aware of, Pharo (Smalltalk) has no concept of restart. Everything happens by adding, deleting or evolving internally to the environment. In fact application development is the same. App version 2 can be easily arrived at by evolving App version 1 live.

And yes we can drop down into the everything is Turing complete and everything can do anything that anything else can do type of argument. Yes we can accomplish this. We hold all state in a database. The app is functions on top of that database. ... And then we accomplish some poor facsimile of this. But it is not natural nor native to the development processes of other languages.

A person brand new to Pharo gives no thought to persistence of state, or committing of code or a whole host of details which go into providing this luxury we enjoy. What is implicit in Pharo must be come explicit elsewhere with difficulty and still will not quite reach where we are by nature.

Pharo (Smalltalk) is different. It is so much bigger, more complete and seamless than the concept of live coding anywhere else. I don't think that most of us here would be fighting this battle (and I include you as one of Pharo people :) if other more "practical" languages approached what Pharo (Smalltalk) offers. If we thought that X language had 80%+ of these features and with some effort we could start bringing the rest. I don't know that we would be here. But they don't.

And nothing in Pharo prevents you from doing similar. Persist your state in an external database. Have your app operate over that database. Use Iceberg to commit your app code to git. And I would wager that it is still a much better experience than other languages doing the same thing. Simply because the Pharo environment understands your code, because your code is its code. The lines are blurred. No big distinction. My separate editor/IDE, my separate debugger, my separate execution environment. Yes I know there are many programmers (probably most) who prefer this separation. I would actually say most don't even know that what Pharo (Smalltalk) does is possible. They have only been taught the other way.

Once again. In other languages everything is external to the environment. The editor, the debugger, the database, ...  Whereas Pharo, all is inside the environment. It opens the capabilities tremendously.

I have played with Jetbrains Community edition. It is very nice for what it is. A dead file language IDE. It possibly is the best. I played with it for a while to consider Kotlin. I opened IntelliJ IDEA. It opened one window, with one Kotlin project, with one 65 line file, and that one file open in the editor. It consumed 710mb of ram. In order to perform the magic it does on dead external languages and source in files requires extraordinary memory. Now if I open a Kotlin REPL within IntelliJ in order to be able to do something remotely lively. It goes up another 10mb and opens an external REPL which now consumes 57mb ram itself. At this point the REPL is idle and doing nothing. If I actually do something in that REPL it now explodes to 368mb of ram. Yes, I know this is Kotlin and therefore Java. Other languages such as Python and PyCharm will be somewhat different as far as the REPL goes. So for due diligence I am now going to download PyCharm, install and do an identical Python file. It is a multi-language benchmark.

While on the website I see a familiar name (Paul Everitt) from when I used to program with Bobo and then when it became Zope, many, many years ago.

PyCharm fresh start with one project, with one 80 line file, with that file open in an editor, weighs in at 675mb of ram. Opening a Python console from PyCharm adds another 10mb and the console itself used 21mb. This is on a Xubuntu 17.04 64bit laptop.

Whereas I have a full Pharo 6 image with an app I am working on which has 26 classes, numerous methods. The app is currently running and has parsed for used several multi-mb files for use. This image currently uses 105mb of ram. It has a much better understanding of the code it contains than does IntelliJ. Because for my app, the Pharo environment contains the IDE and the debugger and my app and all of the other tools available to Pharo. It is exceptionally rich in capabilities. All for currently 1/6th or less of the weight of PyCharm or IntelliJ.

This demonstrates the reasonable efficiency of Pharo (Smalltalk) verses other ways of development. Even if you do simple. Simple editor, repl, etc. you still do not save much if any over Pharo, and you give up much.

In other languages is the IDE one with the language, debugger, environment and application? No, it is not. This is exactly why I titled this Beyond Live Coding.
If you manage to simulate what we have in Pharo you only have a poor facsimile of what Pharo (Smalltalk) is in another language. You are not likely to have a culture or community around it support it.

Personally I find code in file based languages to not be as friendly for committing and source control. I am not good at it. I have a tendency to edit to large a section of code and commit to infrequently. I have not mastered to development cycle sufficiently. In Pharo such is not necessary. Within an image method based commits are exceptionally nice. You do have to learn Monticello and now Iceberg to go beyond the image. But it is being worked on to make it a natural part of development. And as I said, method level commits are native to the way Pharo (Smalltalk) works. So you already do frequent, smaller commits. You would need to insure that you do so in reasonable intervals for Monticello or Iceberg.

Pharo is not perfect, far from it. Hence why I sometimes struggle. It requires much more effort to do things if what you want to do have requirements beyond the image. Such as using libraries or accessing systems external to Pharo. This is a requirement I have. Fortunately for me one of the systems I require has released a REST API for access. So I no longer require going through FFI for that access. I attempted some FFI with NativeBoost and it was not fun. It crashed my image regularly and I was not skilled enough to move beyond. I hope UFFI and Pharo 6 is better. Because it is still very important to access C libraries. If it isn't, then I still might have problems.

Pharo still isn't good at truly small systems. Pharo 7 and 8 should help that nicely. It will be interesting to see how small the VM and image can be and have a working app. I hope it will be nice.

I like the idea of small. Something I can understand. Something that doesn't require GBs of ram for an IDE . I would rather do PicoLisp which has an incredibly fast interpreter, is small and is understandable on both the language and implementation side. It is the one language that intrigues and tempts me. But coming from Pharo it feels primitive. It would take some thought and effort. It might be worth the effort.

The nice thing is that everyone can choose what fits them and their situation. What fits the way they think. What fits their goals. I just struggle with the idea of needing 700mbs to GBs of ram to run an IDE. I would rather do small. Get it in my head and fingertips. Nice thing about choice. My choice isn't required of anyone else. :)

Thanks for the conversation.

Have a great day.

Jimmie

Live coding as a concept does not exist in most languages. The reason why they can do live coding is because out of necessity . Take exceptions for example , almost never you will find any mention about live coding but you will read about code exiting elegantly or not exiting at all and reloading modules etc.

In the end all coders need a lot of flexibility during execution , especially during the process of debugging. Even in Pharo live coding shines the most during debugging process this is why they go hand in hand.

The moment you have live code and state having live objects is just a consequence of having live code. If you approach it from a pure object orientated approach that is not unique either. Because nowadays we have many extremely popular languages like Python and Ruby . Python for example suppose to be a hybrid procedural and OOP language but that is a lie because in Python functions are objects. Ruby is almost Smalltalk. So both Python and Ruby are pure object orientated.

Pharo crash prone behavior is not extensive. Live state will be lost for data living outside Pharo. If the data is accessed or copied from Pharo the copy will be retained through saving the image but if you don't save the live state of Pharo will be lost. Even if you do save the image the live state on the C side won't be stored so it's up to you to restore if you have the means to do so and already took the appropriate measures. This is why I made a big deal for memory mapped files because essentially they store using an auto save feature live c and C++ state.

Another huge problem for Pharo is that crashes can corrupt the image which means you will have to rebuild it from scratch which can be so tedious if you have lost a lot of live state.

Recovery of code is not a big deal for other programming languages cause unlike Smalltalk they are based on source code files and most modern IDEs will auto save your source code file. If you use version control that becomes unnecessary if you commit often like I do because essentially you become unable to lose code.

Smalltalk and Pharo shines in the idea of a single IDE but the community is too small to afford to be fragmented between many different IDEs. In other languages some IDEs are closed source , some are open source but the trend for the past decade is toward open source IDEs. For example I code in Python and by far the most popular IDE is PyCharm , it's closed source but it has a free edition called "Community Edition" that contains most of of the commercial features (80%) it's one source and free and hosted in Github

https://github.com/JetBrains/intellij-community/tree/master/python

The idea of self contained environments is no big deal either because we have git forks , Python even comes with another features that allow you to create isolate environments that like Pharo contain your code, the language and any library used.

Smalltalk was a bIg deal decades ago but no longer is because coders themselves may not be aware of either Smalltalk or Lisp but language and IDE designers and library developers are very much aware as both languages are considered the golden standard and very much respected.

Pharo is important because is a modern implementation of Smalltalk and tries to push its concepts further. Unlike other language it's not just about copying Smalltalk or canibalizing it's ideas. It's crucial that Smalltalk continues to evolve because afterall is a language of innovation.

None the less we live in a time that modern technology is very close to Smalltalk and most of its features have infected modern coding. They are fragmented or not that obvious or out of the box offered but they are there and experienced coders already rely on them to speed up their workflow.

Languages like C distance themselves from Smalltalk because they prioritize performance. It's the usual case of not having your cake and eat it too. But since every language including Smalltalk is based on C for performance you can implement many of the features. Problem is that you will be reducing performance so in that case it takes more sense to use Pharo or Python or what else.

In any case Pharo is a great environment to work with many flaws and unique advantages in the end Pharo is Pharo offering its own unique experience we all love and respect. Emulation is never the same as the actual thing.

On Fri, 21 Jul 2017 at 19:36, Jimmie Houchin <[hidden email]> wrote:

Pharo (Smalltalk) is much more than simply live coding. All of us here
dearly love live coding. Kilon states it is possible to do live coding
in other languages. I have no experience doing so in any other language.
I will say that I know of no other language that it is integral to
development in that language.

Pharo is a live object environment. It has objects that would take
someone with a better understanding of the entire image than myself to
tell when the object was created. But suffice it to say that Pharo
(Smalltalk) have some very old objects, possibly back to 1980 or 1972?

The live environment supports live objects, not just live coding as
understood by someone who may attempt live coding in some language that
does not by nature do live coding. Objects have state and behavior.
Objects in Pharo grow and shrink in behavior and state as messages are
sent to them. They evolve, they dynamic not static.

Pharo's environment can survive crashes. Yes, I know, there are times
when some unrecoverable event happens and you can not recover lost
changes. But the community attempts to fix anything that would cause
such an event. Other languages, do not have such. Segfaulting is normal,
to be expected. You go back to your dead code editor and fix what caused
the segfault and start up again, clean without the state your objects
were in when it happened or even  before. They do autopsies on their
core dumps to attempt to discover what went wrong. Yes, you can
explicitly choose to run a debugger on your app to watch its state and
behavior.

Pharo's environment survives shutdown of its environment and shutdown or
restart of an operating system. Once restarted your Pharo environment is
exactly as it was with all it's live objects and live code. It survives
change of operating systems and computers. All the image needs is the
appropriate vm for the image and the operating system currently running.

Pharo can survive stupid programmer errors. I have written infinite
loops and lived to tell about it. My objects and environment survived. I
have written code which took hours to execute over a large set of data
only to discover I had an error in my code on the last method or so that
executed and it raised a debugger. I fixed it and continued. Nice. In
some other language, you fix and start over executing that multiple hour
process and hope there are  no more errors.

I know of no other language that facilitates this behavior. The editing
of the code is in a different environment from the execution of the
code. The state of the objects in the executing code disappears when the
code ceasing execution whether by choice or error. And yes, some of that
is and can be mitigated by files and databases. Live debugging of code
is by executing an application (debugger) on the application you want to
debug. This all by nature is static and dead. Only to discover problems
in my fixed in time code and objects. You can't fix the debugger while
debugger some program. I don't know that you necessarily fix the program
outside of discover of where to fix it in the separate editing process.
Then you start all over again.

Even if you can overcome ever single one of my perceived technical
difficulties. And I may be in error. I do not program in those
environments and have little experience there. One thing that is
indisputable at least to my understanding is this. Whatever language you
may overcome all of this in, you are quite possibly doing it alone or at
best in very small groups. There is no community or cultural support.

Pharo has a community, a culture which supports the artifact which
provides this live object environment. This is of tremendous value.
Pharo has a culture which supports live objects and live coding. It has
a culture that says your stupid code should not crash the environment.
If it does, we need to fix it.

Pharo is not perfect. Its community is not perfect. But I have not found
anything or place better.

So I am learning C, and dead programming hopefully sufficiently enough
that I can bridge the gap where Pharo does it well or where Pharo could
use a little outside help. But I still get to live in my nice live
object environment.

I have nothing against other languages. I would love it if there were
other languages which had this ability and culture. But I don't find
them anywhere. I have programmed in Python, Lua, played with PicoLisp.
Pharo is home. :)

And to top it all off, Stef's Pharo 7 and 8 roadmap looks pretty sweet.

I just wanted to say that Pharo (Smalltalk) is so much more than what I
perceive other languages understanding of live coding to be. And that I
know of no other language or community which approaches what is done by
nature in Pharo (Smalltalk).

All the above is my naive opinion. Do with it as you will.

Have a great day!


Jimmie