Smalltalk › Pharo › Pharo Smalltalk Users

Stop Thinking in Terms of Files

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

40 messages Options

Martin Bähr

Re: Stop Thinking in Terms of Files

Excerpts from Thierry Goubier's message of 2015-12-07 09:33:37 +0100:

> > > There are versions already for methods. So the functionality is there.
> > only if i save them in monticello though, and i can't just switch the
> > version of a single method, only the whole changeset at once. in git i can
> > limit browsing to a specific file, and then copy an old version of a file
> > into the current workspace. it is a bit crude, but it can be done. i don't
> > see that kind of functionality in pharo. the data is there, but the tools
> > to use the data in this specific manner are not.
> They are there: I implemented and demoed what you describe as a need over
> git a few years ago when I started GitFileTree [1].
> http://forum.world.st/Pharo-dev-Versions-Browser-over-gitfiletree-tp4695490.html

oh, i had no idea. that looks great. i hope i can find some time to look at
this soon.

> Then, probably over more than a year ago, Dale Henrich implemented a lot
> more in his environment (tode).
>
> Now, what we may conclude is that the need may not be that great.

well, part of the problem may be that things like these are hard to find.
worse, unless you know that such functionality exists, you would not think to look for it.
now i know, and now (well, soon :-) i will look!

then the next question is, how do we get more people to be aware of this.
adding it to pharo core would be one way, but probably a better solution would
be to further improve the catalog browser so that i can see categories and
drill those down. (i see that there are quite some improvements already in
pharo 5, so maybe someone is already working on that)

greetings, martin.

--
eKita - the online platform for your entire academic life
--
chief engineer eKita.co
pike programmer pike.lysator.liu.se caudium.net societyserver.org
secretary beijinglug.org
mentor fossasia.org
foresight developer foresightlinux.org realss.com
unix sysadmin
Martin Bähr working in china http://societyserver.org/mbaehr/

horrido

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

But these are implementation details...implementation of the base system. From the perspective of a programmer writing an application, none of this matters.

As I said earlier, the only reason why Smalltalk has to deal with files at all is because we live in a file-based culture. And the reason our culture is so entrenched with files is because we are too heavily invested in them, and we aren't going to budge. Files are about as low a storage abstraction as you can get, and they pre-date even Unix. Yes, files belong in the Stone Age!

kilon.alios wrote

That's the thing you can't take the argument further without diminishing
the value of you argument precisely for the fact that the vm is far closer
related to the image than it is to 0s and 1s. That tight relation is
fundamental to the behavior and existence of the image. It defines its
functionality, purpose and limitations.

The image itself is a file and the fact that it can store live state in a
binary format does not make it unique or any less of a file. In my case I
use blender files, they store the entire live state of the blender window
including images and even Python scripts. Similar examples are countless
out there.

So the answer to the question what makes the image file format unique is
simply.... Nothing
What's the advantage of using the image format compared to other files ?
None

On Mon, 7 Dec 2015 at 15:14, Ben Coman <[hidden email]> wrote:

> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <[hidden email]>
> wrote:
> > "A Smalltalk Image is your entire system. The Image includes all the
> tools
> > required to interact, customize and add functionality to your system, so
> > Smalltalk’s IDE is a very Integrated Development Environment."
> >
> >
> > Thats not the case even for someone like me that has been working with
> > smalltalk for only 2 years. The Image is not even the engine that drives
> > smalltalk . Thats the job of the VM that exists in a completely different
> > universe than smalltalk. It exists in the same universe than many other
> > languages do exists and thats the C universe, the universe of the OS.
> > Essentially what drives your system is not smalltalk is C. The
> diffirence is
> > that for a part of it that is high level enough, Slang is used, a Hybrid
> > language between C and Smalltalk that compiles to C. So while in the
> image
> > everything is , well almost everything, an object all the way down, in
> the
> > VM everything is C all the way down.
>
> To take that argument further, the VM is not even the thing driving
> the image ;). Essentially what drives it are the 1's and 0's of
> machine code. Further, what drives that are the electrons flowing
> through the chip. I think its fair to say that we *code* in Pharo
> without files. Files relate to Pharo only to the same extent that a
> database like Oracle or Postgres can be said to use files. That is,
> when you do SQL queries, are you *thinking* in terms of files, even
> though files are used by the server to store the data? Its just a
> matter of where you draw the line of abstraction.
>
> cheers -ben
>
> > Ironically an image misses the most important tool to even generate this
> C
> > code and thats the VMMaker that has to be installed separately. And of
> > course there are parts of the system that are coded in pure C, like some
> > core functionalities of the VM and of course plugins and external
> libraries
> > that the image has to rely on make things happen.
> >
> > Of course the image is still fairly powerful, you can change the syntax,
> > implement high level libraries, IDE tools and much more. But its not the
> > core of the system just another essential part of it.
> >
> >
> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <[hidden email]>
> > wrote:
> >>>
> >>>
> >>> well, i wouldn't need or even want it in memory, so on disk is fine.
> the
> >>> problem is more likely management of the same. browsing the changes is
> >>> not
> >>> really convenient. ideally i'd like to see versions in the
> class-browser
> >>> and
> >>> in the debugger, where on error i could then take a look at older
> >>> versions for
> >>> comparison, and switch to them to see if maybe the last change was the
> >>> cause of
> >>> the error.
> >>>
> >>> greetings, martin.
> >>>
> >>
> >> There are versions already for methods. So the functionality is there.
> >>
> >> I disagree however with you, I think that changes file was created for
> the
> >> precise scenarios of an image crash/ lockdown. In that case you may
> want to
> >> go back through the code and dont remember which method was triggered or
> >> what else was defined and created. In the case going chronologically
> which
> >> is how the changes file is already organised is far more useful than
> going
> >> method and class based.
> >>
> >> But I do agree it would be useful to extend the tools working with
> changes
> >> , but then none stop anyone from doing so and is not that hard to do.
>
>

kilon.alios

Re: Stop Thinking in Terms of Files

The devil is in the details ;)

It matters to me, I just came across the need to share data between multiple images. So I was pointed by the good people here to the Fuel library that , surprise surprise , it generates binary files that contain objects in their live state that helps you move and share code and data between images. Works well and I really like its design :)

We are not talking here about something sophisticated, we are talking here super basic functionality. Images sharing data and code. What we use ? Files. The image by itself has no functionality to even cover this super basic scenario because as a format is made to be self contained.

How you cant even care for such basic functionality ? Of course you will at some point. Its unavoidable.

The nice thing about files is that they have one very big advantage over the image. That is, specialization. When an app find a specific file , just by looking at its extension it immediately knows the structure of the data and the code that it may contain.

On other hand when you have an object system like the image is, such specifications go outside the window meaning you have to deal with the fact and trust that those that made those images have adhered to specific guidelines so you can make sure that your code wont run in front of some very nasty surprises.

But since the image itself allow you hack so deeply as the syntax of the language , you can't be sure how the data and code will be presented. Sure they will objects, but the format does not really matter so much as the structure itself.

In those cases files win hands down because they tend to be far more restricted on how they are structured. Not because there is anything special to these files, apart from the fact that their authors made sure to follow the specific structure to ensure compatibility with third party apps.

So not only Files are not on the Stone Age but they have evolved the level of specification to a whole new level that have made the foundation of our every day lives.

Sure you could probably replace files with a new way that is more Smalltalk friendly and still retain all the advantages of files and file system but , Smalltalk has not presented such solution to my knowledge. Hence we the smalltalkers we will still keep relying heavily on files for our every day needs until such solution is presented to us. Also with the huge wealth of file formats it would be a pain in the ass to replace them with a smalltalk solution.

In the mean time there are even more pressing matter that the image file has to attend to, which is far more stone age , to use your remark , than files. That is the ability to use full memory of the system and the ability to deal with large data without any large hits on performance. In short good support for 64 bit and big data.

"But these are implementation details...implementation of the base system.
/From the perspective of a programmer writing an application/, none of this
matters.

As I said earlier, the only reason why Smalltalk has to deal with files at
all is because we live in a file-based culture. And the reason our culture
is so entrenched with files is because we are too heavily invested in them,
and we aren't going to budge. *Files are about as low a storage abstraction
as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
Age!"

On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:

But these are implementation details...implementation of the base system.
/From the perspective of a programmer writing an application/, none of this
matters.

As I said earlier, the only reason why Smalltalk has to deal with files at
all is because we live in a file-based culture. And the reason our culture
is so entrenched with files is because we are too heavily invested in them,
and we aren't going to budge. *Files are about as low a storage abstraction
as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
Age!

kilon.alios wrote
> That's the thing you can't take the argument further without diminishing
> the value of you argument precisely for the fact that the vm is far closer
> related to the image than it is to 0s and 1s. That tight relation is
> fundamental to the behavior and existence of the image. It defines its
> functionality, purpose and limitations.
>
> The image itself is a file and the fact that it can store live state in a
> binary format does not make it unique or any less of a file. In my case I
> use blender files, they store the entire live state of the blender window
> including images and even Python scripts. Similar examples are countless
> out there.
>
> So the answer to the question what makes the image file format unique is
> simply.... Nothing
> What's the advantage of using the image format compared to other files ?
> None
>
> On Mon, 7 Dec 2015 at 15:14, Ben Coman <

> btc@

> > wrote:
>
>> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <

> kilon.alios@

> >
>> wrote:
>> > "A Smalltalk Image is your entire system. The Image includes all the
>> tools
>> > required to interact, customize and add functionality to your system,
>> so
>> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >
>> >
>> > Thats not the case even for someone like me that has been working with
>> > smalltalk for only 2 years. The Image is not even the engine that
>> drives
>> > smalltalk . Thats the job of the VM that exists in a completely
>> different
>> > universe than smalltalk. It exists in the same universe than many other
>> > languages do exists and thats the C universe, the universe of the OS.
>> > Essentially what drives your system is not smalltalk is C. The
>> diffirence is
>> > that for a part of it that is high level enough, Slang is used, a
>> Hybrid
>> > language between C and Smalltalk that compiles to C. So while in the
>> image
>> > everything is , well almost everything, an object all the way down, in
>> the
>> > VM everything is C all the way down.
>>
>> To take that argument further, the VM is not even the thing driving
>> the image ;). Essentially what drives it are the 1's and 0's of
>> machine code. Further, what drives that are the electrons flowing
>> through the chip. I think its fair to say that we *code* in Pharo
>> without files. Files relate to Pharo only to the same extent that a
>> database like Oracle or Postgres can be said to use files. That is,
>> when you do SQL queries, are you *thinking* in terms of files, even
>> though files are used by the server to store the data? Its just a
>> matter of where you draw the line of abstraction.
>>
>> cheers -ben
>>
>> > Ironically an image misses the most important tool to even generate
>> this
>> C
>> > code and thats the VMMaker that has to be installed separately. And of
>> > course there are parts of the system that are coded in pure C, like
>> some
>> > core functionalities of the VM and of course plugins and external
>> libraries
>> > that the image has to rely on make things happen.
>> >
>> > Of course the image is still fairly powerful, you can change the
>> syntax,
>> > implement high level libraries, IDE tools and much more. But its not
>> the
>> > core of the system just another essential part of it.
>> >
>> >
>> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <

> kilon.alios@

> >
>> > wrote:
>> >>>
>> >>>
>> >>> well, i wouldn't need or even want it in memory, so on disk is fine.
>> the
>> >>> problem is more likely management of the same. browsing the changes
>> is
>> >>> not
>> >>> really convenient. ideally i'd like to see versions in the
>> class-browser
>> >>> and
>> >>> in the debugger, where on error i could then take a look at older
>> >>> versions for
>> >>> comparison, and switch to them to see if maybe the last change was
>> the
>> >>> cause of
>> >>> the error.
>> >>>
>> >>> greetings, martin.
>> >>>
>> >>
>> >> There are versions already for methods. So the functionality is there.
>> >>
>> >> I disagree however with you, I think that changes file was created for
>> the
>> >> precise scenarios of an image crash/ lockdown. In that case you may
>> want to
>> >> go back through the code and dont remember which method was triggered
>> or
>> >> what else was defined and created. In the case going chronologically
>> which
>> >> is how the changes file is already organised is far more useful than
>> going
>> >> method and class based.
>> >>
>> >> But I do agree it would be useful to extend the tools working with
>> changes
>> >> , but then none stop anyone from doing so and is not that hard to do.
>>
>>

--
View this message in context: http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

horrido

Re: Stop Thinking in Terms of Files

You make a convincing argument. Files are useful.

In modern circumstances, Smalltalk has to coexist with a file-based world. As Joachim wrote earlier, we are well-equipped to deal with files, but we do not think in terms of files when we code our applications. We are not obliged to use a file-based toolchain. The point of my article was to persuade other developers that letting their obsession with file-based tools prevent them from adopting Smalltalk is short-sighted and counter-productive. Files have their uses, but they should look beyond files for other software creation possibilities. Smalltalk has much to offer.

And, yes, it would be very good to have 64-bit support in Smalltalk.

kilon.alios wrote

The devil is in the details ;)

It matters to me, I just came across the need to share data between
multiple images. So I was pointed by the good people here to the Fuel
library that , surprise surprise , it generates binary files that contain
objects in their live state that helps you move and share code and data
between images. Works well and I really like its design :)

We are not talking here about something sophisticated, we are talking here
super basic functionality. Images sharing data and code. What we use ?
Files. The image by itself has no functionality to even cover this super
basic scenario because as a format is made to be self contained.

How you cant even care for such basic functionality ? Of course you will at
some point. Its unavoidable.

The nice thing about files is that they have one very big advantage over
the image. That is, specialization. When an app find a specific file , just
by looking at its extension it immediately knows the structure of the data
and the code that it may contain.

On other hand when you have an object system like the image is, such
specifications go outside the window meaning you have to deal with the fact
and trust that those that made those images have adhered to specific
guidelines so you can make sure that your code wont run in front of some
very nasty surprises.

But since the image itself allow you hack so deeply as the syntax of the
language , you can't be sure how the data and code will be presented. Sure
they will objects, but the format does not really matter so much as the
structure itself.

In those cases files win hands down because they tend to be far more
restricted on how they are structured. Not because there is anything
special to these files, apart from the fact that their authors made sure to
follow the specific structure to ensure compatibility with third party
apps.

So not only Files are not on the Stone Age but they have evolved the level
of specification to a whole new level that have made the foundation of our
every day lives.

Sure you could probably replace files with a new way that is more Smalltalk
friendly and still retain all the advantages of files and file system but ,
Smalltalk has not presented such solution to my knowledge. Hence we the
smalltalkers we will still keep relying heavily on files for our every day
needs until such solution is presented to us. Also with the huge wealth of
file formats it would be a pain in the ass to replace them with a smalltalk
solution.

In the mean time there are even more pressing matter that the image file
has to attend to, which is far more stone age , to use your remark , than
files. That is the ability to use full memory of the system and the ability
to deal with large data without any large hits on performance. In short
good support for 64 bit and big data.

"But these are implementation details...implementation of the base system.
/From the perspective of a programmer writing an application/, none of this
matters.

As I said earlier, the only reason why Smalltalk has to deal with files at
all is because we live in a file-based culture. And the reason our culture
is so entrenched with files is because we are too heavily invested in them,
and we aren't going to budge. *Files are about as low a storage abstraction
as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
Age!"

On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:

> But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in them,
> and we aren't going to budge. *Files are about as low a storage abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the
> Stone
> Age!
>
>
>
> kilon.alios wrote
> > That's the thing you can't take the argument further without diminishing
> > the value of you argument precisely for the fact that the vm is far
> closer
> > related to the image than it is to 0s and 1s. That tight relation is
> > fundamental to the behavior and existence of the image. It defines its
> > functionality, purpose and limitations.
> >
> > The image itself is a file and the fact that it can store live state in a
> > binary format does not make it unique or any less of a file. In my case I
> > use blender files, they store the entire live state of the blender window
> > including images and even Python scripts. Similar examples are countless
> > out there.
> >
> > So the answer to the question what makes the image file format unique is
> > simply.... Nothing
> > What's the advantage of using the image format compared to other files ?
> > None
> >
> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>
> > btc@
>
> > > wrote:
> >
> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>
> > kilon.alios@
>
> > >
> >> wrote:
> >> > "A Smalltalk Image is your entire system. The Image includes all the
> >> tools
> >> > required to interact, customize and add functionality to your system,
> >> so
> >> > Smalltalk’s IDE is a very Integrated Development Environment."
> >> >
> >> >
> >> > Thats not the case even for someone like me that has been working with
> >> > smalltalk for only 2 years. The Image is not even the engine that
> >> drives
> >> > smalltalk . Thats the job of the VM that exists in a completely
> >> different
> >> > universe than smalltalk. It exists in the same universe than many
> other
> >> > languages do exists and thats the C universe, the universe of the OS.
> >> > Essentially what drives your system is not smalltalk is C. The
> >> diffirence is
> >> > that for a part of it that is high level enough, Slang is used, a
> >> Hybrid
> >> > language between C and Smalltalk that compiles to C. So while in the
> >> image
> >> > everything is , well almost everything, an object all the way down, in
> >> the
> >> > VM everything is C all the way down.
> >>
> >> To take that argument further, the VM is not even the thing driving
> >> the image ;). Essentially what drives it are the 1's and 0's of
> >> machine code. Further, what drives that are the electrons flowing
> >> through the chip. I think its fair to say that we *code* in Pharo
> >> without files. Files relate to Pharo only to the same extent that a
> >> database like Oracle or Postgres can be said to use files. That is,
> >> when you do SQL queries, are you *thinking* in terms of files, even
> >> though files are used by the server to store the data? Its just a
> >> matter of where you draw the line of abstraction.
> >>
> >> cheers -ben
> >>
> >> > Ironically an image misses the most important tool to even generate
> >> this
> >> C
> >> > code and thats the VMMaker that has to be installed separately. And of
> >> > course there are parts of the system that are coded in pure C, like
> >> some
> >> > core functionalities of the VM and of course plugins and external
> >> libraries
> >> > that the image has to rely on make things happen.
> >> >
> >> > Of course the image is still fairly powerful, you can change the
> >> syntax,
> >> > implement high level libraries, IDE tools and much more. But its not
> >> the
> >> > core of the system just another essential part of it.
> >> >
> >> >
> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>
> > kilon.alios@
>
> > >
> >> > wrote:
> >> >>>
> >> >>>
> >> >>> well, i wouldn't need or even want it in memory, so on disk is fine.
> >> the
> >> >>> problem is more likely management of the same. browsing the changes
> >> is
> >> >>> not
> >> >>> really convenient. ideally i'd like to see versions in the
> >> class-browser
> >> >>> and
> >> >>> in the debugger, where on error i could then take a look at older
> >> >>> versions for
> >> >>> comparison, and switch to them to see if maybe the last change was
> >> the
> >> >>> cause of
> >> >>> the error.
> >> >>>
> >> >>> greetings, martin.
> >> >>>
> >> >>
> >> >> There are versions already for methods. So the functionality is
> there.
> >> >>
> >> >> I disagree however with you, I think that changes file was created
> for
> >> the
> >> >> precise scenarios of an image crash/ lockdown. In that case you may
> >> want to
> >> >> go back through the code and dont remember which method was triggered
> >> or
> >> >> what else was defined and created. In the case going chronologically
> >> which
> >> >> is how the changes file is already organised is far more useful than
> >> going
> >> >> method and class based.
> >> >>
> >> >> But I do agree it would be useful to extend the tools working with
> >> changes
> >> >> , but then none stop anyone from doing so and is not that hard to do.
> >>
> >>
>
>
>
>
>
> --
> View this message in context:
> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>
>

EuanM

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

>An image IS a file !!!

An image *can be stored to* a file. The image is not the file.

Ceci n'est pas une pipe.

The reason it is image-based is that the image is the primary
representation. The copy of the image stored in a file is simply a
persisting back-up of the image.

Unlike file based systems like, say an SQL d/b. There, the file is
the primary representation, and what is in memory is (generally) both
transient and partial. (i.e. we load in things from disk, and write
things out to disk, and the disk is the canonical representation of
the state.

With the image based system, the file-copy of the image is transient
and often partial. The 'image' of the system (i.e. the current state
of system and contents that is held in memory (including anything
paged out to store in virtual memory) is the primary representation,
and the canonical representation of state.

(Although it could be that the changes file is kept in lock-step
automagically - I've never looked into what the changes file is
actually doing at a moment-to--moment level, at which point the
changes file would be partial and capable of generating or providing a
canonical representation of state ).

On 6 December 2015 at 19:58, Dimitris Chloupis <[hidden email]> wrote:

> "Files belong in the Stone Age"
>
> No they do not !
>
> "Smalltalk is an image-based programming language."
>
> An image IS a file !!!
>
> "An image is essentially a self-contained operating system that manages all
> the code for you, thanks to an easy-to-use IDE"
>
> no its not!!! the vm is the virtual OS , the image is the OS libraries. The
> VM also is a binary file separate from image and comes with a lot more files
> , plugins and external libraries.
>
> Look man you do more harm with these articles than you do good. This
> Smalltalk hype is the worst of its kind and completely misses the point of
> why Smalltalk is great.
>
> It would even make zero diffirence if we were to break the image file down
> to much smaller files, it would still be a live coding enviroment. Actually
> you dont even need those files to be even binary , text source code files
> can still store live state and be all about objects.
>
> And on top of that there are people there asking you the easiest questions
> and you cannot even answer them while at the same time you proclaim the end
> of File's "Stone age".
>
> And on top of that you post this in amber-lang which is not even relevant in
> any way with your article (since its not image based) just to get more
> attention.
>
>
>
>
>
>
>
> On Sun, Dec 6, 2015 at 8:40 PM Sven Van Caekenberghe <[hidden email]> wrote:
>>
>> There are a lot of questions in there, too much to answer.
>>
>> The basic answer is that everything is modelled as objects, even the
>> objects themselves. There is a complete meta model of what could be
>> considered code and even the execution of code. Furthermore, even versioning
>> is modelled nicely too. Of course, Pharo interfaces with the outside world,
>> it can read and write files, it can do networking. The object world and its
>> code can, in various ways, be mapped to concepts of this world. For example,
>> https://github.com/pharo-project/pharo-core contains (a copy, one
>> representation) of all code in Pharo. There is also the option to save the
>> whole object world to a single snapshot file, called an image. Running on
>> servers, containers, embedded devices is all possible, in multiple ways,
>> with or without a (G)UI. At that point Pharo consists of a couple of files,
>> most notably the image file.
>>
>> HTH,
>>
>> Sven
>>
>> > On 06 Dec 2015, at 17:58, horrido <[hidden email]> wrote:
>> >
>> > I submitted an article at Reddit called "Stop Thinking in Terms of
>> > Files."
>> > Some guy with the handle "audioen" wrote the following comment:
>> >
>> > We have heard that smalltalk appears to use model similar to a LISP
>> > machine
>> > of yore in that the programming environment = the OS = the runtime
>> > environment. Once you define a function, it simply exists without being
>> > written to a file or compiled into some process that runs it. You can
>> > just
>> > call it, or undefine it and it ceases to be. From this point of view, it
>> > is
>> > probably perfectly valid to say that it has no files, because it doesn't
>> > need them.
>> >
>> > On the other hand, let's assume that your smalltalk image got a little
>> > bit
>> > corrupted so that some packages/functions/whatever are now missing or
>> > not
>> > functioning. Or, let's say that you accidentally undefined a function
>> > and
>> > that was a mistake and now you really want to get it back. How would you
>> > do
>> > that? The file-based answer is that you hopefully had backups of the
>> > files
>> > that held definition of that function. What passes for a backup in
>> > smalltalk
>> > land?
>> >
>> > And how do you deal with version control? How do you recover from
>> > mistakes?
>> > If you wanted to share your crap to someone else collaboratively through
>> > e.g. github, how would you do that? You'd have to export your functions
>> > into
>> > individual files, probably, and packages as directories into git.
>> > Someone
>> > would check them out, eval them, make changes, and commit updated
>> > functions.
>> > How does this kind of process look like, in a non-file paradigm, if it
>> > is
>> > done at all? (Does smalltalk VM even support networking?)
>> >
>> > In general how do you even dump the state of the VM in some way that you
>> > can
>> > show someone what exactly your project is made of, in textual form? It's
>> > not
>> > very nice to dump an entire image and tell them to just run that. I bet
>> > the
>> > image is much larger and contains historical stuff that you no longer
>> > care
>> > about. What if you really just wanted to publish a recipe that can
>> > construct
>> > something equivalent of that particular image? What does "docker" for
>> > smalltalk look like?
>> >
>> > If you tell us files suck, tell us also how you solve the same problems
>> > that
>> > we solve through files. Especially that collaborative programming
>> > through
>> > github use case interests me a little.
>> > -----
>> >
>> > I must confess, I'm not fully qualified to answer this comment, /at
>> > least,
>> > not optimally/. Perhaps some of you can go to the Reddit link
>> >
>> > <https://www.reddit.com/r/programming/comments/3vjlta/stop_thinking_in_terms_of_files/>
>> > and respond? Thanks.
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> > http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614.html
>> > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>> >
>>
>>
>

EuanM

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

> Look man you do more harm with these articles than you do good.

No. Just... no.

> This Smalltalk hype is the worst of its kind and completely misses the point of why Smalltalk is great.

Speaking of "hype(rbole)"...

On 6 December 2015 at 19:58, Dimitris Chloupis <[hidden email]> wrote:

kilon.alios

Re: Stop Thinking in Terms of Files

In reply to this post by horrido

And this "victory of dead code and date" that files usually promote can further benefit Pharo .

For example lets take one of the main problems of the Pharo image, its size. There is already an effort to modularilise the image to make it small and everything else installable and unistallable.

But why not go a step further and have dead zone files. So you end up with a tiny image containing just the basics (few kbs if possible) even no GUI and IDE tools and instead you have a collection of files that can be read by the image , each file contains a diffirent project, you can view their code and their data but not install them in the image to do so. And only when you decide that "sure I could use this project" then you instruct the image to import the file and make the code and the data live and part of the image. This way the pharo system can keep growing without growing the image at all and let the users decide how much they want to grow the image themselves.

Technically speaking this functionality is already available through monticello , where monticello allow you to view a local or remote repository of code without loading the code to the image (via the "open button"). So its not even a new concept. Though in my mind I imagine all this happening from inside the System Browser.

Thats just one scenario how dead data and code can greatly benefit the Pharo system. I can keep brainstorming forever for the usefulness of files for Pharo.

On Mon, Dec 7, 2015 at 9:20 PM horrido <[hidden email]> wrote:

You make a convincing argument. Files are useful.

In modern circumstances, Smalltalk has to coexist with a file-based world.
As Joachim wrote earlier, we are well-equipped to deal with files, but we do
not think in terms of files when we code our applications. We are not
obliged to use a file-based toolchain. The point of my article was to
persuade other developers that letting their obsession with file-based tools
prevent them from adopting Smalltalk is short-sighted and
counter-productive. Files have their uses, but they should look beyond files
for other software creation possibilities. Smalltalk has much to offer.

And, yes, it would be very good to have 64-bit support in Smalltalk.

kilon.alios wrote
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between
> multiple images. So I was pointed by the good people here to the Fuel
> library that , surprise surprise , it generates binary files that contain
> objects in their live state that helps you move and share code and data
> between images. Works well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will
> at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over
> the image. That is, specialization. When an app find a specific file ,
> just
> by looking at its extension it immediately knows the structure of the data
> and the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the
> fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything
> special to these files, apart from the fact that their authors made sure
> to
> follow the specific structure to ensure compatibility with third party
> apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more
> Smalltalk
> friendly and still retain all the advantages of files and file system but
> ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a
> smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file
> has to attend to, which is far more stone age , to use your remark , than
> files. That is the ability to use full memory of the system and the
> ability
> to deal with large data without any large hits on performance. In short
> good support for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of
> this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in
> them,
> and we aren't going to budge. *Files are about as low a storage
> abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the
> Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <

> horrido.hobbies@

> > wrote:
>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files
>> at
>> all is because we live in a file-based culture. And the reason our
>> culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without
>> diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> a
>> > binary format does not make it unique or any less of a file. In my case
>> I
>> > use blender files, they store the entire live state of the blender
>> window
>> > including images and even Python scripts. Similar examples are
>> countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique
>> is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files
>> ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your
>> system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> other
>> >> > languages do exists and thats the C universe, the universe of the
>> OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the
>> changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> triggered
>> >> or
>> >> >> what else was defined and created. In the case going
>> chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful
>> than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>>

--
View this message in context: http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865872.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Ben Coman

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

Just one simple question...
When you want Pharo to compile some code, which *file* do you pass to
our Opal compiler?

cheers -ben

On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]> wrote:

> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between multiple
> images. So I was pointed by the good people here to the Fuel library that ,
> surprise surprise , it generates binary files that contain objects in their
> live state that helps you move and share code and data between images. Works
> well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over the
> image. That is, specialization. When an app find a specific file , just by
> looking at its extension it immediately knows the structure of the data and
> the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything special
> to these files, apart from the fact that their authors made sure to follow
> the specific structure to ensure compatibility with third party apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more Smalltalk
> friendly and still retain all the advantages of files and file system but ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file has
> to attend to, which is far more stone age , to use your remark , than files.
> That is the ability to use full memory of the system and the ability to deal
> with large data without any large hits on performance. In short good support
> for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in them,
> and we aren't going to budge. *Files are about as low a storage abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:
>>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files at
>> all is because we live in a file-based culture. And the reason our culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> > closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> > a
>> > binary format does not make it unique or any less of a file. In my case
>> > I
>> > use blender files, they store the entire live state of the blender
>> > window
>> > including images and even Python scripts. Similar examples are countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> >> > with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> >> > other
>> >> > languages do exists and thats the C universe, the universe of the OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> >> > in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> >> > of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> >> >> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> >> >> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> >> >> triggered
>> >> or
>> >> >> what else was defined and created. In the case going chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>

philippeback

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

Image size? What a joke when one faces "Uber Jars"...

http://fiji.sc/Uber-JAR

On Mon, Dec 7, 2015 at 11:08 PM, Dimitris Chloupis <[hidden email]> wrote:

And this "victory of dead code and date" that files usually promote can further benefit Pharo .

For example lets take one of the main problems of the Pharo image, its size. There is already an effort to modularilise the image to make it small and everything else installable and unistallable.

But why not go a step further and have dead zone files. So you end up with a tiny image containing just the basics (few kbs if possible) even no GUI and IDE tools and instead you have a collection of files that can be read by the image , each file contains a diffirent project, you can view their code and their data but not install them in the image to do so. And only when you decide that "sure I could use this project" then you instruct the image to import the file and make the code and the data live and part of the image. This way the pharo system can keep growing without growing the image at all and let the users decide how much they want to grow the image themselves.

Technically speaking this functionality is already available through monticello , where monticello allow you to view a local or remote repository of code without loading the code to the image (via the "open button"). So its not even a new concept. Though in my mind I imagine all this happening from inside the System Browser.

Thats just one scenario how dead data and code can greatly benefit the Pharo system. I can keep brainstorming forever for the usefulness of files for Pharo.

On Mon, Dec 7, 2015 at 9:20 PM horrido <[hidden email]> wrote:
You make a convincing argument. Files are useful.

In modern circumstances, Smalltalk has to coexist with a file-based world.
As Joachim wrote earlier, we are well-equipped to deal with files, but we do
not think in terms of files when we code our applications. We are not
obliged to use a file-based toolchain. The point of my article was to
persuade other developers that letting their obsession with file-based tools
prevent them from adopting Smalltalk is short-sighted and
counter-productive. Files have their uses, but they should look beyond files
for other software creation possibilities. Smalltalk has much to offer.

And, yes, it would be very good to have 64-bit support in Smalltalk.

kilon.alios wrote
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between
> multiple images. So I was pointed by the good people here to the Fuel
> library that , surprise surprise , it generates binary files that contain
> objects in their live state that helps you move and share code and data
> between images. Works well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will
> at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over
> the image. That is, specialization. When an app find a specific file ,
> just
> by looking at its extension it immediately knows the structure of the data
> and the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the
> fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything
> special to these files, apart from the fact that their authors made sure
> to
> follow the specific structure to ensure compatibility with third party
> apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more
> Smalltalk
> friendly and still retain all the advantages of files and file system but
> ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a
> smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file
> has to attend to, which is far more stone age , to use your remark , than
> files. That is the ability to use full memory of the system and the
> ability
> to deal with large data without any large hits on performance. In short
> good support for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of
> this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in
> them,
> and we aren't going to budge. *Files are about as low a storage
> abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the
> Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <

> horrido.hobbies@

> > wrote:
>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files
>> at
>> all is because we live in a file-based culture. And the reason our
>> culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without
>> diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> a
>> > binary format does not make it unique or any less of a file. In my case
>> I
>> > use blender files, they store the entire live state of the blender
>> window
>> > including images and even Python scripts. Similar examples are
>> countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique
>> is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files
>> ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your
>> system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> other
>> >> > languages do exists and thats the C universe, the universe of the
>> OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the
>> changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> triggered
>> >> or
>> >> >> what else was defined and created. In the case going
>> chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful
>> than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>>

--
View this message in context: http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865872.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

blake watson

Re: Stop Thinking in Terms of Files

This is undeniably true. UberJars and UberWars are yooooge.

On Mon, Dec 7, 2015 at 3:29 PM, [hidden email] <[hidden email]> wrote:

Image size? What a joke when one faces "Uber Jars"...

http://fiji.sc/Uber-JAR

On Mon, Dec 7, 2015 at 11:08 PM, Dimitris Chloupis <[hidden email]> wrote:
And this "victory of dead code and date" that files usually promote can further benefit Pharo .

For example lets take one of the main problems of the Pharo image, its size. There is already an effort to modularilise the image to make it small and everything else installable and unistallable.

But why not go a step further and have dead zone files. So you end up with a tiny image containing just the basics (few kbs if possible) even no GUI and IDE tools and instead you have a collection of files that can be read by the image , each file contains a diffirent project, you can view their code and their data but not install them in the image to do so. And only when you decide that "sure I could use this project" then you instruct the image to import the file and make the code and the data live and part of the image. This way the pharo system can keep growing without growing the image at all and let the users decide how much they want to grow the image themselves.

Technically speaking this functionality is already available through monticello , where monticello allow you to view a local or remote repository of code without loading the code to the image (via the "open button"). So its not even a new concept. Though in my mind I imagine all this happening from inside the System Browser.

Thats just one scenario how dead data and code can greatly benefit the Pharo system. I can keep brainstorming forever for the usefulness of files for Pharo.

On Mon, Dec 7, 2015 at 9:20 PM horrido <[hidden email]> wrote:
You make a convincing argument. Files are useful.

In modern circumstances, Smalltalk has to coexist with a file-based world.
As Joachim wrote earlier, we are well-equipped to deal with files, but we do
not think in terms of files when we code our applications. We are not
obliged to use a file-based toolchain. The point of my article was to
persuade other developers that letting their obsession with file-based tools
prevent them from adopting Smalltalk is short-sighted and
counter-productive. Files have their uses, but they should look beyond files
for other software creation possibilities. Smalltalk has much to offer.

And, yes, it would be very good to have 64-bit support in Smalltalk.

kilon.alios wrote
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between
> multiple images. So I was pointed by the good people here to the Fuel
> library that , surprise surprise , it generates binary files that contain
> objects in their live state that helps you move and share code and data
> between images. Works well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will
> at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over
> the image. That is, specialization. When an app find a specific file ,
> just
> by looking at its extension it immediately knows the structure of the data
> and the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the
> fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything
> special to these files, apart from the fact that their authors made sure
> to
> follow the specific structure to ensure compatibility with third party
> apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more
> Smalltalk
> friendly and still retain all the advantages of files and file system but
> ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a
> smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file
> has to attend to, which is far more stone age , to use your remark , than
> files. That is the ability to use full memory of the system and the
> ability
> to deal with large data without any large hits on performance. In short
> good support for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of
> this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in
> them,
> and we aren't going to budge. *Files are about as low a storage
> abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the
> Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <

> horrido.hobbies@

> > wrote:
>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files
>> at
>> all is because we live in a file-based culture. And the reason our
>> culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without
>> diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> a
>> > binary format does not make it unique or any less of a file. In my case
>> I
>> > use blender files, they store the entire live state of the blender
>> window
>> > including images and even Python scripts. Similar examples are
>> countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique
>> is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files
>> ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your
>> system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> other
>> >> > languages do exists and thats the C universe, the universe of the
>> OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the
>> changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> triggered
>> >> or
>> >> >> what else was defined and created. In the case going
>> chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful
>> than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>>

--
View this message in context: http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865872.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

kilon.alios

Re: Stop Thinking in Terms of Files

In reply to this post by Ben Coman

You could not ask the worst case person even if you wanted :D

But here we go ...

Will depend , let take my ChronosManager for example

To modify the existing code of ChronosManager , is just text that gets compiled to bytecode immediately on each accept, no need to be loaded from a file, even though it gets added to the source and changes files.

To add code to the image whether its the entire code of ChronosManager or just a commit, it will load the code from st files, plus the extra filetree metadata files for class definition, version control and method categorization. I use Github but its a very similar story for those using Smalltalkhub.

To add code that contains data like an image, like I do with the GUI resources , I use a large collection of PNG files which with the help of the IconFactory will compile the binary data to a bytecode string and also pass it as a form to be used immediately. I am thinking instead to replace the iconfactory with direct access to the png files and creation of the form or a need to exist basis. Lazy loading in short.

In the future that I will need a common ground for all the images that use ChronoManager to share data and live state , I will be using fuel files.

For Ephestos (ChronosManager is part of the Ephestos project) that has to deal with 3d data, the collection of files grow exponentially which I try to keep outside the image because that data can easily reach several GBs and I dont want the image to explode in my face. To give you an example one folder I have with several blender files is around 2 gbs tops, each file being more than 250 MB. Blend files by the way implement their own kind of "object orientation" even though Blender is largely coded in C.

So if you ask me how many files I have to pass to the opal compiler if all I wanted is to stick just with the image my answer would be "a ton" but its a moot a point since the image cant handle so much data anyway. But even in the current scenario, its still a lot of files if you count the pngs and the repository .

I actually now try to remove my dependency on SmaCC because it makes the load time very slow, apparently compilation of large projects in the image is super slow and I will divert the parsing back to the python side which is way faster and since its python syntax i am parsing, python is really good at it anyway and helps me remove one unnecessary dependency. Though I have to say I am very happy with SmaCC and its great to have it around.

In the future I predict that I will have to keep more and more things into files to make loading of Ephestos fast and also its execution because image is not really optimized for what I want it to use it for.

So in my case, its files all the way !

On Tue, Dec 8, 2015 at 1:09 AM Ben Coman <[hidden email]> wrote:

Just one simple question...
When you want Pharo to compile some code, which *file* do you pass to
our Opal compiler?

cheers -ben

On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]> wrote:
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between multiple
> images. So I was pointed by the good people here to the Fuel library that ,
> surprise surprise , it generates binary files that contain objects in their
> live state that helps you move and share code and data between images. Works
> well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over the
> image. That is, specialization. When an app find a specific file , just by
> looking at its extension it immediately knows the structure of the data and
> the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything special
> to these files, apart from the fact that their authors made sure to follow
> the specific structure to ensure compatibility with third party apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more Smalltalk
> friendly and still retain all the advantages of files and file system but ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file has
> to attend to, which is far more stone age , to use your remark , than files.
> That is the ability to use full memory of the system and the ability to deal
> with large data without any large hits on performance. In short good support
> for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in them,
> and we aren't going to budge. *Files are about as low a storage abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:
>>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files at
>> all is because we live in a file-based culture. And the reason our culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> > closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> > a
>> > binary format does not make it unique or any less of a file. In my case
>> > I
>> > use blender files, they store the entire live state of the blender
>> > window
>> > including images and even Python scripts. Similar examples are countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> >> > with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> >> > other
>> >> > languages do exists and thats the C universe, the universe of the OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> >> > in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> >> > of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> >> >> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> >> >> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> >> >> triggered
>> >> or
>> >> >> what else was defined and created. In the case going chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>

Ben Coman

Re: Stop Thinking in Terms of Files

Fair enough ;p
Now I'll get back doing something productive
cheers -ben

On Tue, Dec 8, 2015 at 8:06 AM, Dimitris Chloupis <[hidden email]> wrote:

> You could not ask the worst case person even if you wanted :D
>
> But here we go ...
>
> Will depend , let take my ChronosManager for example
>
> To modify the existing code of ChronosManager , is just text that gets
> compiled to bytecode immediately on each accept, no need to be loaded from a
> file, even though it gets added to the source and changes files.
>
> To add code to the image whether its the entire code of ChronosManager or
> just a commit, it will load the code from st files, plus the extra filetree
> metadata files for class definition, version control and method
> categorization. I use Github but its a very similar story for those using
> Smalltalkhub.
>
> To add code that contains data like an image, like I do with the GUI
> resources , I use a large collection of PNG files which with the help of the
> IconFactory will compile the binary data to a bytecode string and also pass
> it as a form to be used immediately. I am thinking instead to replace the
> iconfactory with direct access to the png files and creation of the form or
> a need to exist basis. Lazy loading in short.
>
> In the future that I will need a common ground for all the images that use
> ChronoManager to share data and live state , I will be using fuel files.
>
> For Ephestos (ChronosManager is part of the Ephestos project) that has to
> deal with 3d data, the collection of files grow exponentially which I try to
> keep outside the image because that data can easily reach several GBs and I
> dont want the image to explode in my face. To give you an example one folder
> I have with several blender files is around 2 gbs tops, each file being more
> than 250 MB. Blend files by the way implement their own kind of "object
> orientation" even though Blender is largely coded in C.
>
> So if you ask me how many files I have to pass to the opal compiler if all I
> wanted is to stick just with the image my answer would be "a ton" but its a
> moot a point since the image cant handle so much data anyway. But even in
> the current scenario, its still a lot of files if you count the pngs and the
> repository .
>
> I actually now try to remove my dependency on SmaCC because it makes the
> load time very slow, apparently compilation of large projects in the image
> is super slow and I will divert the parsing back to the python side which is
> way faster and since its python syntax i am parsing, python is really good
> at it anyway and helps me remove one unnecessary dependency. Though I have
> to say I am very happy with SmaCC and its great to have it around.
>
> In the future I predict that I will have to keep more and more things into
> files to make loading of Ephestos fast and also its execution because image
> is not really optimized for what I want it to use it for.
>
> So in my case, its files all the way !
>
>
> On Tue, Dec 8, 2015 at 1:09 AM Ben Coman <[hidden email]> wrote:
>>
>> Just one simple question...
>> When you want Pharo to compile some code, which *file* do you pass to
>> our Opal compiler?
>>
>> cheers -ben
>>
>> On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]>
>> wrote:
>> > The devil is in the details ;)
>> >
>> > It matters to me, I just came across the need to share data between
>> > multiple
>> > images. So I was pointed by the good people here to the Fuel library
>> > that ,
>> > surprise surprise , it generates binary files that contain objects in
>> > their
>> > live state that helps you move and share code and data between images.
>> > Works
>> > well and I really like its design :)
>> >
>> > We are not talking here about something sophisticated, we are talking
>> > here
>> > super basic functionality. Images sharing data and code. What we use ?
>> > Files. The image by itself has no functionality to even cover this super
>> > basic scenario because as a format is made to be self contained.
>> >
>> > How you cant even care for such basic functionality ? Of course you will
>> > at
>> > some point. Its unavoidable.
>> >
>> > The nice thing about files is that they have one very big advantage over
>> > the
>> > image. That is, specialization. When an app find a specific file , just
>> > by
>> > looking at its extension it immediately knows the structure of the data
>> > and
>> > the code that it may contain.
>> >
>> > On other hand when you have an object system like the image is, such
>> > specifications go outside the window meaning you have to deal with the
>> > fact
>> > and trust that those that made those images have adhered to specific
>> > guidelines so you can make sure that your code wont run in front of some
>> > very nasty surprises.
>> >
>> > But since the image itself allow you hack so deeply as the syntax of the
>> > language , you can't be sure how the data and code will be presented.
>> > Sure
>> > they will objects, but the format does not really matter so much as the
>> > structure itself.
>> >
>> > In those cases files win hands down because they tend to be far more
>> > restricted on how they are structured. Not because there is anything
>> > special
>> > to these files, apart from the fact that their authors made sure to
>> > follow
>> > the specific structure to ensure compatibility with third party apps.
>> >
>> > So not only Files are not on the Stone Age but they have evolved the
>> > level
>> > of specification to a whole new level that have made the foundation of
>> > our
>> > every day lives.
>> >
>> > Sure you could probably replace files with a new way that is more
>> > Smalltalk
>> > friendly and still retain all the advantages of files and file system
>> > but ,
>> > Smalltalk has not presented such solution to my knowledge. Hence we the
>> > smalltalkers we will still keep relying heavily on files for our every
>> > day
>> > needs until such solution is presented to us. Also with the huge wealth
>> > of
>> > file formats it would be a pain in the ass to replace them with a
>> > smalltalk
>> > solution.
>> >
>> > In the mean time there are even more pressing matter that the image file
>> > has
>> > to attend to, which is far more stone age , to use your remark , than
>> > files.
>> > That is the ability to use full memory of the system and the ability to
>> > deal
>> > with large data without any large hits on performance. In short good
>> > support
>> > for 64 bit and big data.
>> >
>> >
>> > "But these are implementation details...implementation of the base
>> > system.
>> > /From the perspective of a programmer writing an application/, none of
>> > this
>> > matters.
>> >
>> > As I said earlier, the only reason why Smalltalk has to deal with files
>> > at
>> > all is because we live in a file-based culture. And the reason our
>> > culture
>> > is so entrenched with files is because we are too heavily invested in
>> > them,
>> > and we aren't going to budge. *Files are about as low a storage
>> > abstraction
>> > as you can get*, and they pre-date even Unix. Yes, files belong in the
>> > Stone
>> > Age!"
>> >
>> > On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]>
>> > wrote:
>> >>
>> >> But these are implementation details...implementation of the base
>> >> system.
>> >> /From the perspective of a programmer writing an application/, none of
>> >> this
>> >> matters.
>> >>
>> >> As I said earlier, the only reason why Smalltalk has to deal with files
>> >> at
>> >> all is because we live in a file-based culture. And the reason our
>> >> culture
>> >> is so entrenched with files is because we are too heavily invested in
>> >> them,
>> >> and we aren't going to budge. *Files are about as low a storage
>> >> abstraction
>> >> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> >> Stone
>> >> Age!
>> >>
>> >>
>> >>
>> >> kilon.alios wrote
>> >> > That's the thing you can't take the argument further without
>> >> > diminishing
>> >> > the value of you argument precisely for the fact that the vm is far
>> >> > closer
>> >> > related to the image than it is to 0s and 1s. That tight relation is
>> >> > fundamental to the behavior and existence of the image. It defines
>> >> > its
>> >> > functionality, purpose and limitations.
>> >> >
>> >> > The image itself is a file and the fact that it can store live state
>> >> > in
>> >> > a
>> >> > binary format does not make it unique or any less of a file. In my
>> >> > case
>> >> > I
>> >> > use blender files, they store the entire live state of the blender
>> >> > window
>> >> > including images and even Python scripts. Similar examples are
>> >> > countless
>> >> > out there.
>> >> >
>> >> > So the answer to the question what makes the image file format unique
>> >> > is
>> >> > simply.... Nothing
>> >> > What's the advantage of using the image format compared to other
>> >> > files ?
>> >> > None
>> >> >
>> >> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>> >>
>> >> > btc@
>> >>
>> >> > > wrote:
>> >> >
>> >> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>> >>
>> >> > kilon.alios@
>> >>
>> >> > >
>> >> >> wrote:
>> >> >> > "A Smalltalk Image is your entire system. The Image includes all
>> >> >> > the
>> >> >> tools
>> >> >> > required to interact, customize and add functionality to your
>> >> >> > system,
>> >> >> so
>> >> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >> >
>> >> >> >
>> >> >> > Thats not the case even for someone like me that has been working
>> >> >> > with
>> >> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> >> drives
>> >> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> >> different
>> >> >> > universe than smalltalk. It exists in the same universe than many
>> >> >> > other
>> >> >> > languages do exists and thats the C universe, the universe of the
>> >> >> > OS.
>> >> >> > Essentially what drives your system is not smalltalk is C. The
>> >> >> diffirence is
>> >> >> > that for a part of it that is high level enough, Slang is used, a
>> >> >> Hybrid
>> >> >> > language between C and Smalltalk that compiles to C. So while in
>> >> >> > the
>> >> >> image
>> >> >> > everything is , well almost everything, an object all the way
>> >> >> > down,
>> >> >> > in
>> >> >> the
>> >> >> > VM everything is C all the way down.
>> >> >>
>> >> >> To take that argument further, the VM is not even the thing driving
>> >> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> >> machine code. Further, what drives that are the electrons flowing
>> >> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> >> without files. Files relate to Pharo only to the same extent that a
>> >> >> database like Oracle or Postgres can be said to use files. That is,
>> >> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> >> though files are used by the server to store the data? Its just a
>> >> >> matter of where you draw the line of abstraction.
>> >> >>
>> >> >> cheers -ben
>> >> >>
>> >> >> > Ironically an image misses the most important tool to even
>> >> >> > generate
>> >> >> this
>> >> >> C
>> >> >> > code and thats the VMMaker that has to be installed separately.
>> >> >> > And
>> >> >> > of
>> >> >> > course there are parts of the system that are coded in pure C,
>> >> >> > like
>> >> >> some
>> >> >> > core functionalities of the VM and of course plugins and external
>> >> >> libraries
>> >> >> > that the image has to rely on make things happen.
>> >> >> >
>> >> >> > Of course the image is still fairly powerful, you can change the
>> >> >> syntax,
>> >> >> > implement high level libraries, IDE tools and much more. But its
>> >> >> > not
>> >> >> the
>> >> >> > core of the system just another essential part of it.
>> >> >> >
>> >> >> >
>> >> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>> >>
>> >> > kilon.alios@
>> >>
>> >> > >
>> >> >> > wrote:
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >> >>> fine.
>> >> >> the
>> >> >> >>> problem is more likely management of the same. browsing the
>> >> >> >>> changes
>> >> >> is
>> >> >> >>> not
>> >> >> >>> really convenient. ideally i'd like to see versions in the
>> >> >> class-browser
>> >> >> >>> and
>> >> >> >>> in the debugger, where on error i could then take a look at
>> >> >> >>> older
>> >> >> >>> versions for
>> >> >> >>> comparison, and switch to them to see if maybe the last change
>> >> >> >>> was
>> >> >> the
>> >> >> >>> cause of
>> >> >> >>> the error.
>> >> >> >>>
>> >> >> >>> greetings, martin.
>> >> >> >>>
>> >> >> >>
>> >> >> >> There are versions already for methods. So the functionality is
>> >> >> >> there.
>> >> >> >>
>> >> >> >> I disagree however with you, I think that changes file was
>> >> >> >> created
>> >> >> >> for
>> >> >> the
>> >> >> >> precise scenarios of an image crash/ lockdown. In that case you
>> >> >> >> may
>> >> >> want to
>> >> >> >> go back through the code and dont remember which method was
>> >> >> >> triggered
>> >> >> or
>> >> >> >> what else was defined and created. In the case going
>> >> >> >> chronologically
>> >> >> which
>> >> >> >> is how the changes file is already organised is far more useful
>> >> >> >> than
>> >> >> going
>> >> >> >> method and class based.
>> >> >> >>
>> >> >> >> But I do agree it would be useful to extend the tools working
>> >> >> >> with
>> >> >> changes
>> >> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> >> do.
>> >> >>
>> >> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> >> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> >> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>> >>
>> >
>>
>

Thierry Goubier

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

Hi Kilon,

I think what you are describing as a need for you is the ability to directly load into an image chunks of image memory space, containing already initialized images and code (i.e. live objects stored one way or another). Fuel could be an answer to that (wasn't that Tanker, the Fuel for code?).

Now, whether those are files or not does not concern me. Give me raw blocks on a disk or magic requests to do over a network to get those image chunks and I'll be happy.

Thierry

2015-12-08 1:06 GMT+01:00 Dimitris Chloupis <[hidden email]>:

You could not ask the worst case person even if you wanted :D

But here we go ...

Will depend , let take my ChronosManager for example

To modify the existing code of ChronosManager , is just text that gets compiled to bytecode immediately on each accept, no need to be loaded from a file, even though it gets added to the source and changes files.

To add code to the image whether its the entire code of ChronosManager or just a commit, it will load the code from st files, plus the extra filetree metadata files for class definition, version control and method categorization. I use Github but its a very similar story for those using Smalltalkhub.

To add code that contains data like an image, like I do with the GUI resources , I use a large collection of PNG files which with the help of the IconFactory will compile the binary data to a bytecode string and also pass it as a form to be used immediately. I am thinking instead to replace the iconfactory with direct access to the png files and creation of the form or a need to exist basis. Lazy loading in short.

In the future that I will need a common ground for all the images that use ChronoManager to share data and live state , I will be using fuel files.

For Ephestos (ChronosManager is part of the Ephestos project) that has to deal with 3d data, the collection of files grow exponentially which I try to keep outside the image because that data can easily reach several GBs and I dont want the image to explode in my face. To give you an example one folder I have with several blender files is around 2 gbs tops, each file being more than 250 MB. Blend files by the way implement their own kind of "object orientation" even though Blender is largely coded in C.

So if you ask me how many files I have to pass to the opal compiler if all I wanted is to stick just with the image my answer would be "a ton" but its a moot a point since the image cant handle so much data anyway. But even in the current scenario, its still a lot of files if you count the pngs and the repository .

I actually now try to remove my dependency on SmaCC because it makes the load time very slow, apparently compilation of large projects in the image is super slow and I will divert the parsing back to the python side which is way faster and since its python syntax i am parsing, python is really good at it anyway and helps me remove one unnecessary dependency. Though I have to say I am very happy with SmaCC and its great to have it around.

In the future I predict that I will have to keep more and more things into files to make loading of Ephestos fast and also its execution because image is not really optimized for what I want it to use it for.

So in my case, its files all the way !

On Tue, Dec 8, 2015 at 1:09 AM Ben Coman <[hidden email]> wrote:
Just one simple question...
When you want Pharo to compile some code, which *file* do you pass to
our Opal compiler?

cheers -ben

On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]> wrote:
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between multiple
> images. So I was pointed by the good people here to the Fuel library that ,
> surprise surprise , it generates binary files that contain objects in their
> live state that helps you move and share code and data between images. Works
> well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over the
> image. That is, specialization. When an app find a specific file , just by
> looking at its extension it immediately knows the structure of the data and
> the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything special
> to these files, apart from the fact that their authors made sure to follow
> the specific structure to ensure compatibility with third party apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more Smalltalk
> friendly and still retain all the advantages of files and file system but ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file has
> to attend to, which is far more stone age , to use your remark , than files.
> That is the ability to use full memory of the system and the ability to deal
> with large data without any large hits on performance. In short good support
> for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in them,
> and we aren't going to budge. *Files are about as low a storage abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:
>>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files at
>> all is because we live in a file-based culture. And the reason our culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> > closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> > a
>> > binary format does not make it unique or any less of a file. In my case
>> > I
>> > use blender files, they store the entire live state of the blender
>> > window
>> > including images and even Python scripts. Similar examples are countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> >> > with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> >> > other
>> >> > languages do exists and thats the C universe, the universe of the OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> >> > in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> >> > of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> >> >> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> >> >> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> >> >> triggered
>> >> or
>> >> >> what else was defined and created. In the case going chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>

kilon.alios

Re: Stop Thinking in Terms of Files

In reply to this post by Ben Coman

:)

Ironically all this discussion is something I have done already with myself to find a good way to be productive with the problems I try to solve with pharo.

So by all mean, do so , and share ;)

On Tue, Dec 8, 2015 at 10:50 AM Ben Coman <[hidden email]> wrote:

Fair enough ;p
Now I'll get back doing something productive
cheers -ben

On Tue, Dec 8, 2015 at 8:06 AM, Dimitris Chloupis <[hidden email]> wrote:
> You could not ask the worst case person even if you wanted :D
>
> But here we go ...
>
> Will depend , let take my ChronosManager for example
>
> To modify the existing code of ChronosManager , is just text that gets
> compiled to bytecode immediately on each accept, no need to be loaded from a
> file, even though it gets added to the source and changes files.
>
> To add code to the image whether its the entire code of ChronosManager or
> just a commit, it will load the code from st files, plus the extra filetree
> metadata files for class definition, version control and method
> categorization. I use Github but its a very similar story for those using
> Smalltalkhub.
>
> To add code that contains data like an image, like I do with the GUI
> resources , I use a large collection of PNG files which with the help of the
> IconFactory will compile the binary data to a bytecode string and also pass
> it as a form to be used immediately. I am thinking instead to replace the
> iconfactory with direct access to the png files and creation of the form or
> a need to exist basis. Lazy loading in short.
>
> In the future that I will need a common ground for all the images that use
> ChronoManager to share data and live state , I will be using fuel files.
>
> For Ephestos (ChronosManager is part of the Ephestos project) that has to
> deal with 3d data, the collection of files grow exponentially which I try to
> keep outside the image because that data can easily reach several GBs and I
> dont want the image to explode in my face. To give you an example one folder
> I have with several blender files is around 2 gbs tops, each file being more
> than 250 MB. Blend files by the way implement their own kind of "object
> orientation" even though Blender is largely coded in C.
>
> So if you ask me how many files I have to pass to the opal compiler if all I
> wanted is to stick just with the image my answer would be "a ton" but its a
> moot a point since the image cant handle so much data anyway. But even in
> the current scenario, its still a lot of files if you count the pngs and the
> repository .
>
> I actually now try to remove my dependency on SmaCC because it makes the
> load time very slow, apparently compilation of large projects in the image
> is super slow and I will divert the parsing back to the python side which is
> way faster and since its python syntax i am parsing, python is really good
> at it anyway and helps me remove one unnecessary dependency. Though I have
> to say I am very happy with SmaCC and its great to have it around.
>
> In the future I predict that I will have to keep more and more things into
> files to make loading of Ephestos fast and also its execution because image
> is not really optimized for what I want it to use it for.
>
> So in my case, its files all the way !
>
>
> On Tue, Dec 8, 2015 at 1:09 AM Ben Coman <[hidden email]> wrote:
>>
>> Just one simple question...
>> When you want Pharo to compile some code, which *file* do you pass to
>> our Opal compiler?
>>
>> cheers -ben
>>
>> On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]>
>> wrote:
>> > The devil is in the details ;)
>> >
>> > It matters to me, I just came across the need to share data between
>> > multiple
>> > images. So I was pointed by the good people here to the Fuel library
>> > that ,
>> > surprise surprise , it generates binary files that contain objects in
>> > their
>> > live state that helps you move and share code and data between images.
>> > Works
>> > well and I really like its design :)
>> >
>> > We are not talking here about something sophisticated, we are talking
>> > here
>> > super basic functionality. Images sharing data and code. What we use ?
>> > Files. The image by itself has no functionality to even cover this super
>> > basic scenario because as a format is made to be self contained.
>> >
>> > How you cant even care for such basic functionality ? Of course you will
>> > at
>> > some point. Its unavoidable.
>> >
>> > The nice thing about files is that they have one very big advantage over
>> > the
>> > image. That is, specialization. When an app find a specific file , just
>> > by
>> > looking at its extension it immediately knows the structure of the data
>> > and
>> > the code that it may contain.
>> >
>> > On other hand when you have an object system like the image is, such
>> > specifications go outside the window meaning you have to deal with the
>> > fact
>> > and trust that those that made those images have adhered to specific
>> > guidelines so you can make sure that your code wont run in front of some
>> > very nasty surprises.
>> >
>> > But since the image itself allow you hack so deeply as the syntax of the
>> > language , you can't be sure how the data and code will be presented.
>> > Sure
>> > they will objects, but the format does not really matter so much as the
>> > structure itself.
>> >
>> > In those cases files win hands down because they tend to be far more
>> > restricted on how they are structured. Not because there is anything
>> > special
>> > to these files, apart from the fact that their authors made sure to
>> > follow
>> > the specific structure to ensure compatibility with third party apps.
>> >
>> > So not only Files are not on the Stone Age but they have evolved the
>> > level
>> > of specification to a whole new level that have made the foundation of
>> > our
>> > every day lives.
>> >
>> > Sure you could probably replace files with a new way that is more
>> > Smalltalk
>> > friendly and still retain all the advantages of files and file system
>> > but ,
>> > Smalltalk has not presented such solution to my knowledge. Hence we the
>> > smalltalkers we will still keep relying heavily on files for our every
>> > day
>> > needs until such solution is presented to us. Also with the huge wealth
>> > of
>> > file formats it would be a pain in the ass to replace them with a
>> > smalltalk
>> > solution.
>> >
>> > In the mean time there are even more pressing matter that the image file
>> > has
>> > to attend to, which is far more stone age , to use your remark , than
>> > files.
>> > That is the ability to use full memory of the system and the ability to
>> > deal
>> > with large data without any large hits on performance. In short good
>> > support
>> > for 64 bit and big data.
>> >
>> >
>> > "But these are implementation details...implementation of the base
>> > system.
>> > /From the perspective of a programmer writing an application/, none of
>> > this
>> > matters.
>> >
>> > As I said earlier, the only reason why Smalltalk has to deal with files
>> > at
>> > all is because we live in a file-based culture. And the reason our
>> > culture
>> > is so entrenched with files is because we are too heavily invested in
>> > them,
>> > and we aren't going to budge. *Files are about as low a storage
>> > abstraction
>> > as you can get*, and they pre-date even Unix. Yes, files belong in the
>> > Stone
>> > Age!"
>> >
>> > On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]>
>> > wrote:
>> >>
>> >> But these are implementation details...implementation of the base
>> >> system.
>> >> /From the perspective of a programmer writing an application/, none of
>> >> this
>> >> matters.
>> >>
>> >> As I said earlier, the only reason why Smalltalk has to deal with files
>> >> at
>> >> all is because we live in a file-based culture. And the reason our
>> >> culture
>> >> is so entrenched with files is because we are too heavily invested in
>> >> them,
>> >> and we aren't going to budge. *Files are about as low a storage
>> >> abstraction
>> >> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> >> Stone
>> >> Age!
>> >>
>> >>
>> >>
>> >> kilon.alios wrote
>> >> > That's the thing you can't take the argument further without
>> >> > diminishing
>> >> > the value of you argument precisely for the fact that the vm is far
>> >> > closer
>> >> > related to the image than it is to 0s and 1s. That tight relation is
>> >> > fundamental to the behavior and existence of the image. It defines
>> >> > its
>> >> > functionality, purpose and limitations.
>> >> >
>> >> > The image itself is a file and the fact that it can store live state
>> >> > in
>> >> > a
>> >> > binary format does not make it unique or any less of a file. In my
>> >> > case
>> >> > I
>> >> > use blender files, they store the entire live state of the blender
>> >> > window
>> >> > including images and even Python scripts. Similar examples are
>> >> > countless
>> >> > out there.
>> >> >
>> >> > So the answer to the question what makes the image file format unique
>> >> > is
>> >> > simply.... Nothing
>> >> > What's the advantage of using the image format compared to other
>> >> > files ?
>> >> > None
>> >> >
>> >> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>> >>
>> >> > btc@
>> >>
>> >> > > wrote:
>> >> >
>> >> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>> >>
>> >> > kilon.alios@
>> >>
>> >> > >
>> >> >> wrote:
>> >> >> > "A Smalltalk Image is your entire system. The Image includes all
>> >> >> > the
>> >> >> tools
>> >> >> > required to interact, customize and add functionality to your
>> >> >> > system,
>> >> >> so
>> >> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >> >
>> >> >> >
>> >> >> > Thats not the case even for someone like me that has been working
>> >> >> > with
>> >> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> >> drives
>> >> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> >> different
>> >> >> > universe than smalltalk. It exists in the same universe than many
>> >> >> > other
>> >> >> > languages do exists and thats the C universe, the universe of the
>> >> >> > OS.
>> >> >> > Essentially what drives your system is not smalltalk is C. The
>> >> >> diffirence is
>> >> >> > that for a part of it that is high level enough, Slang is used, a
>> >> >> Hybrid
>> >> >> > language between C and Smalltalk that compiles to C. So while in
>> >> >> > the
>> >> >> image
>> >> >> > everything is , well almost everything, an object all the way
>> >> >> > down,
>> >> >> > in
>> >> >> the
>> >> >> > VM everything is C all the way down.
>> >> >>
>> >> >> To take that argument further, the VM is not even the thing driving
>> >> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> >> machine code. Further, what drives that are the electrons flowing
>> >> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> >> without files. Files relate to Pharo only to the same extent that a
>> >> >> database like Oracle or Postgres can be said to use files. That is,
>> >> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> >> though files are used by the server to store the data? Its just a
>> >> >> matter of where you draw the line of abstraction.
>> >> >>
>> >> >> cheers -ben
>> >> >>
>> >> >> > Ironically an image misses the most important tool to even
>> >> >> > generate
>> >> >> this
>> >> >> C
>> >> >> > code and thats the VMMaker that has to be installed separately.
>> >> >> > And
>> >> >> > of
>> >> >> > course there are parts of the system that are coded in pure C,
>> >> >> > like
>> >> >> some
>> >> >> > core functionalities of the VM and of course plugins and external
>> >> >> libraries
>> >> >> > that the image has to rely on make things happen.
>> >> >> >
>> >> >> > Of course the image is still fairly powerful, you can change the
>> >> >> syntax,
>> >> >> > implement high level libraries, IDE tools and much more. But its
>> >> >> > not
>> >> >> the
>> >> >> > core of the system just another essential part of it.
>> >> >> >
>> >> >> >
>> >> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>> >>
>> >> > kilon.alios@
>> >>
>> >> > >
>> >> >> > wrote:
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >> >>> fine.
>> >> >> the
>> >> >> >>> problem is more likely management of the same. browsing the
>> >> >> >>> changes
>> >> >> is
>> >> >> >>> not
>> >> >> >>> really convenient. ideally i'd like to see versions in the
>> >> >> class-browser
>> >> >> >>> and
>> >> >> >>> in the debugger, where on error i could then take a look at
>> >> >> >>> older
>> >> >> >>> versions for
>> >> >> >>> comparison, and switch to them to see if maybe the last change
>> >> >> >>> was
>> >> >> the
>> >> >> >>> cause of
>> >> >> >>> the error.
>> >> >> >>>
>> >> >> >>> greetings, martin.
>> >> >> >>>
>> >> >> >>
>> >> >> >> There are versions already for methods. So the functionality is
>> >> >> >> there.
>> >> >> >>
>> >> >> >> I disagree however with you, I think that changes file was
>> >> >> >> created
>> >> >> >> for
>> >> >> the
>> >> >> >> precise scenarios of an image crash/ lockdown. In that case you
>> >> >> >> may
>> >> >> want to
>> >> >> >> go back through the code and dont remember which method was
>> >> >> >> triggered
>> >> >> or
>> >> >> >> what else was defined and created. In the case going
>> >> >> >> chronologically
>> >> >> which
>> >> >> >> is how the changes file is already organised is far more useful
>> >> >> >> than
>> >> >> going
>> >> >> >> method and class based.
>> >> >> >>
>> >> >> >> But I do agree it would be useful to extend the tools working
>> >> >> >> with
>> >> >> changes
>> >> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> >> do.
>> >> >>
>> >> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> >> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> >> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>> >>
>> >
>>
>

kilon.alios

Re: Stop Thinking in Terms of Files

In reply to this post by Thierry Goubier

yes it got lost in my many and long posts.

I am already working with Fuel and as I said I am very happy with it.

I completely agree that whether its files or flying elephants as long as it does the job in a simple and flexible enough way its a good choice.

To summarise for those not wanting to read my long posts my motivation to write all these posts is that I disagree with the OP and the original blog post linked because :

1) files and especially file formats are extremely useful

2) definitely not in the Stone Age as the original author claims because they already solve myriads of modern practical problems that the image does not

3) file formats define a concrete way of representing data and code that the image usually does not abide because its format is far more abstract

4) files in no way interfere with the mission of Smalltalk and Pharo to present a live coding environment to the user that is easily accessible , since there are many examples of files that can store live code and state but even dead code and data can easily be turned to live.

Summary argument: Files are awesome and Pharo loves files and so do I :)

On Tue, Dec 8, 2015 at 11:12 AM Thierry Goubier <[hidden email]> wrote:

Hi Kilon,

I think what you are describing as a need for you is the ability to directly load into an image chunks of image memory space, containing already initialized images and code (i.e. live objects stored one way or another). Fuel could be an answer to that (wasn't that Tanker, the Fuel for code?).

Now, whether those are files or not does not concern me. Give me raw blocks on a disk or magic requests to do over a network to get those image chunks and I'll be happy.

Thierry

2015-12-08 1:06 GMT+01:00 Dimitris Chloupis <[hidden email]>:
You could not ask the worst case person even if you wanted :D

But here we go ...

Will depend , let take my ChronosManager for example

To modify the existing code of ChronosManager , is just text that gets compiled to bytecode immediately on each accept, no need to be loaded from a file, even though it gets added to the source and changes files.

To add code to the image whether its the entire code of ChronosManager or just a commit, it will load the code from st files, plus the extra filetree metadata files for class definition, version control and method categorization. I use Github but its a very similar story for those using Smalltalkhub.

To add code that contains data like an image, like I do with the GUI resources , I use a large collection of PNG files which with the help of the IconFactory will compile the binary data to a bytecode string and also pass it as a form to be used immediately. I am thinking instead to replace the iconfactory with direct access to the png files and creation of the form or a need to exist basis. Lazy loading in short.

In the future that I will need a common ground for all the images that use ChronoManager to share data and live state , I will be using fuel files.

For Ephestos (ChronosManager is part of the Ephestos project) that has to deal with 3d data, the collection of files grow exponentially which I try to keep outside the image because that data can easily reach several GBs and I dont want the image to explode in my face. To give you an example one folder I have with several blender files is around 2 gbs tops, each file being more than 250 MB. Blend files by the way implement their own kind of "object orientation" even though Blender is largely coded in C.

So if you ask me how many files I have to pass to the opal compiler if all I wanted is to stick just with the image my answer would be "a ton" but its a moot a point since the image cant handle so much data anyway. But even in the current scenario, its still a lot of files if you count the pngs and the repository .

I actually now try to remove my dependency on SmaCC because it makes the load time very slow, apparently compilation of large projects in the image is super slow and I will divert the parsing back to the python side which is way faster and since its python syntax i am parsing, python is really good at it anyway and helps me remove one unnecessary dependency. Though I have to say I am very happy with SmaCC and its great to have it around.

In the future I predict that I will have to keep more and more things into files to make loading of Ephestos fast and also its execution because image is not really optimized for what I want it to use it for.

So in my case, its files all the way !

On Tue, Dec 8, 2015 at 1:09 AM Ben Coman <[hidden email]> wrote:
Just one simple question...
When you want Pharo to compile some code, which *file* do you pass to
our Opal compiler?

cheers -ben

On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]> wrote:
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between multiple
> images. So I was pointed by the good people here to the Fuel library that ,
> surprise surprise , it generates binary files that contain objects in their
> live state that helps you move and share code and data between images. Works
> well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over the
> image. That is, specialization. When an app find a specific file , just by
> looking at its extension it immediately knows the structure of the data and
> the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything special
> to these files, apart from the fact that their authors made sure to follow
> the specific structure to ensure compatibility with third party apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more Smalltalk
> friendly and still retain all the advantages of files and file system but ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file has
> to attend to, which is far more stone age , to use your remark , than files.
> That is the ability to use full memory of the system and the ability to deal
> with large data without any large hits on performance. In short good support
> for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in them,
> and we aren't going to budge. *Files are about as low a storage abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:
>>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files at
>> all is because we live in a file-based culture. And the reason our culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> > closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> > a
>> > binary format does not make it unique or any less of a file. In my case
>> > I
>> > use blender files, they store the entire live state of the blender
>> > window
>> > including images and even Python scripts. Similar examples are countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> >> > with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> >> > other
>> >> > languages do exists and thats the C universe, the universe of the OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> >> > in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> >> > of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> >> >> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> >> >> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> >> >> triggered
>> >> or
>> >> >> what else was defined and created. In the case going chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>

philippeback

Re: Stop Thinking in Terms of Files

In reply to this post by Thierry Goubier

Sounds like Marea which never saw the light of day as a part of a Pharo release/official package.

http://rmod.inria.fr/web/software/Marea

http://www.jot.fm/issues/issue_2013_01/article2.pdf

On Tue, Dec 8, 2015 at 10:11 AM, Thierry Goubier <[hidden email]> wrote:

Hi Kilon,

I think what you are describing as a need for you is the ability to directly load into an image chunks of image memory space, containing already initialized images and code (i.e. live objects stored one way or another). Fuel could be an answer to that (wasn't that Tanker, the Fuel for code?).

Now, whether those are files or not does not concern me. Give me raw blocks on a disk or magic requests to do over a network to get those image chunks and I'll be happy.

Thierry

2015-12-08 1:06 GMT+01:00 Dimitris Chloupis <[hidden email]>:
You could not ask the worst case person even if you wanted :D

But here we go ...

Will depend , let take my ChronosManager for example

To modify the existing code of ChronosManager , is just text that gets compiled to bytecode immediately on each accept, no need to be loaded from a file, even though it gets added to the source and changes files.

To add code to the image whether its the entire code of ChronosManager or just a commit, it will load the code from st files, plus the extra filetree metadata files for class definition, version control and method categorization. I use Github but its a very similar story for those using Smalltalkhub.

To add code that contains data like an image, like I do with the GUI resources , I use a large collection of PNG files which with the help of the IconFactory will compile the binary data to a bytecode string and also pass it as a form to be used immediately. I am thinking instead to replace the iconfactory with direct access to the png files and creation of the form or a need to exist basis. Lazy loading in short.

In the future that I will need a common ground for all the images that use ChronoManager to share data and live state , I will be using fuel files.

For Ephestos (ChronosManager is part of the Ephestos project) that has to deal with 3d data, the collection of files grow exponentially which I try to keep outside the image because that data can easily reach several GBs and I dont want the image to explode in my face. To give you an example one folder I have with several blender files is around 2 gbs tops, each file being more than 250 MB. Blend files by the way implement their own kind of "object orientation" even though Blender is largely coded in C.

So if you ask me how many files I have to pass to the opal compiler if all I wanted is to stick just with the image my answer would be "a ton" but its a moot a point since the image cant handle so much data anyway. But even in the current scenario, its still a lot of files if you count the pngs and the repository .

I actually now try to remove my dependency on SmaCC because it makes the load time very slow, apparently compilation of large projects in the image is super slow and I will divert the parsing back to the python side which is way faster and since its python syntax i am parsing, python is really good at it anyway and helps me remove one unnecessary dependency. Though I have to say I am very happy with SmaCC and its great to have it around.

In the future I predict that I will have to keep more and more things into files to make loading of Ephestos fast and also its execution because image is not really optimized for what I want it to use it for.

So in my case, its files all the way !

On Tue, Dec 8, 2015 at 1:09 AM Ben Coman <[hidden email]> wrote:
Just one simple question...
When you want Pharo to compile some code, which *file* do you pass to
our Opal compiler?

cheers -ben

On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]> wrote:
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between multiple
> images. So I was pointed by the good people here to the Fuel library that ,
> surprise surprise , it generates binary files that contain objects in their
> live state that helps you move and share code and data between images. Works
> well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over the
> image. That is, specialization. When an app find a specific file , just by
> looking at its extension it immediately knows the structure of the data and
> the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything special
> to these files, apart from the fact that their authors made sure to follow
> the specific structure to ensure compatibility with third party apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more Smalltalk
> friendly and still retain all the advantages of files and file system but ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file has
> to attend to, which is far more stone age , to use your remark , than files.
> That is the ability to use full memory of the system and the ability to deal
> with large data without any large hits on performance. In short good support
> for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in them,
> and we aren't going to budge. *Files are about as low a storage abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:
>>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files at
>> all is because we live in a file-based culture. And the reason our culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> > closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> > a
>> > binary format does not make it unique or any less of a file. In my case
>> > I
>> > use blender files, they store the entire live state of the blender
>> > window
>> > including images and even Python scripts. Similar examples are countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> >> > with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> >> > other
>> >> > languages do exists and thats the C universe, the universe of the OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> >> > in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> >> > of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> >> >> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> >> >> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> >> >> triggered
>> >> or
>> >> >> what else was defined and created. In the case going chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>

EuanM

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

Again, you seem to be using "the image" mainly to mean "the .image file", and only occassionally to mean the actual image itself. They are distinct (albeit tightly related) things.

On 7 Dec 2015 22:09, "Dimitris Chloupis" <[hidden email]> wrote:

And this "victory of dead code and date" that files usually promote can further benefit Pharo .

For example lets take one of the main problems of the Pharo image, its size. There is already an effort to modularilise the image to make it small and everything else installable and unistallable.

But why not go a step further and have dead zone files. So you end up with a tiny image containing just the basics (few kbs if possible) even no GUI and IDE tools and instead you have a collection of files that can be read by the image , each file contains a diffirent project, you can view their code and their data but not install them in the image to do so. And only when you decide that "sure I could use this project" then you instruct the image to import the file and make the code and the data live and part of the image. This way the pharo system can keep growing without growing the image at all and let the users decide how much they want to grow the image themselves.

Technically speaking this functionality is already available through monticello , where monticello allow you to view a local or remote repository of code without loading the code to the image (via the "open button"). So its not even a new concept. Though in my mind I imagine all this happening from inside the System Browser.

Thats just one scenario how dead data and code can greatly benefit the Pharo system. I can keep brainstorming forever for the usefulness of files for Pharo.

On Mon, Dec 7, 2015 at 9:20 PM horrido <[hidden email]> wrote:
You make a convincing argument. Files are useful.

In modern circumstances, Smalltalk has to coexist with a file-based world.
As Joachim wrote earlier, we are well-equipped to deal with files, but we do
not think in terms of files when we code our applications. We are not
obliged to use a file-based toolchain. The point of my article was to
persuade other developers that letting their obsession with file-based tools
prevent them from adopting Smalltalk is short-sighted and
counter-productive. Files have their uses, but they should look beyond files
for other software creation possibilities. Smalltalk has much to offer.

And, yes, it would be very good to have 64-bit support in Smalltalk.

kilon.alios wrote
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between
> multiple images. So I was pointed by the good people here to the Fuel
> library that , surprise surprise , it generates binary files that contain
> objects in their live state that helps you move and share code and data
> between images. Works well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will
> at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over
> the image. That is, specialization. When an app find a specific file ,
> just
> by looking at its extension it immediately knows the structure of the data
> and the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the
> fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything
> special to these files, apart from the fact that their authors made sure
> to
> follow the specific structure to ensure compatibility with third party
> apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more
> Smalltalk
> friendly and still retain all the advantages of files and file system but
> ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a
> smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file
> has to attend to, which is far more stone age , to use your remark , than
> files. That is the ability to use full memory of the system and the
> ability
> to deal with large data without any large hits on performance. In short
> good support for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of
> this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in
> them,
> and we aren't going to budge. *Files are about as low a storage
> abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the
> Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <

> horrido.hobbies@

> > wrote:
>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files
>> at
>> all is because we live in a file-based culture. And the reason our
>> culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without
>> diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> a
>> > binary format does not make it unique or any less of a file. In my case
>> I
>> > use blender files, they store the entire live state of the blender
>> window
>> > including images and even Python scripts. Similar examples are
>> countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique
>> is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files
>> ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your
>> system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> other
>> >> > languages do exists and thats the C universe, the universe of the
>> OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the
>> changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> triggered
>> >> or
>> >> >> what else was defined and created. In the case going
>> chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful
>> than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>>

--
View this message in context: http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865872.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

kilon.alios

Re: Stop Thinking in Terms of Files

whats the diffirence ?

On Tue, Dec 8, 2015 at 1:23 PM EuanM <[hidden email]> wrote:

Again, you seem to be using "the image" mainly to mean "the .image file", and only occassionally to mean the actual image itself. They are distinct (albeit tightly related) things.

On 7 Dec 2015 22:09, "Dimitris Chloupis" <[hidden email]> wrote:
And this "victory of dead code and date" that files usually promote can further benefit Pharo .

For example lets take one of the main problems of the Pharo image, its size. There is already an effort to modularilise the image to make it small and everything else installable and unistallable.

But why not go a step further and have dead zone files. So you end up with a tiny image containing just the basics (few kbs if possible) even no GUI and IDE tools and instead you have a collection of files that can be read by the image , each file contains a diffirent project, you can view their code and their data but not install them in the image to do so. And only when you decide that "sure I could use this project" then you instruct the image to import the file and make the code and the data live and part of the image. This way the pharo system can keep growing without growing the image at all and let the users decide how much they want to grow the image themselves.

Technically speaking this functionality is already available through monticello , where monticello allow you to view a local or remote repository of code without loading the code to the image (via the "open button"). So its not even a new concept. Though in my mind I imagine all this happening from inside the System Browser.

Thats just one scenario how dead data and code can greatly benefit the Pharo system. I can keep brainstorming forever for the usefulness of files for Pharo.

On Mon, Dec 7, 2015 at 9:20 PM horrido <[hidden email]> wrote:
You make a convincing argument. Files are useful.

In modern circumstances, Smalltalk has to coexist with a file-based world.
As Joachim wrote earlier, we are well-equipped to deal with files, but we do
not think in terms of files when we code our applications. We are not
obliged to use a file-based toolchain. The point of my article was to
persuade other developers that letting their obsession with file-based tools
prevent them from adopting Smalltalk is short-sighted and
counter-productive. Files have their uses, but they should look beyond files
for other software creation possibilities. Smalltalk has much to offer.

And, yes, it would be very good to have 64-bit support in Smalltalk.

kilon.alios wrote
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between
> multiple images. So I was pointed by the good people here to the Fuel
> library that , surprise surprise , it generates binary files that contain
> objects in their live state that helps you move and share code and data
> between images. Works well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will
> at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over
> the image. That is, specialization. When an app find a specific file ,
> just
> by looking at its extension it immediately knows the structure of the data
> and the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the
> fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything
> special to these files, apart from the fact that their authors made sure
> to
> follow the specific structure to ensure compatibility with third party
> apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more
> Smalltalk
> friendly and still retain all the advantages of files and file system but
> ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a
> smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file
> has to attend to, which is far more stone age , to use your remark , than
> files. That is the ability to use full memory of the system and the
> ability
> to deal with large data without any large hits on performance. In short
> good support for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of
> this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in
> them,
> and we aren't going to budge. *Files are about as low a storage
> abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the
> Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <

> horrido.hobbies@

> > wrote:
>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files
>> at
>> all is because we live in a file-based culture. And the reason our
>> culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without
>> diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> a
>> > binary format does not make it unique or any less of a file. In my case
>> I
>> > use blender files, they store the entire live state of the blender
>> window
>> > including images and even Python scripts. Similar examples are
>> countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique
>> is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files
>> ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your
>> system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> other
>> >> > languages do exists and thats the C universe, the universe of the
>> OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the
>> changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> triggered
>> >> or
>> >> >> what else was defined and created. In the case going
>> chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful
>> than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>>

--
View this message in context: http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865872.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Mariano Martinez Peck

Re: Stop Thinking in Terms of Files

In reply to this post by philippeback

On Tue, Dec 8, 2015 at 8:11 AM, [hidden email] <[hidden email]> wrote:

Sounds like Marea which never saw the light of day as a part of a Pharo release/official package.

http://rmod.inria.fr/web/software/Marea
http://www.jot.fm/issues/issue_2013_01/article2.pdf

As sometimes happens with research, it ended up with a prototype. Now with a fast become in Spur it would be interesting to give it another pass :)

Yes, a subset of Marea. Marea allows you to swap out a given object graph (ideally, unused) to secondary place (files, another image, nosql db, whatever) and let lightweight proxies in the boundary of the graph you swap. If the graph happened to be needed, the proxies take care to bring them all again, materialize, replace proxies with target objects etc until you got back the original graph.

Now, it happens that in Smalltalk classes, methods, package etc are all objects graph at the end. That's why I said subset. Because Marea deal with objects graph, whether they are classes/method/packages or domain objects...

So yeah, I did an experiment in which I swapped out almost every class/methods of a running seaside image and then I navigated every where in the app so that needed classes that I swapped out would swap it. Then saved the image. Once you saved the image, the image can continue to work perfectly with it's associated secondary memory (files, another image, no sql db...whatever place where you put a fuel BLOB). Guess the image size for such a working/running Seaside app? 4MB. That's give you an idea of the kernel classes really needed to run for example a seaside app :)

Best,

On Tue, Dec 8, 2015 at 10:11 AM, Thierry Goubier <[hidden email]> wrote:
Hi Kilon,

I think what you are describing as a need for you is the ability to directly load into an image chunks of image memory space, containing already initialized images and code (i.e. live objects stored one way or another). Fuel could be an answer to that (wasn't that Tanker, the Fuel for code?).

Now, whether those are files or not does not concern me. Give me raw blocks on a disk or magic requests to do over a network to get those image chunks and I'll be happy.

Thierry

2015-12-08 1:06 GMT+01:00 Dimitris Chloupis <[hidden email]>:
You could not ask the worst case person even if you wanted :D

But here we go ...

Will depend , let take my ChronosManager for example

To modify the existing code of ChronosManager , is just text that gets compiled to bytecode immediately on each accept, no need to be loaded from a file, even though it gets added to the source and changes files.

To add code to the image whether its the entire code of ChronosManager or just a commit, it will load the code from st files, plus the extra filetree metadata files for class definition, version control and method categorization. I use Github but its a very similar story for those using Smalltalkhub.

To add code that contains data like an image, like I do with the GUI resources , I use a large collection of PNG files which with the help of the IconFactory will compile the binary data to a bytecode string and also pass it as a form to be used immediately. I am thinking instead to replace the iconfactory with direct access to the png files and creation of the form or a need to exist basis. Lazy loading in short.

In the future that I will need a common ground for all the images that use ChronoManager to share data and live state , I will be using fuel files.

For Ephestos (ChronosManager is part of the Ephestos project) that has to deal with 3d data, the collection of files grow exponentially which I try to keep outside the image because that data can easily reach several GBs and I dont want the image to explode in my face. To give you an example one folder I have with several blender files is around 2 gbs tops, each file being more than 250 MB. Blend files by the way implement their own kind of "object orientation" even though Blender is largely coded in C.

So if you ask me how many files I have to pass to the opal compiler if all I wanted is to stick just with the image my answer would be "a ton" but its a moot a point since the image cant handle so much data anyway. But even in the current scenario, its still a lot of files if you count the pngs and the repository .

I actually now try to remove my dependency on SmaCC because it makes the load time very slow, apparently compilation of large projects in the image is super slow and I will divert the parsing back to the python side which is way faster and since its python syntax i am parsing, python is really good at it anyway and helps me remove one unnecessary dependency. Though I have to say I am very happy with SmaCC and its great to have it around.

In the future I predict that I will have to keep more and more things into files to make loading of Ephestos fast and also its execution because image is not really optimized for what I want it to use it for.

So in my case, its files all the way !

On Tue, Dec 8, 2015 at 1:09 AM Ben Coman <[hidden email]> wrote:
Just one simple question...
When you want Pharo to compile some code, which *file* do you pass to
our Opal compiler?

cheers -ben

On Tue, Dec 8, 2015 at 2:02 AM, Dimitris Chloupis <[hidden email]> wrote:
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between multiple
> images. So I was pointed by the good people here to the Fuel library that ,
> surprise surprise , it generates binary files that contain objects in their
> live state that helps you move and share code and data between images. Works
> well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over the
> image. That is, specialization. When an app find a specific file , just by
> looking at its extension it immediately knows the structure of the data and
> the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything special
> to these files, apart from the fact that their authors made sure to follow
> the specific structure to ensure compatibility with third party apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more Smalltalk
> friendly and still retain all the advantages of files and file system but ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file has
> to attend to, which is far more stone age , to use your remark , than files.
> That is the ability to use full memory of the system and the ability to deal
> with large data without any large hits on performance. In short good support
> for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in them,
> and we aren't going to budge. *Files are about as low a storage abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <[hidden email]> wrote:
>>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files at
>> all is because we live in a file-based culture. And the reason our culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> > closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> > a
>> > binary format does not make it unique or any less of a file. In my case
>> > I
>> > use blender files, they store the entire live state of the blender
>> > window
>> > including images and even Python scripts. Similar examples are countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> >> > with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> >> > other
>> >> > languages do exists and thats the C universe, the universe of the OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> >> > in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> >> > of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> >> >>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> >> >> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> >> >> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> >> >> triggered
>> >> or
>> >> >> what else was defined and created. In the case going chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> >> >> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>

Mariano
http://marianopeck.wordpress.com

EuanM

Re: Stop Thinking in Terms of Files

In reply to this post by kilon.alios

I think it's potentially could be provided using Monticello as a basis, rather than "technically already available through Monticello",

In many ways, you are describing the Catalog Browser / Configurstion Browser of Pharo, which is built on top of Monticello, Metacello, and Versionner.

On 7 Dec 2015 22:09, "Dimitris Chloupis" <[hidden email]> wrote:

And this "victory of dead code and date" that files usually promote can further benefit Pharo .

For example lets take one of the main problems of the Pharo image, its size. There is already an effort to modularilise the image to make it small and everything else installable and unistallable.

But why not go a step further and have dead zone files. So you end up with a tiny image containing just the basics (few kbs if possible) even no GUI and IDE tools and instead you have a collection of files that can be read by the image , each file contains a diffirent project, you can view their code and their data but not install them in the image to do so. And only when you decide that "sure I could use this project" then you instruct the image to import the file and make the code and the data live and part of the image. This way the pharo system can keep growing without growing the image at all and let the users decide how much they want to grow the image themselves.

Technically speaking this functionality is already available through monticello , where monticello allow you to view a local or remote repository of code without loading the code to the image (via the "open button"). So its not even a new concept. Though in my mind I imagine all this happening from inside the System Browser.

Thats just one scenario how dead data and code can greatly benefit the Pharo system. I can keep brainstorming forever for the usefulness of files for Pharo.

On Mon, Dec 7, 2015 at 9:20 PM horrido <[hidden email]> wrote:
You make a convincing argument. Files are useful.

In modern circumstances, Smalltalk has to coexist with a file-based world.
As Joachim wrote earlier, we are well-equipped to deal with files, but we do
not think in terms of files when we code our applications. We are not
obliged to use a file-based toolchain. The point of my article was to
persuade other developers that letting their obsession with file-based tools
prevent them from adopting Smalltalk is short-sighted and
counter-productive. Files have their uses, but they should look beyond files
for other software creation possibilities. Smalltalk has much to offer.

And, yes, it would be very good to have 64-bit support in Smalltalk.

kilon.alios wrote
> The devil is in the details ;)
>
> It matters to me, I just came across the need to share data between
> multiple images. So I was pointed by the good people here to the Fuel
> library that , surprise surprise , it generates binary files that contain
> objects in their live state that helps you move and share code and data
> between images. Works well and I really like its design :)
>
> We are not talking here about something sophisticated, we are talking here
> super basic functionality. Images sharing data and code. What we use ?
> Files. The image by itself has no functionality to even cover this super
> basic scenario because as a format is made to be self contained.
>
> How you cant even care for such basic functionality ? Of course you will
> at
> some point. Its unavoidable.
>
> The nice thing about files is that they have one very big advantage over
> the image. That is, specialization. When an app find a specific file ,
> just
> by looking at its extension it immediately knows the structure of the data
> and the code that it may contain.
>
> On other hand when you have an object system like the image is, such
> specifications go outside the window meaning you have to deal with the
> fact
> and trust that those that made those images have adhered to specific
> guidelines so you can make sure that your code wont run in front of some
> very nasty surprises.
>
> But since the image itself allow you hack so deeply as the syntax of the
> language , you can't be sure how the data and code will be presented. Sure
> they will objects, but the format does not really matter so much as the
> structure itself.
>
> In those cases files win hands down because they tend to be far more
> restricted on how they are structured. Not because there is anything
> special to these files, apart from the fact that their authors made sure
> to
> follow the specific structure to ensure compatibility with third party
> apps.
>
> So not only Files are not on the Stone Age but they have evolved the level
> of specification to a whole new level that have made the foundation of our
> every day lives.
>
> Sure you could probably replace files with a new way that is more
> Smalltalk
> friendly and still retain all the advantages of files and file system but
> ,
> Smalltalk has not presented such solution to my knowledge. Hence we the
> smalltalkers we will still keep relying heavily on files for our every day
> needs until such solution is presented to us. Also with the huge wealth of
> file formats it would be a pain in the ass to replace them with a
> smalltalk
> solution.
>
> In the mean time there are even more pressing matter that the image file
> has to attend to, which is far more stone age , to use your remark , than
> files. That is the ability to use full memory of the system and the
> ability
> to deal with large data without any large hits on performance. In short
> good support for 64 bit and big data.
>
>
> "But these are implementation details...implementation of the base system.
> /From the perspective of a programmer writing an application/, none of
> this
> matters.
>
> As I said earlier, the only reason why Smalltalk has to deal with files at
> all is because we live in a file-based culture. And the reason our culture
> is so entrenched with files is because we are too heavily invested in
> them,
> and we aren't going to budge. *Files are about as low a storage
> abstraction
> as you can get*, and they pre-date even Unix. Yes, files belong in the
> Stone
> Age!"
>
> On Mon, Dec 7, 2015 at 6:45 PM horrido <

> horrido.hobbies@

> > wrote:
>
>> But these are implementation details...implementation of the base system.
>> /From the perspective of a programmer writing an application/, none of
>> this
>> matters.
>>
>> As I said earlier, the only reason why Smalltalk has to deal with files
>> at
>> all is because we live in a file-based culture. And the reason our
>> culture
>> is so entrenched with files is because we are too heavily invested in
>> them,
>> and we aren't going to budge. *Files are about as low a storage
>> abstraction
>> as you can get*, and they pre-date even Unix. Yes, files belong in the
>> Stone
>> Age!
>>
>>
>>
>> kilon.alios wrote
>> > That's the thing you can't take the argument further without
>> diminishing
>> > the value of you argument precisely for the fact that the vm is far
>> closer
>> > related to the image than it is to 0s and 1s. That tight relation is
>> > fundamental to the behavior and existence of the image. It defines its
>> > functionality, purpose and limitations.
>> >
>> > The image itself is a file and the fact that it can store live state in
>> a
>> > binary format does not make it unique or any less of a file. In my case
>> I
>> > use blender files, they store the entire live state of the blender
>> window
>> > including images and even Python scripts. Similar examples are
>> countless
>> > out there.
>> >
>> > So the answer to the question what makes the image file format unique
>> is
>> > simply.... Nothing
>> > What's the advantage of using the image format compared to other files
>> ?
>> > None
>> >
>> > On Mon, 7 Dec 2015 at 15:14, Ben Coman <
>>
>> > btc@
>>
>> > > wrote:
>> >
>> >> On Mon, Dec 7, 2015 at 3:37 PM, Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> wrote:
>> >> > "A Smalltalk Image is your entire system. The Image includes all the
>> >> tools
>> >> > required to interact, customize and add functionality to your
>> system,
>> >> so
>> >> > Smalltalk’s IDE is a very Integrated Development Environment."
>> >> >
>> >> >
>> >> > Thats not the case even for someone like me that has been working
>> with
>> >> > smalltalk for only 2 years. The Image is not even the engine that
>> >> drives
>> >> > smalltalk . Thats the job of the VM that exists in a completely
>> >> different
>> >> > universe than smalltalk. It exists in the same universe than many
>> other
>> >> > languages do exists and thats the C universe, the universe of the
>> OS.
>> >> > Essentially what drives your system is not smalltalk is C. The
>> >> diffirence is
>> >> > that for a part of it that is high level enough, Slang is used, a
>> >> Hybrid
>> >> > language between C and Smalltalk that compiles to C. So while in the
>> >> image
>> >> > everything is , well almost everything, an object all the way down,
>> in
>> >> the
>> >> > VM everything is C all the way down.
>> >>
>> >> To take that argument further, the VM is not even the thing driving
>> >> the image ;). Essentially what drives it are the 1's and 0's of
>> >> machine code. Further, what drives that are the electrons flowing
>> >> through the chip. I think its fair to say that we *code* in Pharo
>> >> without files. Files relate to Pharo only to the same extent that a
>> >> database like Oracle or Postgres can be said to use files. That is,
>> >> when you do SQL queries, are you *thinking* in terms of files, even
>> >> though files are used by the server to store the data? Its just a
>> >> matter of where you draw the line of abstraction.
>> >>
>> >> cheers -ben
>> >>
>> >> > Ironically an image misses the most important tool to even generate
>> >> this
>> >> C
>> >> > code and thats the VMMaker that has to be installed separately. And
>> of
>> >> > course there are parts of the system that are coded in pure C, like
>> >> some
>> >> > core functionalities of the VM and of course plugins and external
>> >> libraries
>> >> > that the image has to rely on make things happen.
>> >> >
>> >> > Of course the image is still fairly powerful, you can change the
>> >> syntax,
>> >> > implement high level libraries, IDE tools and much more. But its not
>> >> the
>> >> > core of the system just another essential part of it.
>> >> >
>> >> >
>> >> > On Mon, Dec 7, 2015 at 9:24 AM Dimitris Chloupis <
>>
>> > kilon.alios@
>>
>> > >
>> >> > wrote:
>> >> >>>
>> >> >>>
>> >> >>> well, i wouldn't need or even want it in memory, so on disk is
>> fine.
>> >> the
>> >> >>> problem is more likely management of the same. browsing the
>> changes
>> >> is
>> >> >>> not
>> >> >>> really convenient. ideally i'd like to see versions in the
>> >> class-browser
>> >> >>> and
>> >> >>> in the debugger, where on error i could then take a look at older
>> >> >>> versions for
>> >> >>> comparison, and switch to them to see if maybe the last change was
>> >> the
>> >> >>> cause of
>> >> >>> the error.
>> >> >>>
>> >> >>> greetings, martin.
>> >> >>>
>> >> >>
>> >> >> There are versions already for methods. So the functionality is
>> there.
>> >> >>
>> >> >> I disagree however with you, I think that changes file was created
>> for
>> >> the
>> >> >> precise scenarios of an image crash/ lockdown. In that case you may
>> >> want to
>> >> >> go back through the code and dont remember which method was
>> triggered
>> >> or
>> >> >> what else was defined and created. In the case going
>> chronologically
>> >> which
>> >> >> is how the changes file is already organised is far more useful
>> than
>> >> going
>> >> >> method and class based.
>> >> >>
>> >> >> But I do agree it would be useful to extend the tools working with
>> >> changes
>> >> >> , but then none stop anyone from doing so and is not that hard to
>> do.
>> >>
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865820.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>>

--
View this message in context: http://forum.world.st/Stop-Thinking-in-Terms-of-Files-tp4865614p4865872.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.