Working Directory Changes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Working Directory Changes

Guillermo Polito
Hi all,

I'd like to push a really core change in file management: the working directory. This is really needed for command line apps, when you have your app deployed in some directory and you're launching it from another one. The current implementation, where workingDirectory = imageDirectory, forces to have absolute paths or extra handling all over the place to manage this complexity.

Rajula proposed a couple of months ago a solution for this based on the getcwd functions. You can read in his blogpost why using getcwd is better than $PWD in general here:


Now, since accessing the working directory is a core part of Pharo but based on UFFI, his implementation was breaking the build process. We cannot and we will not integrate UFFI in the bootstrap because it depends mainly on the compiler which is a big beast. Instead, I propose that only for this core-core-core feature, we use directly FFI.

In other words, the bootstrap will include just a couple of classes to manage the basics of FFI. And the working directory will be fetched by using this low level API. Such a call looks like this:

(ExternalLibraryFunction
    name: 'getcwd'
    module: 'libc.dylib'
    callType: 1
    returnType: ExternalType char asPointerType
    argumentTypes: {
        ExternalType char asPointerType.
        ExternalType long })
            invokeWith: buffer with: bufferSize.

We reviewed it with Pablo two days ago. The build process works with this implementation and tests are still running. The pull request is in here:



Thanks,
Guille, Pablo and Rajula

--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Pavel Krivanek-3
From my experiments to execute FFI calls without FFI packages in the image I know that we can do it without it on even lower level but then we will anyway need to somehow handle library paths and so on. So to add these small packages into the bootstrapped kernel is totally reasonable and will open a lot of next possibilities.

So definitely yes!

Cheers,
-- Pavel

2017-10-11 10:27 GMT+02:00 Guillermo Polito <[hidden email]>:
Hi all,

I'd like to push a really core change in file management: the working directory. This is really needed for command line apps, when you have your app deployed in some directory and you're launching it from another one. The current implementation, where workingDirectory = imageDirectory, forces to have absolute paths or extra handling all over the place to manage this complexity.

Rajula proposed a couple of months ago a solution for this based on the getcwd functions. You can read in his blogpost why using getcwd is better than $PWD in general here:


Now, since accessing the working directory is a core part of Pharo but based on UFFI, his implementation was breaking the build process. We cannot and we will not integrate UFFI in the bootstrap because it depends mainly on the compiler which is a big beast. Instead, I propose that only for this core-core-core feature, we use directly FFI.

In other words, the bootstrap will include just a couple of classes to manage the basics of FFI. And the working directory will be fetched by using this low level API. Such a call looks like this:

(ExternalLibraryFunction
    name: 'getcwd'
    module: 'libc.dylib'
    callType: 1
    returnType: ExternalType char asPointerType
    argumentTypes: {
        ExternalType char asPointerType.
        ExternalType long })
            invokeWith: buffer with: bufferSize.

We reviewed it with Pablo two days ago. The build process works with this implementation and tests are still running. The pull request is in here:



Thanks,
Guille, Pablo and Rajula

--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13


Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Torsten Bergmann
In reply to this post by Guillermo Polito
For "safe" environments it would not make sense to offer already in a
minimal base support for
"unsafe" foreign calls and an (U)FFI.

Out of curiosity: why not moving this to the VM itself to provide as a
service to the image?

Thanks
T.


Am 11.10.2017 um 10:27 schrieb Guillermo Polito:

> Hi all,
>
> I'd like to push a really core change in file management: the working
> directory. This is really needed for command line apps, when you have
> your app deployed in some directory and you're launching it from
> another one. The current implementation, where workingDirectory =
> imageDirectory, forces to have absolute paths or extra handling all
> over the place to manage this complexity.
>
> Rajula proposed a couple of months ago a solution for this based on
> the getcwd functions. You can read in his blogpost why using getcwd is
> better than $PWD in general here:
>
> https://vineetreddy.wordpress.com/2017/05/17/pwd-vs-getcwd/
>
> Now, since accessing the working directory is a core part of Pharo but
> based on UFFI, his implementation was breaking the build process. We
> cannot and we will not integrate UFFI in the bootstrap because it
> depends mainly on the compiler which is a big beast. Instead, I
> propose that only for this core-core-core feature, we use directly FFI.
>
> In other words, the bootstrap will include just a couple of classes to
> manage the basics of FFI. And the working directory will be fetched by
> using this low level API. Such a call looks like this:
>
> (ExternalLibraryFunction
>     name: 'getcwd'
>     module: 'libc.dylib'
>     callType: 1
>     returnType: ExternalType char asPointerType
>     argumentTypes: {
>         ExternalType char asPointerType.
>         ExternalType long })
>             invokeWith: buffer with: bufferSize.
>
> We reviewed it with Pablo two days ago. The build process works with
> this implementation and tests are still running. The pull request is
> in here:
>
> https://github.com/pharo-project/pharo/pull/92
>
>
> Thanks,
> Guille, Pablo and Rajula
>
> --
>
>
>
> Guille Polito
>
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research - _http://www.cnrs.fr_
>
>
> *Web:* _http://guillep.github.io_
>
> *Phone: *+33 06 52 70 66 13
>


Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

EstebanLM
In reply to this post by Guillermo Polito
in general, this core-core functions is better to implement them as primitives. 
and you put a fallback using UFFI (in case the primitive is not there).
of course, your solution works… but it adds dependency with FFI-Kernel, which is also not good to have it in the bootstrap IMO.

take a look at OSEnvironment>>getEnv implementation. 

we can go that direction and I think it will be better.

Esteban

On 11 Oct 2017, at 10:27, Guillermo Polito <[hidden email]> wrote:

Hi all,

I'd like to push a really core change in file management: the working directory. This is really needed for command line apps, when you have your app deployed in some directory and you're launching it from another one. The current implementation, where workingDirectory = imageDirectory, forces to have absolute paths or extra handling all over the place to manage this complexity.

Rajula proposed a couple of months ago a solution for this based on the getcwd functions. You can read in his blogpost why using getcwd is better than $PWD in general here:


Now, since accessing the working directory is a core part of Pharo but based on UFFI, his implementation was breaking the build process. We cannot and we will not integrate UFFI in the bootstrap because it depends mainly on the compiler which is a big beast. Instead, I propose that only for this core-core-core feature, we use directly FFI.

In other words, the bootstrap will include just a couple of classes to manage the basics of FFI. And the working directory will be fetched by using this low level API. Such a call looks like this:

(ExternalLibraryFunction
    name: 'getcwd'
    module: 'libc.dylib'
    callType: 1
    returnType: ExternalType char asPointerType
    argumentTypes: {
        ExternalType char asPointerType.
        ExternalType long })
            invokeWith: buffer with: bufferSize.

We reviewed it with Pablo two days ago. The build process works with this implementation and tests are still running. The pull request is in here:



Thanks,
Guille, Pablo and Rajula

--
   
Guille Polito
Research Engineer


Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Pavel Krivanek-3
In reply to this post by Torsten Bergmann


2017-10-11 10:48 GMT+02:00 Torsten Bergmann <[hidden email]>:
For "safe" environments it would not make sense to offer already in a minimal base support for
"unsafe" foreign calls and an (U)FFI.

They are always accessible as soon as the VM supports them. You can call FFI calls and do not have any FFI code in the image. To do not have FFI in the image does not make it safer.

-- Pavel
 

Out of curiosity: why not moving this to the VM itself to provide as a service to the image?

Thanks
T.


Am 11.10.2017 um 10:27 schrieb Guillermo Polito:
Hi all,

I'd like to push a really core change in file management: the working directory. This is really needed for command line apps, when you have your app deployed in some directory and you're launching it from another one. The current implementation, where workingDirectory = imageDirectory, forces to have absolute paths or extra handling all over the place to manage this complexity.

Rajula proposed a couple of months ago a solution for this based on the getcwd functions. You can read in his blogpost why using getcwd is better than $PWD in general here:

https://vineetreddy.wordpress.com/2017/05/17/pwd-vs-getcwd/

Now, since accessing the working directory is a core part of Pharo but based on UFFI, his implementation was breaking the build process. We cannot and we will not integrate UFFI in the bootstrap because it depends mainly on the compiler which is a big beast. Instead, I propose that only for this core-core-core feature, we use directly FFI.

In other words, the bootstrap will include just a couple of classes to manage the basics of FFI. And the working directory will be fetched by using this low level API. Such a call looks like this:

(ExternalLibraryFunction
    name: 'getcwd'
    module: 'libc.dylib'
    callType: 1
    returnType: ExternalType char asPointerType
    argumentTypes: {
        ExternalType char asPointerType.
        ExternalType long })
            invokeWith: buffer with: bufferSize.

We reviewed it with Pablo two days ago. The build process works with this implementation and tests are still running. The pull request is in here:

https://github.com/pharo-project/pharo/pull/92


Thanks,
Guille, Pablo and Rajula

--

       

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - _http://www.cnrs.fr_


*Web:* _http://guillep.github.io_

*Phone: *<a href="tel:%2B33%2006%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Torsten Bergmann
Yes - but with having #getCurrentDirectory as a primitive you have a
clean interface and boundary
to the underlying platform and can nonetheless provide them even when a
(sandboxed) VM might not allow
external calls.

So for instance SqueakJS is able to run in the browser - even when the
browser does usually
not offer a file system or native calls. So Berts JavaScript
implementation of the VM primitives just
mocks the file system primitives I guess and it just continues to work.

This allows to put the Smalltalk systems (Squeak, Pharo, Cuis, ...) into
a sandbox which might be good/necessary
for safe environments like cloud based solutions or other.

I'm interested in a good command line support too - but we should think
also about such scenarios because
they make our future even brighter.

Side note:
The original Squeak VM has the possibility to limit write access only to
a sandbox directory (UserDirectory and SecureDirectory)
as an example:
http://squeakvm.org/win32/settings.html

Thanks
T.


Am 11.10.2017 um 11:08 schrieb Pavel Krivanek:

>
>
> 2017-10-11 10:48 GMT+02:00 Torsten Bergmann <[hidden email]
> <mailto:[hidden email]>>:
>
>     For "safe" environments it would not make sense to offer already
>     in a minimal base support for
>     "unsafe" foreign calls and an (U)FFI.
>
>
> They are always accessible as soon as the VM supports them. You can
> call FFI calls and do not have any FFI code in the image. To do not
> have FFI in the image does not make it safer.
>
> -- Pavel
>
>
>     Out of curiosity: why not moving this to the VM itself to provide
>     as a service to the image?
>
>     Thanks
>     T.
>
>
>     Am 11.10.2017 um 10:27 schrieb Guillermo Polito:
>
>         Hi all,
>
>         I'd like to push a really core change in file management: the
>         working directory. This is really needed for command line
>         apps, when you have your app deployed in some directory and
>         you're launching it from another one. The current
>         implementation, where workingDirectory = imageDirectory,
>         forces to have absolute paths or extra handling all over the
>         place to manage this complexity.
>
>         Rajula proposed a couple of months ago a solution for this
>         based on the getcwd functions. You can read in his blogpost
>         why using getcwd is better than $PWD in general here:
>
>         https://vineetreddy.wordpress.com/2017/05/17/pwd-vs-getcwd/
>         <https://vineetreddy.wordpress.com/2017/05/17/pwd-vs-getcwd/>
>
>         Now, since accessing the working directory is a core part of
>         Pharo but based on UFFI, his implementation was breaking the
>         build process. We cannot and we will not integrate UFFI in the
>         bootstrap because it depends mainly on the compiler which is a
>         big beast. Instead, I propose that only for this
>         core-core-core feature, we use directly FFI.
>
>         In other words, the bootstrap will include just a couple of
>         classes to manage the basics of FFI. And the working directory
>         will be fetched by using this low level API. Such a call looks
>         like this:
>
>         (ExternalLibraryFunction
>             name: 'getcwd'
>             module: 'libc.dylib'
>             callType: 1
>             returnType: ExternalType char asPointerType
>             argumentTypes: {
>                 ExternalType char asPointerType.
>                 ExternalType long })
>                     invokeWith: buffer with: bufferSize.
>
>         We reviewed it with Pablo two days ago. The build process
>         works with this implementation and tests are still running.
>         The pull request is in here:
>
>         https://github.com/pharo-project/pharo/pull/92
>         <https://github.com/pharo-project/pharo/pull/92>
>
>
>         Thanks,
>         Guille, Pablo and Rajula
>
>         --
>
>
>
>         Guille Polito
>
>         Research Engineer
>
>         Centre de Recherche en Informatique, Signal et Automatique de
>         Lille
>
>         CRIStAL - UMR 9189
>
>         French National Center for Scientific Research -
>         _http://www.cnrs.fr_
>
>
>         *Web:* _http://guillep.github.io_
>
>         *Phone: *+33 06 52 70 66 13 <tel:%2B33%2006%2052%2070%2066%2013>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Mariano Martinez Peck
In reply to this post by Guillermo Polito
Hi Guille,

I think some time ago we discussed exactly this or at least very similar [1]. AFAIR, we didn't arrive to an agreement. Or maybe we did but I was never able to implement it, I don't remember. But I would appreciate if you can at least read the discussion to see if there is something worth considering with your new changes.

Cheers,




On Wed, Oct 11, 2017 at 5:27 AM, Guillermo Polito <[hidden email]> wrote:
Hi all,

I'd like to push a really core change in file management: the working directory. This is really needed for command line apps, when you have your app deployed in some directory and you're launching it from another one. The current implementation, where workingDirectory = imageDirectory, forces to have absolute paths or extra handling all over the place to manage this complexity.

Rajula proposed a couple of months ago a solution for this based on the getcwd functions. You can read in his blogpost why using getcwd is better than $PWD in general here:


Now, since accessing the working directory is a core part of Pharo but based on UFFI, his implementation was breaking the build process. We cannot and we will not integrate UFFI in the bootstrap because it depends mainly on the compiler which is a big beast. Instead, I propose that only for this core-core-core feature, we use directly FFI.

In other words, the bootstrap will include just a couple of classes to manage the basics of FFI. And the working directory will be fetched by using this low level API. Such a call looks like this:

(ExternalLibraryFunction
    name: 'getcwd'
    module: 'libc.dylib'
    callType: 1
    returnType: ExternalType char asPointerType
    argumentTypes: {
        ExternalType char asPointerType.
        ExternalType long })
            invokeWith: buffer with: bufferSize.

We reviewed it with Pablo two days ago. The build process works with this implementation and tests are still running. The pull request is in here:



Thanks,
Guille, Pablo and Rajula

--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:06%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Denis Kudriashov
In reply to this post by EstebanLM
2017-10-11 10:54 GMT+02:00 Esteban Lorenzano <[hidden email]>:
in general, this core-core functions is better to implement them as primitives. 
and you put a fallback using UFFI (in case the primitive is not there).
of course, your solution works… but it adds dependency with FFI-Kernel, which is also not good to have it in the bootstrap IMO.

take a look at OSEnvironment>>getEnv implementation. 

we can go that direction and I think it will be better.

+1 for new primitive. Because instead any new platform will require implemented FFI which is not easy. 
 

Esteban

On 11 Oct 2017, at 10:27, Guillermo Polito <[hidden email]> wrote:

Hi all,

I'd like to push a really core change in file management: the working directory. This is really needed for command line apps, when you have your app deployed in some directory and you're launching it from another one. The current implementation, where workingDirectory = imageDirectory, forces to have absolute paths or extra handling all over the place to manage this complexity.

Rajula proposed a couple of months ago a solution for this based on the getcwd functions. You can read in his blogpost why using getcwd is better than $PWD in general here:


Now, since accessing the working directory is a core part of Pharo but based on UFFI, his implementation was breaking the build process. We cannot and we will not integrate UFFI in the bootstrap because it depends mainly on the compiler which is a big beast. Instead, I propose that only for this core-core-core feature, we use directly FFI.

In other words, the bootstrap will include just a couple of classes to manage the basics of FFI. And the working directory will be fetched by using this low level API. Such a call looks like this:

(ExternalLibraryFunction
    name: 'getcwd'
    module: 'libc.dylib'
    callType: 1
    returnType: ExternalType char asPointerType
    argumentTypes: {
        ExternalType char asPointerType.
        ExternalType long })
            invokeWith: buffer with: bufferSize.

We reviewed it with Pablo two days ago. The build process works with this implementation and tests are still running. The pull request is in here:



Thanks,
Guille, Pablo and Rajula

--
   
Guille Polito
Research Engineer


Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13


Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Alistair Grant
On Wed, Oct 11, 2017 at 01:32:52PM +0200, Denis Kudriashov wrote:

> 2017-10-11 10:54 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>
>     in general, this core-core functions is better to implement them as
>     primitives.
>     and you put a fallback using UFFI (in case the primitive is not there).
>     of course, your solution works? but it adds dependency with FFI-Kernel,
>     which is also not good to have it in the bootstrap IMO.
>
>     take a look at OSEnvironment>>getEnv implementation.
>
>     we can go that direction and I think it will be better.
>
>
> +1 for new primitive. Because instead any new platform will require implemented
> FFI which is not easy.


Would it make sense to add this to FilePlugin?  The plugin has to be
working for general file system access anyway.


Cheers,
Alistair


>     Esteban
>    
>
>         On 11 Oct 2017, at 10:27, Guillermo Polito <[hidden email]>
>         wrote:
>
>         Hi all,
>
>         I'd like to push a really core change in file management: the working
>         directory. This is really needed for command line apps, when you have
>         your app deployed in some directory and you're launching it from
>         another one. The current implementation, where workingDirectory =
>         imageDirectory, forces to have absolute paths or extra handling all
>         over the place to manage this complexity.
>
>         Rajula proposed a couple of months ago a solution for this based on the
>         getcwd functions. You can read in his blogpost why using getcwd is
>         better than $PWD in general here:
>
>         https://vineetreddy.wordpress.com/2017/05/17/pwd-vs-getcwd/
>
>         Now, since accessing the working directory is a core part of Pharo but
>         based on UFFI, his implementation was breaking the build process. We
>         cannot and we will not integrate UFFI in the bootstrap because it
>         depends mainly on the compiler which is a big beast. Instead, I propose
>         that only for this core-core-core feature, we use directly FFI.
>
>         In other words, the bootstrap will include just a couple of classes to
>         manage the basics of FFI. And the working directory will be fetched by
>         using this low level API. Such a call looks like this:
>
>         (ExternalLibraryFunction
>             name: 'getcwd'
>             module: 'libc.dylib'
>             callType: 1
>             returnType: ExternalType char asPointerType
>             argumentTypes: {
>                 ExternalType char asPointerType.
>                 ExternalType long })
>                     invokeWith: buffer with: bufferSize.
>
>         We reviewed it with Pablo two days ago. The build process works with
>         this implementation and tests are still running. The pull request is in
>         here:
>
>         https://github.com/pharo-project/pharo/pull/92
>
>
>         Thanks,
>         Guille, Pablo and Rajula
>
>         --
>            [presentation]      Guille Polito
>         [CNRS-filaire]         Research Engineer
>
>
>                                Centre de Recherche en Informatique, Signal et
>                                Automatique de Lille
>                                CRIStAL - UMR 9189
>                                French National Center for Scientific Research -
>                                http://www.cnrs.fr
>
>                                Web: http://guillep.github.io
>                                Phone: +33 06 52 70 66 13
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Denis Kudriashov


2017-10-11 14:14 GMT+02:00 Alistair Grant <[hidden email]>:
On Wed, Oct 11, 2017 at 01:32:52PM +0200, Denis Kudriashov wrote:
> 2017-10-11 10:54 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>
>     in general, this core-core functions is better to implement them as
>     primitives.
>     and you put a fallback using UFFI (in case the primitive is not there).
>     of course, your solution works? but it adds dependency with FFI-Kernel,
>     which is also not good to have it in the bootstrap IMO.
>
>     take a look at OSEnvironment>>getEnv implementation.
>
>     we can go that direction and I think it will be better.
>
>
> +1 for new primitive. Because instead any new platform will require implemented
> FFI which is not easy.


Would it make sense to add this to FilePlugin?  The plugin has to be
working for general file system access anyway.

I think it is good idea
 


Cheers,
Alistair


>     Esteban
>
>
>         On 11 Oct 2017, at 10:27, Guillermo Polito <[hidden email]>
>         wrote:
>
>         Hi all,
>
>         I'd like to push a really core change in file management: the working
>         directory. This is really needed for command line apps, when you have
>         your app deployed in some directory and you're launching it from
>         another one. The current implementation, where workingDirectory =
>         imageDirectory, forces to have absolute paths or extra handling all
>         over the place to manage this complexity.
>
>         Rajula proposed a couple of months ago a solution for this based on the
>         getcwd functions. You can read in his blogpost why using getcwd is
>         better than $PWD in general here:
>
>         https://vineetreddy.wordpress.com/2017/05/17/pwd-vs-getcwd/
>
>         Now, since accessing the working directory is a core part of Pharo but
>         based on UFFI, his implementation was breaking the build process. We
>         cannot and we will not integrate UFFI in the bootstrap because it
>         depends mainly on the compiler which is a big beast. Instead, I propose
>         that only for this core-core-core feature, we use directly FFI.
>
>         In other words, the bootstrap will include just a couple of classes to
>         manage the basics of FFI. And the working directory will be fetched by
>         using this low level API. Such a call looks like this:
>
>         (ExternalLibraryFunction
>             name: 'getcwd'
>             module: 'libc.dylib'
>             callType: 1
>             returnType: ExternalType char asPointerType
>             argumentTypes: {
>                 ExternalType char asPointerType.
>                 ExternalType long })
>                     invokeWith: buffer with: bufferSize.
>
>         We reviewed it with Pablo two days ago. The build process works with
>         this implementation and tests are still running. The pull request is in
>         here:
>
>         https://github.com/pharo-project/pharo/pull/92
>
>
>         Thanks,
>         Guille, Pablo and Rajula
>
>         --
>            [presentation]      Guille Polito
>         [CNRS-filaire]         Research Engineer
>
>
>                                Centre de Recherche en Informatique, Signal et
>                                Automatique de Lille
>                                CRIStAL - UMR 9189
>                                French National Center for Scientific Research -
>                                http://www.cnrs.fr
>
>                                Web: http://guillep.github.io
>                                Phone: <a href="tel:%2B33%2006%2052%2070%2066%2013" value="+33652706613">+33 06 52 70 66 13
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Guillermo Polito
Hi all,

Now I had some minutes to take a further look at this.

** Mariano, yes, I remember that conversation. What we propose is aligned to that:

FileSystem workingDirectory shouldBe == getcwd

And **almost always**

FileSystem workingDirectory shouldBe ==  $PWD

This is because the real working directory is the one held by the process. $PWD is managed by bash and it can get out of sync with getcwd. For more info take a look at rajula's blogpost in the first email.


** About security, I don't really buy the argument.
 - There are many aspects of security: internal in the image, external. FFI is not the only thing to *care about*
 - right now we can do stuff like
    - FileSystem root ensureDeleteAll (and given pharo started with sudo...)
    - nil become: true
    - OpalCompiler compile: 'someMethodThatOverriddenBreaksEverything ...'
 - FFI plugin is shipped with the VM
    - so removing FFI support is as easy as removing the plugin
    - but this means that even if that FFI support is not in the image, any piece of code can do:
         - Myclass compile: 'test Metacello install: ''FFI''; please'; test.
         - or create a method calling the FFI primitives
 
 This does not mean I don't think about security, but I believe it should be managed in some other way. A security solution for real needs to manage reflection as well as external resources, and control how dependencies and packages are loaded. Because, moreover, I believe that while you may want to forbid FFI calls in some part of your app, you may want that some other parts of the system can use FFI.

** About modularity. But I'm sure I'm one of the guys that cares the most about modularity and cries aloud every time we introduce bad dependencies. Now, I'm proposing the introduction of FFI because I think its manageable and it's being controlled. It does not affect the bootstrap, and I don't think it will forbid us to get even smaller kernels.

** The most important argument, that at least I did not see anybody complain about, is backwards compatibility. This implementation may break backwards compatibility, because this does not hold anymore:

FileSystem imageDirectory == FileSystem workingDirectory

Well, yes, it holds if you start your vm/image from the same directory where the image is located. Which is what most users do.

** I also talked with Esteban about his reasons. His concern was about consistency. If you take a look at how getEnv is resolved:

getEnv: varName
"This method calls the Standard C Library getenv() function."
"OSEnvironment current getEnv: 'HOME' "
<primitive: 'primitiveGetenv' module: '' error: ec>
ec ifNil: [ ^self getEnvViaFFI: varName ].
ec == #'bad argument' ifTrue: [
varName isString 
ifFalse: [ ^self getEnv: varName asString ] ].
self primitiveFail

If first tries to call a primitive and then, as fallback, it calls FFI by default.

I changed the pull request to do the same thing about the working directory. That way, as soon as we have implemented the primitive, that will be in usage instead of the FFI call.

Guille

On Wed, Oct 11, 2017 at 3:07 PM, Denis Kudriashov <[hidden email]> wrote:


2017-10-11 14:14 GMT+02:00 Alistair Grant <[hidden email]>:
On Wed, Oct 11, 2017 at 01:32:52PM +0200, Denis Kudriashov wrote:
> 2017-10-11 10:54 GMT+02:00 Esteban Lorenzano <[hidden email]>:
>
>     in general, this core-core functions is better to implement them as
>     primitives.
>     and you put a fallback using UFFI (in case the primitive is not there).
>     of course, your solution works? but it adds dependency with FFI-Kernel,
>     which is also not good to have it in the bootstrap IMO.
>
>     take a look at OSEnvironment>>getEnv implementation.
>
>     we can go that direction and I think it will be better.
>
>
> +1 for new primitive. Because instead any new platform will require implemented
> FFI which is not easy.


Would it make sense to add this to FilePlugin?  The plugin has to be
working for general file system access anyway.

I think it is good idea
 


Cheers,
Alistair


>     Esteban
>
>
>         On 11 Oct 2017, at 10:27, Guillermo Polito <[hidden email]>
>         wrote:
>
>         Hi all,
>
>         I'd like to push a really core change in file management: the working
>         directory. This is really needed for command line apps, when you have
>         your app deployed in some directory and you're launching it from
>         another one. The current implementation, where workingDirectory =
>         imageDirectory, forces to have absolute paths or extra handling all
>         over the place to manage this complexity.
>
>         Rajula proposed a couple of months ago a solution for this based on the
>         getcwd functions. You can read in his blogpost why using getcwd is
>         better than $PWD in general here:
>
>         https://vineetreddy.wordpress.com/2017/05/17/pwd-vs-getcwd/
>
>         Now, since accessing the working directory is a core part of Pharo but
>         based on UFFI, his implementation was breaking the build process. We
>         cannot and we will not integrate UFFI in the bootstrap because it
>         depends mainly on the compiler which is a big beast. Instead, I propose
>         that only for this core-core-core feature, we use directly FFI.
>
>         In other words, the bootstrap will include just a couple of classes to
>         manage the basics of FFI. And the working directory will be fetched by
>         using this low level API. Such a call looks like this:
>
>         (ExternalLibraryFunction
>             name: 'getcwd'
>             module: 'libc.dylib'
>             callType: 1
>             returnType: ExternalType char asPointerType
>             argumentTypes: {
>                 ExternalType char asPointerType.
>                 ExternalType long })
>                     invokeWith: buffer with: bufferSize.
>
>         We reviewed it with Pablo two days ago. The build process works with
>         this implementation and tests are still running. The pull request is in
>         here:
>
>         https://github.com/pharo-project/pharo/pull/92
>
>
>         Thanks,
>         Guille, Pablo and Rajula
>
>         --
>            [presentation]      Guille Polito
>         [CNRS-filaire]         Research Engineer
>
>
>                                Centre de Recherche en Informatique, Signal et
>                                Automatique de Lille
>                                CRIStAL - UMR 9189
>                                French National Center for Scientific Research -
>                                http://www.cnrs.fr
>
>                                Web: http://guillep.github.io
>                                Phone: <a href="tel:%2B33%2006%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13
>
>
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: Working Directory Changes

Alistair Grant
Hi Guille,

On Thu, Oct 12, 2017 at 11:53:15AM +0200, Guillermo Polito wrote:

>
> ** The most important argument, that at least I did not see anybody complain
> about, is backwards compatibility. This implementation may break backwards
> compatibility, because this does not hold anymore:
>
> FileSystem imageDirectory == FileSystem workingDirectory
>
> Well, yes, it holds if you start your vm/image from the same directory where
> the image is located. Which is what most users do.
>

I gave poor Rajula quite a bit of trouble over this concern.  The end
result was that several people replied to say they don't care.

I later found out that this assumption was only introduced about half
way through Pharo 6 development.  Prior to that it was possible to
change the workingDirectory as it was stored in an instance variable
that could be changed.  So there will be very little code in existence
that makes this assumption.

Cheers,
Alistair