Pool for sysconf constants?

> Hi David,
>
>
> > On Oct 16, 2020, at 2:47 PM, David T. Lewis <[hidden email]> wrote:
> >
> > ???Hi Eliot,
> >
> > Replying back on vm-dev.
> >
> > Indeed I see the problem now, thanks.
> >
> > The closest that I can come to a useful suggestion is to write a
> > little C program to print out the index values of interest for a
> > given platform, then read that into the image to build the pool.
> > The index values are supposed to be opaque but I doubt that they
> > would change much in practice.
>
> This is what an ExternalPool does *automatically*.  One creates a pool
> containing a set of named variables, adds metadata to the pool, and
> then when the pool needs to initialize or change, it generates the C
> program automatically, runs it, reaps the values, and updates class
> side code to initialize the constants for specific platforms.  When it
> is run on a different platform it generates the program for that
> platform, etc.  The management problem therefore is how to manage the
> executable programs because these are needed if one wants to deploy on
> a specific platform.
>
> I came up with the idea many years ago at ParcPlace when the problem
> first surfaced (that different platforms, though they may use the same
> names for symbolic constants, abstract types, etc, but don't use the
> same values, concrete types, etc).  Monty write the implementation.
> Marcel has shepherded it into source.squeak.org/FFI.  Marcel has
> written an excellent and comprehensive class comment for ExternalPool.
> I urge that everyone interested in deployment using FFIs read it ASAP.
> That goes for me too, because things have moved on with the
> implementation.  There's no need to manage external C programs;
> there's a need to merge the definition code that is automatically
> generated when run on different platforms.
>
> I should push thoguh and sdo a sysconf pool to learn how the current
> system really works :-)
>
> >
> > I am looking only at a Linux system as I write this, and the only
> > way I can find to discover the available parameter names is to look
> > in /usr/include/bits/confname.h to see what is defined. I can't
> > spot any other way to get at the information, and I can't find any
> > runtime utility that provides a listing.
> >
> > But a little utility program that could be used to update the
> > pool might be good enough. Attaching an example C program, I'm
> > not sure if it helps.
> >
> > Dave
> >
> >
> >> On Tue, Oct 13, 2020 at 01:49:00PM -0700, Eliot Miranda wrote:
> >> Hi David,
> >>
> >>> On Mon, Oct 12, 2020 at 8:28 PM David T. Lewis <[hidden email]> wrote:
> >>>
> >>> On Sun, Oct 11, 2020 at 11:15:09AM -0700, Eliot Miranda wrote:
> >>>> Hi All,
> >>>>
> >>>>    I want to implement a number of processors query on MacOS & other
> >>>> unices to complement the info provided by GetSystemInfo on Windows. The
> >>>> natural way to do this on unices is to use sysconf (3), and of course
> >>> this
> >>>> points to the issue for which FFI pools are needed, the constant names
> >>> for
> >>>> sysconf such as _SC_NPROCESSORS_ONLN are defined, but their values are
> >>>> implementation dependent.
> >>>>
> >>>> But before I go amd implement an FFI pool for this I thought I'd ask
> >>>> a) anyone already done this?  Is it published anywhere?
> >>>> b) how are we going to organize such pools so that people don't have to
> >>>> reinvent the wheel?
> >>>> c) should there be a pool to cover sysconf or to cover unistd.h?  (the
> >>>> point here is that while the pool might start off small, it could grow
> >>> and
> >>>> grow and maybe unistd.h is too much surface to cover)
> >>>>
> >>>> thoughts, suggestions?
> >>>
> >>> Hi Eliot,
> >>>
> >>> At first glance, it looks to me like the hard part of the problem is to
> >>> know what sysconf names are available to be queried on any given platform
> >>> at runtime.  If you know that, or if you are only interested in a limited
> >>> number of well-known parameters (which seems likely), then it might be
> >>> simplest to just implement the sysconf(3) call either as an FFI call or
> >>> a call through the OSProcess plugin.
> >>>
> >>
> >> Well, I *am* implementing it as an FFI call in Terf.  The issue is deriving
> >> the right integer values for the particular symbolic names, because these
> >> vary across implementations. And that's what FFIPools are designed to
> >> manage.
> >> Right now I don't have the FFIPool so I've hard-wired the constant:
> >>
> >> !CMacOSXPlatform methodsFor: 'accessing' stamp: 'eem 10/11/2020 17:53'
> >> prior: 47990341!
> >> numberOfCPUs
> >>        "Warning, warning, Will Robertson!!!! The sysconf method bakes in a
> >> decidedly implementation-specific name for the support library *and* bakes
> >> in a value for a symbolic constant.  Both of these are implementation
> >> dependent and subject to change.  That said..."
> >>        ^self sysconf: 58
> >>
> >>        "CPlatform current numberOfCPUs"! !
> >> !CMacOSXPlatform methodsFor: 'private' stamp: 'eem 10/11/2020 17:51'!
> >> sysconf: systemInfo
> >>        "Answer a value from sysconf(3)."
> >>        <apicall: longlong 'sysconf' (long) module:
> >> '/usr/lib/system/libsystem_c.dylib'>
> >>        ^self externalCallFailed
> >>
> >>        " | scnprocsonline |
> >>        scnprocsonline := 58.
> >>        self current sysconf: scnprocsonline"! !
> >>
> >> There's much to be unhappy about in this code: libSystem.dylib is the
> >> abstract name, but one can't use it because the search facilities can't
> >> chain through the libraries that libSystem.B.dylib (ibSystem.dylib is a
> >> symbolic link) refers to.  So this could break in some future MacOS,
> >> whereas if libSystem.dylib did work, or if there was a symbolic name we
> >> could use that meant "the C library goddamnit", then it could be relied
> >> upon.  58 is meaningful only on MacOS, and presumably only since a
> >> particular version.  That's why we need an FFIPool to manage the
> >> cross-platform variations.  The type is longlong, which means we're not
> >> using a syntax that matches the declaration, and this will work only on
> >> 64-bits.  We can fix the FFI to interpret C names accoding to the
> >> playtform, but we have a serious backwards-compatibility problem in that
> >> all of the FFI definitions up til now have been written under this
> >> assumption.
> >>
> >> But I suspect that I am not answering the right question here. If I am
> >>> off base, can you give an example of some of the parameters that you
> >>> would want to be able to query?
> >>>
> >>
> >> First off Terf needs _SC_NUM_PROCESSORS_ONLN, which is notionally the
> >> number of processors online, but now answers the number of cores, which
> >> shows you how fast these ideas go stale (_SC_NUM_PROCESSORS_CONF is the
> >> number of processors configured ;-) ).  Other ones that could be meaningful
> >> for applications include _SC_ARG_MAX (The maximum bytes of argument to
> >> execve(2)) _SC_CHILD_MAX (The maximum number of simultaneous processes per
> >> user id), _SC_OPEN_MAX (The maximum number of open files per user id),
> >> _SC_STREAM_MAX (The minimum maximum number of streams that a process may
> >> have open at any one time), _SC_PHYS_PAGES (The number of pages of physical
> >> memory).
> >>
> >> There are many others, quite a few I can't imagine ever being compelling.
> >>
> >> Now yes, one can imagine implementing a cross-platform abstraction for
> >> these but it's a lot of effort for little gain.  It;s easier just to suck
> >> it up and implement the FFI call.
> >>
> >> But I'm imagining these things will get used by the community and I don't
> >> want people to have to reinvent the wheel.  This kind of stuff doesn't
> >> belong in trunk, but it does belong somewhere where we can share and
> >> coevolve the facilities.
> >>
> >>
> >>> Thanks,
> >>> Dave
> >>>
> >>>
> >>>
> >>
> >> --
> >> _,,,^..^,,,_
> >> best, Eliot
> >
> >>
> >
> > <scparams.c>
>