Pi vm builds need a priority checker applet

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Pi vm builds need a priority checker applet

timrowledge

A persistent problem with the PI vm is that users of xrdp need to wrap the start-up in ‘sudo -E’ because.. well, never mind, unix is just plain stupid. Whatever the issue, we can test for the problem of needing to use sudo by running a trivial test program in the shell script; all it does is try changing a thread priority and returns whether it worked or not.

So far as I know this is only an issue on the Pi but then I don’t use an x86 linux machine. I’d imagine any machine with an older kernel might need the check too but others will have to say. If it is of use to general linux machines then it is pretty obvious that we should put the source in platforms/unix/vm and add the build to the general make process. It if is really only a Pi issue then maybe someone has a strong view on having the source/build elsewhere. Any thoughts?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: SDP: Search and Destroy Pointer


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

fniephaus
 

On Thu, Aug 18, 2016 at 12:17 AM tim Rowledge <[hidden email]> wrote:

A persistent problem with the PI vm is that users of xrdp need to wrap the start-up in ‘sudo -E’ because.. well, never mind, unix is just plain stupid. Whatever the issue, we can test for the problem of needing to use sudo by running a trivial test program in the shell script; all it does is try changing a thread priority and returns whether it worked or not.

Just to make sure: it is not enough to create a squeak.conf as describe at [1] and reboot?
Could you please give [2] a try on a raspi? Feedback is more than welcome as we are about to release Squeak 5.1 very soon.

Thanks,
Fabio



So far as I know this is only an issue on the Pi but then I don’t use an x86 linux machine. I’d imagine any machine with an older kernel might need the check too but others will have to say. If it is of use to general linux machines then it is pretty obvious that we should put the source in platforms/unix/vm and add the build to the general make process. It if is really only a Pi issue then maybe someone has a strong view on having the source/build elsewhere. Any thoughts?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: SDP: Search and Destroy Pointer


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

timrowledge


> On 17-08-2016, at 3:34 PM, Fabio Niephaus <[hidden email]> wrote:
>
>
> On Thu, Aug 18, 2016 at 12:17 AM tim Rowledge <[hidden email]> wrote:
>
> A persistent problem with the PI vm is that users of xrdp need to wrap the start-up in ‘sudo -E’ because.. well, never mind, unix is just plain stupid. Whatever the issue, we can test for the problem of needing to use sudo by running a trivial test program in the shell script; all it does is try changing a thread priority and returns whether it worked or not.
>
> Just to make sure: it is not enough to create a squeak.conf as describe at [1] and reboot?

Unfortunately that doesn’t work when using xrdp for your remote display. It’s not even a strictly squeak issue - some googling will reveal a depressingly long list of problems to do with the same error. The Raspbian kernel; has no problems at all with squeak when using a direct display. It doesn’t have this particular problem with vnc - but it *does* have a fatal problem with trying to use the composition input window calls. Again, not a specifically squeak, raspbian nor ARM issue but a generic bit of nastiness in linux.

> Could you please give [2] a try on a raspi? Feedback is more than welcome as we are about to release Squeak 5.1 very soon.

As long as it’s made from the very latest source Eliot put out today it will be ok. *Except* I note that the HostWindowPlugin is missing from the plugins/int file in build.linuxARMv6/squeak.cog.spur and in fact after a quick look it is missing form all the linux related build directories that I checked. Whilst the plugin is not complete it does provide support for title and size aspects and so I’d recommend including it.

5.1 will be the basis of the next Pi Scratch release so I have several millions of reasons to help getting it right...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Don't sweat petty things....or pet sweaty things.


Reply | Threaded
Open this post in threaded view
|

VM version naming (was Re: [Vm-dev] Pi vm builds need a priority checker applet)

timrowledge

Oh, something I spotted when looking at  https://github.com/OpenSmalltalk/opensmalltalk-vm/releases#linux as you suggested - the vm version numbering seems to be the old style from when we used the svn source tree. Unless I misunderstood we now use the github date string versioning (my latest vm from this afternoon is 5.0-201608171728 for example).

If this has been changed then I need to know pretty urgently in order to fix the Pi scratch/squeak shell scripts!

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- Can't find log base two of 65536 without a calculator.


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

David T. Lewis
In reply to this post by timrowledge
 
On Wed, Aug 17, 2016 at 03:17:05PM -0700, tim Rowledge wrote:
>
> A persistent problem with the PI vm is that users of xrdp need to wrap the start-up in ???sudo -E??? because.. well, never mind, unix is just plain stupid. Whatever the issue, we can test for the problem of needing to use sudo by running a trivial test program in the shell script; all it does is try changing a thread priority and returns whether it worked or not.
>
> So far as I know this is only an issue on the Pi but then I don???t use an x86 linux machine. I???d imagine any machine with an older kernel might need the check too but others will have to say. If it is of use to general linux machines then it is pretty obvious that we should put the source in platforms/unix/vm and add the build to the general make process. It if is really only a Pi issue then maybe someone has a strong view on having the source/build elsewhere. Any thoughts?
>
> tim

It would be good if we could figure out the reason that you need to use
sudo when running with xrdp. Running Squeak with root privileges is a
Very Bad Idea, and we certainly should not be encouraging kids to do that.
Hopefully there is a way to convince xrdp to behave properly without the sudo.

Unfortunately I don't have a Pi readily at hand to help right now. Can you
say anything about what error messages or symptoms that you see when you
run under xrdp without the "sudo -E"? Is it related to the privileges for
the heartbeat thread priority?

Dave


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

timrowledge


> On 17-08-2016, at 4:42 PM, David T. Lewis <[hidden email]> wrote:
>
>
> It would be good if we could figure out the reason that you need to use
> sudo when running with xrdp. Running Squeak with root privileges is a
> Very Bad Idea, and we certainly should not be encouraging kids to do that.
> Hopefully there is a way to convince xrdp to behave properly without the sudo.

I wish - and if anyone can work out wtf is happening a lot of other people will likely be happy too. It’s not a new thing so far as I can see.

>
> Unfortunately I don't have a Pi readily at hand to help right now. Can you
> say anything about what error messages or symptoms that you see when you
> run under xrdp without the "sudo -E"? Is it related to the privileges for
> the heartbeat thread priority?

It appears to be the heartbeat stuff. Something in the various bits of code that are used by xrdp gets upset by it even though the basic system is fine with it.
There’s a fairly long explanation I put up on stackoverflow to see if the unix gurus there might know, but so far there hasn’t even been an attempt at an answer.
http://stackoverflow.com/questions/38384606/how-to-avoid-using-sudo-for-certain-programs-under-xrdp-in-raspbian

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: BIK: Buggered if I Know


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

David T. Lewis
 
On Wed, Aug 17, 2016 at 05:43:04PM -0700, tim Rowledge wrote:

>
>
> > On 17-08-2016, at 4:42 PM, David T. Lewis <[hidden email]> wrote:
> >
> >
> > It would be good if we could figure out the reason that you need to use
> > sudo when running with xrdp. Running Squeak with root privileges is a
> > Very Bad Idea, and we certainly should not be encouraging kids to do that.
> > Hopefully there is a way to convince xrdp to behave properly without the sudo.
>
> I wish - and if anyone can work out wtf is happening a lot of other people will likely be happy too. It???s not a new thing so far as I can see.
>
> >
> > Unfortunately I don't have a Pi readily at hand to help right now. Can you
> > say anything about what error messages or symptoms that you see when you
> > run under xrdp without the "sudo -E"? Is it related to the privileges for
> > the heartbeat thread priority?
>
> It appears to be the heartbeat stuff. Something in the various bits of code that are used by xrdp gets upset by it even though the basic system is fine with it.
> There???s a fairly long explanation I put up on stackoverflow to see if the unix gurus there might know, but so far there hasn???t even been an attempt at an answer.
> http://stackoverflow.com/questions/38384606/how-to-avoid-using-sudo-for-certain-programs-under-xrdp-in-raspbian
>

Ah, ok, your summary on SO covers it very well.

Is there a specific error message that you see when you run without the sudo?
For example, if you connect to the Pi display with xrdp, then run Squeak from
a terminal window, then I expect that you would see an error message in the
console window indicating that the pthread priority could not be set. And then
if you ran Squeak with sudo, there there would be no error. Is that right?

Dave
Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

timrowledge


> On 17-08-2016, at 6:05 PM, David T. Lewis <[hidden email]> wrote:
>
>
> Is there a specific error message that you see when you run without the sudo?
> For example, if you connect to the Pi display with xrdp, then run Squeak from
> a terminal window, then I expect that you would see an error message in the
> console window indicating that the pthread priority could not be set. And then
> if you ran Squeak with sudo, there there would be no error. Is that right?

Yup. It’s the usual message about "pthread_setschedparam failed: Operation not permitted”. What makes me laugh’ (for certain odd definitions of the word laugh) is that one remote display setup fails, one is fine - but the first is ok with the composition input class of window and the second isn’t. I suppose the fact that it appears to have been an issue for several years is amusing too.

I guess that in the way of these things there will be some very small config parameter to tweak to make it all wonderful, but it seems like a lot of people have been looking for it….

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Don't diddle code to make it faster; find a better algorithm.


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

fniephaus
In reply to this post by timrowledge
 

On Thu, Aug 18, 2016 at 1:00 AM tim Rowledge <[hidden email]> wrote:


> On 17-08-2016, at 3:34 PM, Fabio Niephaus <[hidden email]> wrote:
>
>
> On Thu, Aug 18, 2016 at 12:17 AM tim Rowledge <[hidden email]> wrote:
>
> A persistent problem with the PI vm is that users of xrdp need to wrap the start-up in ‘sudo -E’ because.. well, never mind, unix is just plain stupid. Whatever the issue, we can test for the problem of needing to use sudo by running a trivial test program in the shell script; all it does is try changing a thread priority and returns whether it worked or not.
>
> Just to make sure: it is not enough to create a squeak.conf as describe at [1] and reboot?

Unfortunately that doesn’t work when using xrdp for your remote display. It’s not even a strictly squeak issue - some googling will reveal a depressingly long list of problems to do with the same error. The Raspbian kernel; has no problems at all with squeak when using a direct display. It doesn’t have this particular problem with vnc - but it *does* have a fatal problem with trying to use the composition input window calls. Again, not a specifically squeak, raspbian nor ARM issue but a generic bit of nastiness in linux.

> Could you please give [2] a try on a raspi? Feedback is more than welcome as we are about to release Squeak 5.1 very soon.

As long as it’s made from the very latest source Eliot put out today it will be ok. *Except* I note that the HostWindowPlugin is missing from the plugins/int file in build.linuxARMv6/squeak.cog.spur and in fact after a quick look it is missing form all the linux related build directories that I checked. Whilst the plugin is not complete it does provide support for title and size aspects and so I’d recommend including it.

5.1 will be the basis of the next Pi Scratch release so I have several millions of reasons to help getting it right...

Shall we just open the vm with `sudo -E` on ARM platforms in the 5.1 release? This seems to be a reasonable solution for now. Usually, a user will be prompted to enter a password and we can explain why. Once we figure out a better solution, we can iterate :)

BTW: There's going to be an ARM-only bundle for the Squeak 5.1 release as well.

Best,
Fabio
 


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Don't sweat petty things....or pet sweaty things.


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

timrowledge


> On 18-08-2016, at 9:39 AM, Fabio Niephaus <[hidden email]> wrote:
>

>
> Shall we just open the vm with `sudo -E` on ARM platforms in the 5.1 release? This seems to be a reasonable solution for now. Usually, a user will be prompted to enter a password and we can explain why. Once we figure out a better solution, we can iterate :)

No, I don’t think that is a good way to go; it leaves files with root ownership etc. We need to include the little test program and use it and do the sudo thing only when needed.  For the Pi scratch stuff I use this snippet -

# find the latest vm
SQVM=`ls /usr/lib/squeak/5.0-20*/squeak|sort -r | head -1`
# and thus its directory
BIN=`/usr/bin/dirname $SQVM`
$BIN/TestPiPriority > /dev/null 2>&1
if [ $? -eq 1 ]; then
      WRAPPER="sudo -E”
fi

The problem is that you then need a secondary shell script to actually fire up squeak because (so far as I can work out) you can’t do
$WRAPPER LD_LIBRARY_PATH=“$PLUGINS:$SVMLLP:${LD_LIBRARY_PATH}” $GDB “$BIN/squeak” etc
And an attempt at
 LD_LIBRARY_PATH=“$PLUGINS:$SVMLLP:${LD_LIBRARY_PATH}” $WRAPPER $GDB “$BIN/squeak” etc
didn’t work either.

Anyone that knows how to solve this anything like ‘properly’ would be a real hero.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Oxymorons: Political science


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

timrowledge


> On 18-08-2016, at 10:02 AM, tim Rowledge <[hidden email]> wrote:
>
> Anyone that knows how to solve this anything like ‘properly’ would be a real hero.
>

Just on a wild guess I tried adding an xrdp.conf file in the style of the squeak.conf file suggested on https://github.com/OpenSmalltalk/opensmalltalk-vm/releases#linux but it made no difference :-(


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Fractured Idiom:- QUE SERA SERF - Life is feudal


Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

David T. Lewis
 
On Thu, Aug 18, 2016 at 10:10:43AM -0700, tim Rowledge wrote:
>
>
> > On 18-08-2016, at 10:02 AM, tim Rowledge <[hidden email]> wrote:
> >
> > Anyone that knows how to solve this anything like ???properly??? would be a real hero.
> >
>
> Just on a wild guess I tried adding an xrdp.conf file in the style of the squeak.conf file suggested on https://github.com/OpenSmalltalk/opensmalltalk-vm/releases#linux but it made no difference :-(
>

Try this and see if it works:

  http://florent.clairambault.fr/xrdp-and-the-ulimits-nofile-issue


Quoting from that post:

   You might have noticed for xrdp on Debian (but quite possibly with a
   lot of other Linux tools and other Linux distributions) the user limits
   (described in /etc/security/limits.conf) are not enforced. Which meant
   in my case that any session open with xrdp was opened with a max number
   of open files (nofile) set to 1024.
   
   To fix this, edit the file /etc/pam.d/common-session and add the following line:
   
        session    required   pam_limits.so


Dave
 
Reply | Threaded
Open this post in threaded view
|

Re: Pi vm builds need a priority checker applet

timrowledge


> On 18-08-2016, at 11:27 AM, David T. Lewis <[hidden email]> wrote:
> Try this and see if it works:
>
>  http://florent.clairambault.fr/xrdp-and-the-ulimits-nofile-issue

Good grief! It %^$#%ing works!

I knew it would be stupid-simple thing in the end but that really takes the biscuit, dunks it in spoon-dissolving tea and slurps it.

I did have a bit of ‘fun’ since I managed to delete the /etc/pam.d/common-session file. Guess how much you can do without that in place. Luckily I had a spare uSD card so I could reboot the Pi (not politely mind you because… no common-sessions file), mount the prior uSD, copy a file across to the right place, swap back to the original uSD card. Sigh.
Still, with it back in place and edited and the system rebooted(again) I can now start squeak without using sudo. Yippee! That means I can simplify the shell scripts and stop worrying about the TestPiPriority stuff.

Thank you very much Dave.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
If you think C++ is not overly complicated, just what is a protected abstract virtual base pure virtual private destructor and when was the last time you needed one? -- Tom Cargin

Reply | Threaded
Open this post in threaded view
|

Re: VM version naming (was Re: [Vm-dev] Pi vm builds need a priority checker applet)

fniephaus
In reply to this post by timrowledge
 
On Thu, Aug 18, 2016 at 1:06 AM tim Rowledge <[hidden email]> wrote:

Oh, something I spotted when looking at  https://github.com/OpenSmalltalk/opensmalltalk-vm/releases#linux as you suggested - the vm version numbering seems to be the old style from when we used the svn source tree. Unless I misunderstood we now use the github date string versioning (my latest vm from this afternoon is 5.0-201608171728 for example).

Do you mean it's weird that the version number still starts with "5.0-"?
 

If this has been changed then I need to know pretty urgently in order to fix the Pi scratch/squeak shell scripts!

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- Can't find log base two of 65536 without a calculator.


Reply | Threaded
Open this post in threaded view
|

Re: VM version naming (was Re: [Vm-dev] Pi vm builds need a priority checker applet)

timrowledge


> On 18-08-2016, at 3:52 PM, Fabio Niephaus <[hidden email]> wrote:
>
> On Thu, Aug 18, 2016 at 1:06 AM tim Rowledge <[hidden email]> wrote:
>
> Oh, something I spotted when looking at  https://github.com/OpenSmalltalk/opensmalltalk-vm/releases#linux as you suggested - the vm version numbering seems to be the old style from when we used the svn source tree. Unless I misunderstood we now use the github date string versioning (my latest vm from this afternoon is 5.0-201608171728 for example).
>
> Do you mean it's weird that the version number still starts with "5.0-“?

No, it’s the r3732 type numbering that concerns me. When one builds a new vm it will be, for example, 5.0-201608171728 with the date/time making the version data and an svn release number. Because we’re not using svn anymore...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: XM: Exclusive Maybe


Reply | Threaded
Open this post in threaded view
|

Re: VM version naming (was Re: [Vm-dev] Pi vm builds need a priority checker applet)

fniephaus
 
On Fri, Aug 19, 2016 at 12:59 AM tim Rowledge <[hidden email]> wrote:


> On 18-08-2016, at 3:52 PM, Fabio Niephaus <[hidden email]> wrote:
>
> On Thu, Aug 18, 2016 at 1:06 AM tim Rowledge <[hidden email]> wrote:
>
> Oh, something I spotted when looking at  https://github.com/OpenSmalltalk/opensmalltalk-vm/releases#linux as you suggested - the vm version numbering seems to be the old style from when we used the svn source tree. Unless I misunderstood we now use the github date string versioning (my latest vm from this afternoon is 5.0-201608171728 for example).
>
> Do you mean it's weird that the version number still starts with "5.0-“?

No, it’s the r3732 type numbering that concerns me. When one builds a new vm it will be, for example, 5.0-201608171728 with the date/time making the version data and an svn release number. Because we’re not using svn anymore...

The new vms should not contain svn release numbers anymore. Eliot recently [1] got rid of the rest I think.
Or are you saying that we should update the linked release notes?

 
tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: XM: Exclusive Maybe


Reply | Threaded
Open this post in threaded view
|

Re: VM version naming (was Re: [Vm-dev] Pi vm builds need a priority checker applet)

timrowledge


> On 18-08-2016, at 4:07 PM, Fabio Niephaus <[hidden email]> wrote:
> Or are you saying that we should update the linked release notes?

Yup; for example the page you referred me to a few messages back - https://github.com/OpenSmalltalk/opensmalltalk-vm/releases#linux - refers to r3732 which is going to confuse people later.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: LCD: Launch Cartridge Disk