Sound mixing makes nasty noises

Ha, ran the script in a terminal and on each piano keypress I get:
sound_Start(default)
soundStart: snd_add_pcm_handler: Function not implemented.

... Handling two mouses and two keyboards is quite confusing
... Have to run to pick up my wife

Hoping for some enlightenment tomorrow :-)

Cheers,

Herbert

Am 24.12.2020 um 22:05 schrieb Herbert König:

> On 2020-12-24, at 1:21 PM, Herbert König <[hidden email]> wrote:
>
> Ha, ran the script in a terminal and on each piano keypress I get:
> sound_Start(default)
> soundStart: snd_add_pcm_handler: Function not implemented.

That's an error to do with some asynchronous handler stuff in alsa. I have no idea why it would fail.

All in all this sound stuff is getting very, very, frustrating


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: NOP: Randomize the PSW and then branch

On 25/12/20 2:35 am, Herbert König wrote:
> I tried the -vmsound alsa and pulse but I'm not sure If I used the right
> place in the script. I tried editing the last line to:
> exec "${VM}" "-vm-sound-alsa" "${IMAGE}"  (and pulse and w/o the "" but
> my Linux fu is not enough to know if that's ok.
ALSA is a bit long in the tooth. Pulse is the preferred audio server
these days.

You could try running Squeak with padsp which mixes the OSS API calls
from squeak vm through Pulse audio service:

  exec padsp "${VM}" ...

The GUI mixer panel is pavucontrol (from pavucontrol package).

HTH .. Subbu

tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build/squeak -vm-sound-pulse myimage

> On 2020-12-24, at 1:21 PM, Herbert König wrote:

>

> Ha, ran the script in a terminal and on each piano keypress I get:

> sound_Start(default)

> soundStart: snd_add_pcm_handler: Function not implemented.

That's an error to do with some asynchronous handler stuff in alsa. I have no idea why it would fail.

All in all this sound stuff is getting very, very, frustrating

tim

--

tim Rowledge; [hidden email]; http://www.rowledge.org/tim

Strange OpCodes: NOP: Randomize the PSW and then branch

Am 25.12.2020 um 05:41 schrieb tim Rowledge: Same on my RPi4. Audio is working except Squeak and Scratch1, same
error. So I tried my Pi2 which I haven't updated in times but mounted on
a RPi touch display -->no sound at all and no Volume control in the
taskbar.

Since the RPI2 I use them for Audio recording and playing and audio
measurements. They all saw an IQaudio DAC + and Digiamp so maybe that's
part of the problem. Everything always worked out of the box, just I
never used Squeak.

Maybe it has to do with the recent switch from Alsa to Pulse Audio (just
names for me) but the Pi2 should not have seen this.

Anyway I got some time so I will check the messages from Subbu and Bruce
and then just do the reverse. Use Squeak audio on Windows (Just checked,
5.3 has the nasty effects) and record with my soundcard on a Pi.

It would be cool if someone can implement Your suggestion on how to
write Squeak's audio output buffers to a file because then I could
discern if the whole mess gets created by Squeak or if the OS plays a
role too.

Cheers,

Herbert

On 25/12/20 6:49 pm, Herbert König wrote:
> It would be cool if someone can implement Your suggestion on how to
> write Squeak's audio output buffers to a file because then I could
> discern if the whole mess gets created by Squeak or if the OS plays a
> role too.

You may use parec (sound capture to file) and paplay (file to sound
output) to test the audio subsystem of the OS to rule out any audio
driver or mixer configuration problems [1]. The corresponding ALSA
utilities are arecord and aplay. The mixer is alsamixer.

padsp acts as a relay between the older /dev/dsp (audio device under
OSS) and the newer /dev/snd/* (audio i/o devices).

[1] https://manpages.debian.org/testing/pulseaudio-utils/pacat.1.en.html

HTH .. Subbu

Hi Subbu,

thanks, that did it. Squeak playing Audio with the problems described,
Scratch one through the menu still w/o audio (as expected) but both
showing up in the pavucontrol. Except that now I have a crackling noise
when using the volume control (Crossfade still being a secret to some
audio implementors, really :-)

So on to electrical recording and then try to implement Tim's suggestion
on recording the buffers from Squeak.

Cheers,

Herbert

Am 25.12.2020 um 13:02 schrieb K K Subbu:

Hey you seem to know Linux's audio stuff.

Thanks!

Herbert

Am 25.12.2020 um 14:54 schrieb K K Subbu:

I just downloaded Squeak on my up to date Linux Mint computer (a Core I 3 from 2009) and right at the start (before the message you report) I get the message copied below.

No sound with vm-sound-alsa but sound with vm-sound-pulse and always the same message. But it sounds like high latency and not as bad as on the Pi (with the keyboard). When waggling the keyboard Squeak goes from 5% to 56% CPU. That seems a lot to me.

Cheers,

Herbert

"Using /home/herbert/Schreibtisch/Squeak5.3alpha-19256-64bit-201911012148-Linux/bin/squeak...
pthread_setschedparam failed: Operation not permitted
This VM uses a separate heartbeat thread to update its internal clock
and handle events.  For best operation, this thread should run at a
higher priority, however the VM was unable to change the priority.  The
effect is that heavily loaded systems may experience some latency
issues.  If this occurs, please create the appropriate configuration
file in /etc/security/limits.d/ as shown below:

cat <<END | sudo tee /etc/security/limits.d/squeak.conf
*      hard    rtprio  2
*      soft    rtprio  2
END

and report to the squeak mailing list whether this improves behaviour.

You will need to log out and log back in for the limits to take effect.
For more information please see
https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
"

So another data point.

Linux Mint 20 (so basically Ubuntu) on x86-64.  Happens to be a MacBook Air but I don't think that's important.

My normal install only build vm-sound-OSS.  And with that there is no sound and a complaint in the terminal about /dev/dsp not existing.  Which is true, /dev/dsp does not exist.

But if install libpluse-dev before I build the VM then I also get vm-sound-pluse.

tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build/squeak -vm-sound-pulse myimage

Good news - I get sound!

Bad news - Well it seems a bit drunk in that it pauses about every second to two.  It acts like a buffering issue.  It is slightly worse without doing the RT priority limits.conf file.

I'll also poke at this a bit.

cheers

bruce

25 December 2020 05:41 tim Rowledge [hidden email] wrote:

> On 2020-12-24, at 1:21 PM, Herbert König wrote:

>

> Ha, ran the script in a terminal and on each piano keypress I get:

> sound_Start(default)

> soundStart: snd_add_pcm_handler: Function not implemented.

That's an error to do with some asynchronous handler stuff in alsa. I have no idea why it would fail.

All in all this sound stuff is getting very, very, frustrating

tim

--

tim Rowledge; [hidden email]; http://www.rowledge.org/tim

Strange OpCodes: NOP: Randomize the PSW and then branch

I'm glad a few people around here do - there seems to be a bewildering mix of things out there.

Looking at the assorted C code for soundplugin, sqUnixSound, sqUnixSoundALSA & SsqUnixSoundPulseAudio I see a bunch of places where pointer types don't agree and just maybe there are some problems there. The worst potential for pointer issues is avoided in practice because we never actually pass any value for startIndex other than 0 (the '1' used in all the Smalltalk is pre-decremented in SoundPlugin>primitiveSoundPlaySamples)

Both the ALSA & pulseaudio files appear to connect to the semaphore in a reasonable manner. Ther pulseaudio code is clearly long untouched and in need of a lot of tidying up and the addition of a number of new routines like sound_GetDefaultSoundPlayer - try using that prim and it just dumps you VM with extreme prejudice!

I think one of the really puzzling things right now is how does my older Pi setup *not* make such terrible noises? The scratch image & vm involved are *identical* to the ones on the newer-but-still-alsa Pi.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
When flying inverted, remember that down is up and up is expensive

I've managed to record some samples with parecord on the pulseaudio Pi. As usual, the man page for parecord etc was about as helpful as a chocolate teapot in a sauna, but some flailing around on google eventually found a page with something that worked.

 parecord --channels=1 -d alsa_output.platform-bcm2835_audio.analog-mono.2.monitor testsound.wav

records the output of the headphone socket channel until you ctrl-C. I also tried with the HDMI output in case this is all down to some bug in the headphpone output - which has happened in the past. When playing the two samples I couldn't hear any difference, so I think that saves one level of nightmare.

I'm loathe to dump a meg-and-a-half of horrible noise file in list mail, or indeed on the swiki - so you ought to be able to download from https://mega.nz/folder/iN1wWajL#SkljQIWhHP8c6McE9HDwDg 

Currently there are two files, the Pi4-pulse-1.wav is the noise made by my code snippet plus some waggling of mouse over the normal pianokeyboardmorph, and -2 is waggling the mouse over the scratch note chooser keyboard morph.

I'd add files for the alsa Pi but I can't make arecord work at all. As usual, making sense of a man page is ridiculous.

I've tried a variety -

arecord -d 5 -f cd -D -t wav output Pi4-alsa-1.wav
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM -t
arecord: main:828: audio open error: No such file or directory

arecord -d 5 -f cd -D output -t wav  Pi4-alsa-1.wav
arecord: main:828: audio open error: No such file or directory

 arecord -d 5 -f cd  -t wav  Pi4-alsa-1.wav
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM input
arecord: main:828: audio open error: No such file or directory

arecord -d 5 -f cd  -t wav  foo
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM input
arecord: main:828: audio open error: No such file or directory

arecord -d 5 -D output  -t wav  Pi4-alsa-1.wav
arecord: main:828: audio open error: No such file or directory

 arecord -d 5 -D output  -t wav  ./Pi4-alsa-1.wav
arecord: main:828: audio open error: No such file or directory

 arecord -d 5 foo.wav
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM input
arecord: main:828: audio open error: No such file or directory

I even found the arecord.c sources and looked for the error message - of course, there is no reasonable way to work out what file is not there...

Advice on a suitable command most welcome.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: BBL: Branch on Burned out Light

On 26/12/20 4:09 am, tim Rowledge wrote:
> I'd add files for the alsa Pi but I can't make arecord work at all. As usual, making sense of a man page is ridiculous.

;-). A man(ual) page is detailed spec. Reading all options can be quite
intimidating. I just skip to the examples section in the beginning. I
only use options to override defaults.

> arecord -d 5 -f cd -D -t wav output Pi4-alsa-1.wav
> ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM -t
> arecord: main:828: audio open error: No such file or directory

-D requires an argument.

$ arecord -d 5 -f cd Pi4-alsa-1.wav

will sample mic at 44khz (16-bit little endian) for 5 seconds and store
it in wav format file (-t wav is the default anyway).

> arecord -d 5 -f cd -D output -t wav  Pi4-alsa-1.wav
> arecord: main:828: audio open error: No such file or directory

The "output" argument to -D is not a valid audio input device. What
valid audio input devices does "arecord -l" show? You could also use
alsamixer TUI to see all valid input/output devices, check their
gain/vol settings and test recording and playback.

The default device is not available (bad driver?) or misconfigured. Try
listing audio devices with arecord -l and then pick the appropriate name
to use with -D.

HTH .. Subbu

On 26/12/20 1:02 am, tim Rowledge wrote:
> I think one of the really puzzling things right now is how does my
> older Pi setup*not*  make such terrible noises? The scratch image &
> vm involved are*identical*  to the ones on the newer-but-still-alsa
> Pi.

Audio drivers and servers can make a difference between distro versions.

Didn't Raspbian switch to pulse audio server to pulse in their recent
release [1]?

[1]
https://www.raspberrypi.org/blog/new-raspberry-pi-os-release-december-2020/

Regards .. Subbu

Tim, thanks for the recording.

No idea if that happens inside Squeak but basically clipping is not
working. In the image instead of going from positive full-scale to
negative full-scale the signal should just stay at the resp. full-scale.
Instead of clipping 32767 + 1 to 32767  the code uses 32768 as being -
full-scale (wraparound).  See pulse1PosWrap.png. For negative full-scale
the symmetric thing happens. The signal should stay at full-scale there.

mixSampleCount: n into: aSoundBuffer startingAt: startIndex leftVol:
leftVol rightVol: rightVol  seems to do the right thing:
         leftVol > 0 ifTrue: [
             i := (2 * outIndex) - 1.
             s := (aSoundBuffer at: i) + ((sample * leftVol) //
ScaleFactor).
             s >  32767 ifTrue: [s :=  32767].  "clipping!"
             s < -32767 ifTrue: [s := -32767].  "clipping!"
             aSoundBuffer at: i put: s].
except we can go to 32767 and -32768 IIRC but that's no problem.

If you look at Pi4-pulse-1.wav second 5.2352 it seems there's multiple
wraparounds going on. See pulse1MultipleWrap.
For this I have no explanation because the signal if it were unclipped
looks like going to about 1.5 * full-scale.

pulse2ClipAndWrap shows that somewhere (red mark) the normal clipping is
going on (sounds bad if this strong, that's why I suggested a limiter)
and close to this the wrapping is going on. Maybe there are places where
the clipping is taking something >32767 as max.

Or (a wild idea):
SoundBuffer stores 16 bit unsigned quantities. So if a signal is first
put into a sound buffer w/o clipping and only the mixing does the
clipping and that happens in several places (several sounds generated
independently) that could explain the wrapping. That's why I gave up
when I saw that SoundBuffer only uses 16 bits.

Here Eliot's suggestion from Dec 18:

"But adding a signed 32-bit, or even a signed 64-bit collection for
mixing would be straight forward, except for ensuring it mixes down to
16 bits. A signed 32-bit collection would allow 32k 16-bit sources to be
additively mixed without clipping, right? So an underlying 64-bit
collection is unnecessary. So a MixedSound that adds into an underlying
32-bit signed sequence, and updates a count of added sounds, and does
interpolation when adding sounds of different sample rates, and outputs
16-bit samples derived by dividing the 32-bit sums by the count of added
sounds would be a solution right? "

could make sense. I'd suggest using 32 Bit Float. Seeing loudness being
0.0 to 1.0 and scale in Envelope having Float values > 1 (I see even 25)
I hope that this already happens. But looking at e.g. FMSound's class
var SineTable I see it is a SoundBuffer, on which we operate with float
operations and then we must go back to 16 Bit unsigned to store the
result. Lots of room for human error in this.

All audio precessing (incl. mixing) should be done in Float (usually
normalized to +-1.0) and then use a limiter and then go to 16 bit sound
buffers. Or better use 32 bit Float for output to the OS too. Usually AD
and DA conversion is done with 24 bit by moderately cheap audio cards so
I guess most OSses use 32 bit float. But that's just a guess.



Cheers,

Herbert




Am 25.12.2020 um 23:39 schrieb tim Rowledge:
> I've managed to record some samples with parecord on the pulseaudio Pi. As usual, the man page for parecord etc was about as helpful as a chocolate teapot in a sauna, but some flailing around on google eventually found a page with something that worked.
>
......

So another data point.

Linux Mint 20 (so basically Ubuntu) on x86-64.  Happens to be a MacBook Air but I don't think that's important.

My normal install only build vm-sound-OSS.  And with that there is no sound and a complaint in the terminal about /dev/dsp not existing.  Which is true, /dev/dsp does not exist.

But if install libpluse-dev before I build the VM then I also get vm-sound-pluse.

tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build/squeak -vm-sound-pulse myimage

Good news - I get sound!

Bad news - Well it seems a bit drunk in that it pauses about every second to two.  It acts like a buffering issue.  It is slightly worse without doing the RT priority limits.conf file.

I'll also poke at this a bit.

cheers

bruce

25 December 2020 05:41 tim Rowledge <[hidden email]> wrote:

> On 2020-12-24, at 1:21 PM, Herbert König wrote:

>

> Ha, ran the script in a terminal and on each piano keypress I get:

> sound_Start(default)

> soundStart: snd_add_pcm_handler: Function not implemented.

That's an error to do with some asynchronous handler stuff in alsa. I have no idea why it would fail.

All in all this sound stuff is getting very, very, frustrating

tim

--

tim Rowledge; [hidden email]; http://www.rowledge.org/tim

Strange OpCodes: NOP: Randomize the PSW and then branch

<>

On 26/12/20 1:02 am, tim Rowledge wrote:

> I think one of the really puzzling things right now is how does my

> older Pi setup*not* make such terrible noises? The scratch image &

> vm involved are*identical* to the ones on the newer-but-still-alsa

> Pi.

Audio drivers and servers can make a difference between distro versions.

Didn't Raspbian switch to pulse audio server to pulse in their recent

release [1]?

[1]

https://www.raspberrypi.org/blog/new-raspberry-pi-os-release-december-2020/

Regards .. Subbu

On Fri, Dec 25, 2020 at 11:32:52AM -0800, tim Rowledge wrote:
Tim, I adapted your earlier code example as follows:

        volume := 0.30.
        { 55 . 60 . 64 }
                collect: [ :e |
                        FMSound new
                                setPitch: (AbstractSound pitchForMIDIKey: e)
                                dur: 5
                                loudness: volume]
                thenDo: [:s |  s play. (Delay forSeconds: 1) wait ].


This plays a major triad, which makes it very easy to hear the clipping
distortion. On my Linux PC with pulse audio, I get audible clipping
distortion when volume in the snippet above is set to anything higher
than 0.30. Results are very similar regardless of the VM and the image
version. I tried Squeak 3.8, Squeak trunk, interpreter VM, Spur64 VM,
and SqueakJS on Chrome.

Maybe you can compare this on Raspbery Pi and see if you get the same
result.

Dave

> On 2020-12-25, at 10:28 PM, K K Subbu <[hidden email]> wrote:
>
> On 26/12/20 4:09 am, tim Rowledge wrote:
>> I'd add files for the alsa Pi but I can't make arecord work at all. As usual, making sense of a man page is ridiculous.
>
> ;-). A man(ual) page is detailed spec. Reading all options can be quite intimidating. I just skip to the examples section in the beginning. I only use options to override defaults.

Me too - except it always seems that the examples section has been deleted when things are really confusing. I claim that this is all part of a Vast Global Conspiracy To Confuse Us. It's working.


>
> $ arecord -d 5 -f cd Pi4-alsa-1.wav
>
> will sample mic at 44khz (16-bit little endian) for 5 seconds and store it in wav format file (-t wav is the default anyway).

Yeah, I saw that example somewhere but it returns the same old  "arecord: main:828: audio open error: No such file or directory" error as the others.

All the other cases I posted were as recommended on assorted pages, which is terribly frustrating. I think maybe my alsa-Pi has got some wider alsa setup issue - which is strange since I haven't ever deliberately done any sound stuff on this particular unit and so it should be completely default. None of the examples to test the sound seem to work  - 'sudo aplay /usr/share/sounds/alsa/Front_Center.wav' makes no sound at all. And yet Squeak does make faint and feeble (scratchy) noises ... totally nuts.

I guess I can try a totally fresh load-up on yet another Pi to compare.

>
> The "output" argument to -D is not a valid audio input device. What valid audio input devices does "arecord -l" show?

Well, 'arecord -l' shows no devices, which isn't surprising on a bare Pi. But this is supposed to be recording what gets sent to the headphone output, so all the examples I've seen have referred to things like 'output' and 'default' and 'hw:1,0' etc etc. The parecord on the pulse-Pi had no problem.

OK, I'll put this email aside for a moment while I fire up the 'very old alsa Pi' to see if it can do better.

Later - the old alsa-Pi system (that 2017 version OS) plays sound from Scratch perfectly well, and even when playing a looped sample guitar I can thrash up and down the pianokeyboard without glitches. I can bang on the 'normal' pianokeyboardmorph without scratchy-glitches. My code snippet plays fine even with volume set to 100% with either a plan FMSound or an organ1 sound (that adds the envelope stuff to the mix). The test samples stuff ' aplay /usr/share/sounds/alsa/Front_Center.wav' works fine and clear.

However, I can't record the headphone output with arecord which makes it difficult to share. I bet there's some magic incantation to make the headphone out channel appear as an input.

So - 4 y.o. Pi system makes nice sounds. iMac makes nice sounds. Newer alsa Pi makes nasty sounds but may be borked in other ways. New pulse-Pi makes nasty sounds. Herbert has shown convincing evidence of bad clipping occurring. Dave's chord example makes graunchy noises on all machines *except iMac*, which is interesting, and if the volume is increased even the mac makes graunch.

I guess this implies we need to check the clipping code at the very least. And I've already mentioned that the pulse sound driver needs a bunch of additional code for new stuff. Such fun!

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
The downside of being better than everyone else is that people tend to assume you're pretentious

> On 2020-12-26, at 4:57 AM, Herbert König <[hidden email]> wrote:
>
> Tim, thanks for the recording.
>
> No idea if that happens inside Squeak but basically clipping is not
> working. In the image instead of going from positive full-scale to
> negative full-scale the signal should just stay at the resp. full-scale.
> Instead of clipping 32767 + 1 to 32767  the code uses 32768 as being -
> full-scale (wraparound).  See pulse1PosWrap.png.

Staring at the generated C code it looks possibly suspicious that we have

short int *aSoundBuffer;
sqInt s;
                if (leftVol > 0) {
                        i = (2 * sliceIndex) - 1;
                        s = (aSoundBuffer[i]) + ((sample * leftVol) / ScaleFactor);
                        if (s > 0x7FFF) {
                                s = 0x7FFF;
                        }
                        if (s < -32767) {
                                s = -32767;
                        }
                        aSoundBuffer[i] = s;
                }
But surely no C compiler screws that up these days?

I added printfs to show when over/underflow happened and indeed it happens - duh.

I've compared latest and ancient (as in ~2016 vintage) copies of SoundPlugin.c, SoundGenerationPlugin.c and sqUnixSoundALSA.c and found nothing that looks anything other than trivial changes - dates, some macros, etc.

There doesn't seem to be any place other than primitiveMixFMSound() that is involved in clipping sample values. None of the lowest level send-to-OS code looks to do any sort of casting or limiting.

I've even just for grins tried limiting the 's' value with a s & 0x3FFF just to see what happens. It sounds a bit strange but also seems to remove the clicks, so maybe there is something in it.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: BYEBYE: Store in Write-Only Storage

I searched all method sources for 3276 and only found one suspicious
place in :
adjustVolumeTo: vol overMSecs: mSecs
     "Adjust the volume of this sound to the given volume, a number in
the range [0.0..1.0], over the given number of milliseconds. The volume
will be changed a little bit on each sample until the desired volume is
reached."

     | newScaledVol |

     self flag: #bob.        "I removed the upper limit to allow making
sounds louder. hmm..."

     newScaledVol := (32768.0 * vol) truncated.
"<------------------------------- might exceed 32767 if vol = 1.0 ->
wrap not clip"
     newScaledVol = scaledVol ifTrue: [^ self].

but changing that to 32767 didn't help. Also I didn't find the version
where Bob had not yet removed the limit (my oldest image being 3.6) so
that is most probably not the culprit. Also I looked at a lot of in
image code but didn't find anything.

Next step is to try to record the output before it goes to the OS and
see if the image or some primitive creates the problem.

Cheers,

Herbert



Am 27.12.2020 um 01:57 schrieb tim Rowledge: