Audio, Streams, and MailMessage question

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Audio, Streams, and MailMessage question

Brian Brown-2
Hello All,

        I am using SoundRecorder to record and I can use  
SoundRecorder>>storeWAVOnFileNamed: to create a file of the recorded  
sound without fail. The method #storeWAVOnFileNamed: uses  
SoundRecorder>>storeWAVSamplesOn: aStream to save the sampled sound  
to the FileStream. I would like to use that same method so I can get  
the WAV data without writing it to a file and reading it back out again.

I have tried the following:

        s := ReadWriteStream on: (ByteArray new: 5000).
        s binary.
        recorder recordedSound storeWAVSamplesOn: s.
        ^ s.

and then later, I send the  stream (s) to  
MailMessage>>addAttachmentFrom:withName:

The resultant email message looks fine and has an attached wav file  
with 0 bytes.


MailMessage>>addAttachmentFrom:withName: creates a MIMEDocument and  
sends #upToEnd to the Stream to get the data. Senders in the image  
usually pass in a FileStream, and I don't understand the difference  
as to why another kind of stream would not work.

Any help would be welcome!

- Brian


Reply | Threaded
Open this post in threaded view
|

Re: Audio, Streams, and MailMessage question

Hans-Martin Mosner
Brian Brown wrote:

>
> I have tried the following:
>
>     s := ReadWriteStream on: (ByteArray new: 5000).
>     s binary.
>     recorder recordedSound storeWAVSamplesOn: s.
>     ^ s.
>
> and then later, I send the  stream (s) to
> MailMessage>>addAttachmentFrom:withName:
>
> The resultant email message looks fine and has an attached wav file
> with 0 bytes.
>
>
> MailMessage>>addAttachmentFrom:withName: creates a MIMEDocument and
> sends #upToEnd to the Stream to get the data. Senders in the image
> usually pass in a FileStream, and I don't understand the difference as
> to why another kind of stream would not work.
>
You're probably not resetting the ReadWriteStream. As it is positioned
at the end after writing, reading up to end will return 0 bytes :-)

Reply | Threaded
Open this post in threaded view
|

Re: Audio, Streams, and MailMessage question

Brian Brown-2

On Aug 22, 2007, at 12:35 PM, Hans-Martin Mosner wrote:

> You're probably not resetting the ReadWriteStream. As it is positioned
> at the end after writing, reading up to end will return 0 bytes :-)
>

True! That worked as far as getting some data... unfortunately, the  
audio is just white noise!

If I write out the audio using #storeWAVOnFileNamed: at the same time  
I use #storeWAVSamplesOn: the file I write to disk is normal, but the  
other one is worthless.

If I look them with a binary editor, they are definitely different.

Any other ideas?

- Brian


Reply | Threaded
Open this post in threaded view
|

Re: Audio, Streams, and MailMessage question

Igor Stasenko
On 22/08/07, Brian Brown <[hidden email]> wrote:

>
> On Aug 22, 2007, at 12:35 PM, Hans-Martin Mosner wrote:
>
> > You're probably not resetting the ReadWriteStream. As it is positioned
> > at the end after writing, reading up to end will return 0 bytes :-)
> >
>
> True! That worked as far as getting some data... unfortunately, the
> audio is just white noise!
>

This can be signs of GC actions. Squeak GC is moving GC, so it
possible that buffer you using for capturing sound is moved in memory,
and stream doesn't aware of it and reads data from old region of
memory, which replaced by other data.



> If I write out the audio using #storeWAVOnFileNamed: at the same time
> I use #storeWAVSamplesOn: the file I write to disk is normal, but the
> other one is worthless.
>
> If I look them with a binary editor, they are definitely different.
>
> Any other ideas?
>
> - Brian
>
>
>


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Audio, Streams, and MailMessage question

johnmci
Twitch. Sound Recording primitives

I can't say I've heard anyone complain they are failing because there  
is GC activity.

Technically most VMs interface to the operating system sound api by  
allocation
platform specific structures, then usually record sound into a buffer  
area allocated
by the SoundPlugin or Operating system, when it's full (or whatever  
the condition is) the semaphore
setup by SoundRecorder>>startRecording is signaled by the  
SoundPlugin. Which
wakes up the task waiting for the VM to provide a buffer.

The SoundRecorder>>recordLoop then invokes
primRecordSamplesInto:startingAt:  which is responsible for moving  
the recorded sound
data from the platform specific area into the Smalltalk SoundBuffer  
instance.

At no time does the VM sound recording logic talk directly to any  
smalltalk object, and the
primRecordSamplesInto:startingAt should move the data in a monotonic  
process without any
GC interference.

I'd check in recordLoop exactly what the data is that is coming out  
of the primitive  It could be not what you think.

primRecordSamplesInto:startingAt:


Now the other issue is I'm not sure you identified the platform and  
VM you are using.
It's always possible if you have multiple input devices perhaps the  
one we are reading from
is not the one you think you are using.


Bugs in the last year or so:

> "workaround for OSS emulation on top
> on ALSA (on Linux environments)"
> (Delay forMilliseconds: 20) wait.

This code in 3.9  ""dgd" on April 4, 2006:"
would make the macintosh carbon and unix VM drop data since it was  
messing up the ability of the recording loop to
empty out the buffer, so you lost data. This resulted in only  
recording 50% of the actual data, so you recorded 20 seconds
of audio, but on playback got 10 seconds.

Crash

On the macintosh carbon (fixed) and unix VM (not fixed) the  
SoundPlugin logic would ask the recorder to stop, this would signal to
the os-x core audio to stop, and then it would cleanup the data  
structures. However because the core audio is running on multiple  
pthreads it was possible to have a race condition between the cleanup  
and the possible last write of data from the hardware abstractions  
layer which resulted in a crash. The solution was to check the data  
structures carefully before use and toss that last couple of bytes of  
data if needbe.

On Aug 23, 2007, at 1:38 AM, Igor Stasenko wrote:

> On 22/08/07, Brian Brown <[hidden email]> wrote:
>>
>> On Aug 22, 2007, at 12:35 PM, Hans-Martin Mosner wrote:
>>
>>> You're probably not resetting the ReadWriteStream. As it is  
>>> positioned
>>> at the end after writing, reading up to end will return 0 bytes :-)
>>>
>>
>> True! That worked as far as getting some data... unfortunately, the
>> audio is just white noise!
>>
>
> This can be signs of GC actions. Squeak GC is moving GC, so it
> possible that buffer you using for capturing sound is moved in memory,
> and stream doesn't aware of it and reads data from old region of
> memory, which replaced by other data.
>
>
>
>> If I write out the audio using #storeWAVOnFileNamed: at the same time
>> I use #storeWAVSamplesOn: the file I write to disk is normal, but the
>> other one is worthless.
>>
>> If I look them with a binary editor, they are definitely different.
>>
>> Any other ideas?
>>
>> - Brian
>>
>>
>>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===



Reply | Threaded
Open this post in threaded view
|

Re: Audio, Streams, and MailMessage question

Brian Brown-2

On Aug 23, 2007, at 3:20 AM, John M McIntosh wrote:

> Twitch. Sound Recording primitives
>

Ha! You should get that checked ;)


I'm not having any problems with sound buffers - the SoundRecorder  
instance is working fine - I can play and record without issues. The  
issue I'm having is in getting a wav version of the data without  
writing it to a file. If I write to a file using  
#storeWAVOnFileNamed: and read it again, it's fine. If I use  
#storeWAVSamplesOn: with a stream, the resultant buffer is not  
usable. The only difference I can see is in the type of Stream that  
is being passed into #storeWAVSamplesOn:, as that method is also  
called from #storeWAVOnFileNamed:


I'm setting my stream to binary, but I see that the binary message  
does distinctly different things between a FileStream and  
RWBinaryOrTextStream...

I'm stumped at this point, but I'm no expert on Streams either.

- Brian


> I can't say I've heard anyone complain they are failing because  
> there is GC activity.
>
> Technically most VMs interface to the operating system sound api by  
> allocation
> platform specific structures, then usually record sound into a  
> buffer area allocated
> by the SoundPlugin or Operating system, when it's full (or whatever  
> the condition is) the semaphore
> setup by SoundRecorder>>startRecording is signaled by the  
> SoundPlugin. Which
> wakes up the task waiting for the VM to provide a buffer.
>
> The SoundRecorder>>recordLoop then invokes
> primRecordSamplesInto:startingAt:  which is responsible for moving  
> the recorded sound
> data from the platform specific area into the Smalltalk SoundBuffer  
> instance.
>
> At no time does the VM sound recording logic talk directly to any  
> smalltalk object, and the
> primRecordSamplesInto:startingAt should move the data in a  
> monotonic process without any
> GC interference.
>
> I'd check in recordLoop exactly what the data is that is coming out  
> of the primitive  It could be not what you think.
>
> primRecordSamplesInto:startingAt:
>
>
> Now the other issue is I'm not sure you identified the platform and  
> VM you are using.
> It's always possible if you have multiple input devices perhaps the  
> one we are reading from
> is not the one you think you are using.
>
>
> Bugs in the last year or so:
>
>> "workaround for OSS emulation on top
>> on ALSA (on Linux environments)"
>> (Delay forMilliseconds: 20) wait.
>
> This code in 3.9  ""dgd" on April 4, 2006:"
> would make the macintosh carbon and unix VM drop data since it was  
> messing up the ability of the recording loop to
> empty out the buffer, so you lost data. This resulted in only  
> recording 50% of the actual data, so you recorded 20 seconds
> of audio, but on playback got 10 seconds.
>
> Crash
>
> On the macintosh carbon (fixed) and unix VM (not fixed) the  
> SoundPlugin logic would ask the recorder to stop, this would signal to
> the os-x core audio to stop, and then it would cleanup the data  
> structures. However because the core audio is running on multiple  
> pthreads it was possible to have a race condition between the  
> cleanup and the possible last write of data from the hardware  
> abstractions layer which resulted in a crash. The solution was to  
> check the data structures carefully before use and toss that last  
> couple of bytes of data if needbe.
>
> On Aug 23, 2007, at 1:38 AM, Igor Stasenko wrote:
>
>> On 22/08/07, Brian Brown <[hidden email]> wrote:
>>>
>>> On Aug 22, 2007, at 12:35 PM, Hans-Martin Mosner wrote:
>>>
>>>> You're probably not resetting the ReadWriteStream. As it is  
>>>> positioned
>>>> at the end after writing, reading up to end will return 0 bytes :-)
>>>>
>>>
>>> True! That worked as far as getting some data... unfortunately, the
>>> audio is just white noise!
>>>
>>
>> This can be signs of GC actions. Squeak GC is moving GC, so it
>> possible that buffer you using for capturing sound is moved in  
>> memory,
>> and stream doesn't aware of it and reads data from old region of
>> memory, which replaced by other data.
>>
>>
>>
>>> If I write out the audio using #storeWAVOnFileNamed: at the same  
>>> time
>>> I use #storeWAVSamplesOn: the file I write to disk is normal, but  
>>> the
>>> other one is worthless.
>>>
>>> If I look them with a binary editor, they are definitely different.
>>>
>>> Any other ideas?
>>>
>>> - Brian
>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
> --
> ======================================================================
> =====
> John M. McIntosh <[hidden email]>
> Corporate Smalltalk Consulting Ltd.  http://
> www.smalltalkconsulting.com
> ======================================================================
> =====
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Audio, Streams, and MailMessage question

Andreas.Raab
Brian Brown wrote:
> I'm setting my stream to binary, but I see that the binary message does
> distinctly different things between a FileStream and
> RWBinaryOrTextStream...
>
> I'm stumped at this point, but I'm no expert on Streams either.

It's rather simple. If you look at the actual code (you have stepped it
in the debugger to see what's happening or at least browsed to places
that implement it, have you?) then you'll find a couple of places that
look like:

   (aBinaryStream isKindOf: StandardFileStream)
     ifTrue:["optimization for files: write sound buffer directly to file"
       aBinaryStream next: (samples size // 2) putAll: samples
startingAt: 1]  "size in words"
     ifFalse:[  "for non-file streams:"
       1 to: samples monoSampleCount do: [:i | aBinaryStream int16:
(samples at: i)]].

There is your difference, right there.

Cheers,
   - Andreas