Music beat detection and 60FPS graphics on Pharo?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Music beat detection and 60FPS graphics on Pharo?

philippeback
I'd like to do something like this: https://www.youtube.com/watch?v=vL7D4eU0lYE

to showcase FFI, Bloc, and the Pharo liveliness.

And once I get the basics working, level up in getting it working in a VR headset.

Is there anyone having a binding to a library for dealing with the sound part, like FMod or other things like that?

I see this as a medium term project, so, like I can get it working by the end of the year.

TIA

Phil
Reply | Threaded
Open this post in threaded view
|

Re: Music beat detection and 60FPS graphics on Pharo?

NorbertHartl
I think that should be easy. I did something like that for a club installation in the early 2000. Squeak had line-in sound and a FFT library. Combined with Morphic a super tool for doing fun stuff.

Norbert
Am 14.10.2017 um 10:01 schrieb "[hidden email]" <[hidden email]>:

I'd like to do something like this: https://www.youtube.com/watch?v=vL7D4eU0lYE

to showcase FFI, Bloc, and the Pharo liveliness.

And once I get the basics working, level up in getting it working in a VR headset.

Is there anyone having a binding to a library for dealing with the sound part, like FMod or other things like that?

I see this as a medium term project, so, like I can get it working by the end of the year.

TIA

Phil
Reply | Threaded
Open this post in threaded view
|

Re: Music beat detection and 60FPS graphics on Pharo?

Sven Van Caekenberghe-2


> On 14 Oct 2017, at 11:05, Norbert Hartl <[hidden email]> wrote:
>
> I think that should be easy. I did something like that for a club installation in the early 2000. Squeak had line-in sound and a FFT library. Combined with Morphic a super tool for doing fun stuff.

Picturing Norbert as a DJ ...

> Norbert
> Am 14.10.2017 um 10:01 schrieb "[hidden email]" <[hidden email]>:
>
>> I'd like to do something like this: https://www.youtube.com/watch?v=vL7D4eU0lYE
>>
>> to showcase FFI, Bloc, and the Pharo liveliness.
>>
>> And once I get the basics working, level up in getting it working in a VR headset.
>>
>> Is there anyone having a binding to a library for dealing with the sound part, like FMod or other things like that?
>>
>> I see this as a medium term project, so, like I can get it working by the end of the year.
>>
>> TIA
>>
>> Phil


Reply | Threaded
Open this post in threaded view
|

Re: Music beat detection and 60FPS graphics on Pharo?

NorbertHartl

> Am 14.10.2017 um 14:22 schrieb Sven Van Caekenberghe <[hidden email]>:
>
>
>
>> On 14 Oct 2017, at 11:05, Norbert Hartl <[hidden email]> wrote:
>>
>> I think that should be easy. I did something like that for a club installation in the early 2000. Squeak had line-in sound and a FFT library. Combined with Morphic a super tool for doing fun stuff.
>
> Picturing Norbert as a DJ …
>
It's VJ ;)

Norbert



>> Norbert
>> Am 14.10.2017 um 10:01 schrieb "[hidden email]" <[hidden email]>:
>>
>>> I'd like to do something like this: https://www.youtube.com/watch?v=vL7D4eU0lYE
>>>
>>> to showcase FFI, Bloc, and the Pharo liveliness.
>>>
>>> And once I get the basics working, level up in getting it working in a VR headset.
>>>
>>> Is there anyone having a binding to a library for dealing with the sound part, like FMod or other things like that?
>>>
>>> I see this as a medium term project, so, like I can get it working by the end of the year.
>>>
>>> TIA
>>>
>>> Phil

Reply | Threaded
Open this post in threaded view
|

Re: Music beat detection and 60FPS graphics on Pharo?

Clément Béra
In reply to this post by philippeback
Hi Phil,

I did that using Ronie's OpenAL binding and Merwann's wav parser back in 2014-2015:

MCSmalltalkhubRepository
owner: 'ronsaldo'
project: 'OpenAL'

MCSmalltalkhubRepository
owner: 'MerwanOuddane'
project: 'WAVParser'

I did things such as (build yourself something derived from this code I did not check if it works):

MyClass>>example
WAVParser wavFromStream: ('resources/music/spellNoise.wav' asFileReference readStream binary readStream).
contextAttributes := ALContextAttributes new.
device := OpenALC openDefaultDevice.
context := device createContext: contextAttributes asList.
context process.
context currentDuring: [
"Create the buffer"
buffer := self makeBufferFromHeader: wav header data: wav data asByteArray.
"Create the source"
source := OpenAL genSource.
OpenAL sourcei: source param: AL_BUFFER value: buffer.
"Play the source"
OpenAL sourcePlay: source.
"Play for sometime "
(Delay forSeconds: wav header soundDuration) wait.
"Delete the source and the buffer"
OpenAL deleteSource: source;
deleteBuffer: buffer
].
OpenALC nullCurrentContext.
context destroy.
device close.

MyClass>>makeBufferFromHeader: header data: data
| buffer |
buffer := OpenAL genBuffer.
OpenAL bufferData: buffer format: (self readFormatFromHeader: header) data: data size: data size freq: header sampleRate.
^ buffer

MyClass>>readFormatFromHeader: header
^ header channels = 1
ifTrue: [ 
header bitsPerSample = 8
ifTrue: [ AL_FORMAT_MONO8 ]
ifFalse: [ AL_FORMAT_MONO16 ] ]
ifFalse: [ 
header bitsPerSample = 8
ifTrue: [ AL_FORMAT_STEREO8 ]
ifFalse: [ AL_FORMAT_STEREO16 ] ]

I am pretty sure with a little work those things could work again. I have never succeeded in playing mp3 files from Pharo though (but I'm on mac and I remembered trying solutions that did not work on Mac but may work on other OS).

Have fun guys :-)

On Sat, Oct 14, 2017 at 10:01 AM, [hidden email] <[hidden email]> wrote:
I'd like to do something like this: https://www.youtube.com/watch?v=vL7D4eU0lYE

to showcase FFI, Bloc, and the Pharo liveliness.

And once I get the basics working, level up in getting it working in a VR headset.

Is there anyone having a binding to a library for dealing with the sound part, like FMod or other things like that?

I see this as a medium term project, so, like I can get it working by the end of the year.

TIA

Phil



--
Clément Béra
Pharo consortium engineer
Bâtiment B 40, avenue Halley 59650 Villeneuve d'Ascq
Reply | Threaded
Open this post in threaded view
|

Re: Music beat detection and 60FPS graphics on Pharo?

philippeback
Thanks, I'll try that out!

Phil

On Sat, Oct 14, 2017 at 5:36 PM, Clément Bera <[hidden email]> wrote:
Hi Phil,

I did that using Ronie's OpenAL binding and Merwann's wav parser back in 2014-2015:

MCSmalltalkhubRepository
owner: 'ronsaldo'
project: 'OpenAL'

MCSmalltalkhubRepository
owner: 'MerwanOuddane'
project: 'WAVParser'

I did things such as (build yourself something derived from this code I did not check if it works):

MyClass>>example
WAVParser wavFromStream: ('resources/music/spellNoise.wav' asFileReference readStream binary readStream).
contextAttributes := ALContextAttributes new.
device := OpenALC openDefaultDevice.
context := device createContext: contextAttributes asList.
context process.
context currentDuring: [
"Create the buffer"
buffer := self makeBufferFromHeader: wav header data: wav data asByteArray.
"Create the source"
source := OpenAL genSource.
OpenAL sourcei: source param: AL_BUFFER value: buffer.
"Play the source"
OpenAL sourcePlay: source.
"Play for sometime "
(Delay forSeconds: wav header soundDuration) wait.
"Delete the source and the buffer"
OpenAL deleteSource: source;
deleteBuffer: buffer
].
OpenALC nullCurrentContext.
context destroy.
device close.

MyClass>>makeBufferFromHeader: header data: data
| buffer |
buffer := OpenAL genBuffer.
OpenAL bufferData: buffer format: (self readFormatFromHeader: header) data: data size: data size freq: header sampleRate.
^ buffer

MyClass>>readFormatFromHeader: header
^ header channels = 1
ifTrue: [ 
header bitsPerSample = 8
ifTrue: [ AL_FORMAT_MONO8 ]
ifFalse: [ AL_FORMAT_MONO16 ] ]
ifFalse: [ 
header bitsPerSample = 8
ifTrue: [ AL_FORMAT_STEREO8 ]
ifFalse: [ AL_FORMAT_STEREO16 ] ]

I am pretty sure with a little work those things could work again. I have never succeeded in playing mp3 files from Pharo though (but I'm on mac and I remembered trying solutions that did not work on Mac but may work on other OS).

Have fun guys :-)

On Sat, Oct 14, 2017 at 10:01 AM, [hidden email] <[hidden email]> wrote:
I'd like to do something like this: https://www.youtube.com/watch?v=vL7D4eU0lYE

to showcase FFI, Bloc, and the Pharo liveliness.

And once I get the basics working, level up in getting it working in a VR headset.

Is there anyone having a binding to a library for dealing with the sound part, like FMod or other things like that?

I see this as a medium term project, so, like I can get it working by the end of the year.

TIA

Phil



--
Clément Béra
Pharo consortium engineer