audio / source code

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

audio / source code

Jay Hardesty-2
>Chris Double wrote:
>
>> On 13/04/10 10:32, Jay Hardesty wrote:
>> Objects and apps for varying and hybridizing musical scores.
>> Morphic style widgets for control and visualization of musical structure.
>> Rendering MIDI data into audio (via Quicktime is the only current
>> solution there I suppose?)
>
> You can use HTML 5 audio via data URL's to generate sound and play it -
> although it's a bit of a pain doing it this way. There is work being
> done on working out an HTML audio generation API. See here for some
> discussion:
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=490705
>
> If you generate the audio on the server and serve it as a WAV file then
> you can use HTML 5 audio to play it on recent Chrome, Safari, Opera and
> Firefox builds (I think all those support WAV).
>
> Chris.
> --
> http://bluishcoder.co.nz

Thank you Chris - those developments definitely seem worth tracking.
For now, my aim's to make web apps that don't require a server process
generating audio (already do that for Seaside apps), and so having a
sound library to render MIDI scores on the client side is important -
without having to serve one sample-by-sample in html. So unfortunately
Quicktime seems like the only thing (I hate general  MIDI as much as
anybody, but at least it works as a placeholder, for proof-of-concept
apps)


> Robert Krahn wrote:
>
>Of course you can also add source code and subdirectories elsewhere. Assuming you want to add audio support you might want add a subdirectory audio to http://www.lively->kernel.org/repository/lively-wiki/.
>[...]
>Important: Source code contributions MUST be MIT licensed. Please only upload your code if you are willing to accept this license.
>
>Best,
>Robert

Thanks - it's good to know that. Not sure yet if LK is an appropriate
place at this time for something as domain specific as my music
projects, but will keep the licensing guidelines in mind when
evaluating potential paths forward.

Jay

Reply | Threaded
Open this post in threaded view
|

Lively Audio

Dan Ingalls-4
Hi, Guys, especially Chris -

I took a run at this a couple of years back -- it's why Lively has a piano keyboard.
In fact I had it in the opening screen for a few months as motivation.
The most fun I had with Lively was firing it up in Java once (part of
a big Java / JavaScript demo at Sun)  and being able to play
that keyboard through a midi synth API.  So...

> >Chris Double wrote:
> >...
> >> Rendering MIDI data into audio (via Quicktime is the only current
> >> solution there I suppose?)

Can you say what you mean here?  Is there a simple way to call Quicktime
from JavaScript to, eg, play F sharp in an oboe timbre for 0.8 seconds?

> > You can use HTML 5 audio via data URL's to generate sound and play it -
> > although it's a bit of a pain doing it this way.

Do you mean one data URL per note (88 of them), with no control over, eg,
duration, pitch, etc?  Or is there something better?

>There is work being
> > done on working out an HTML audio generation API. See here for some
>> discussion:
>>
>> https://bugzilla.mozilla.org/show_bug.cgi?id=490705
>>
>> If you generate the audio on the server and serve it as a WAV file then
>> you can use HTML 5 audio to play it on recent Chrome, Safari, Opera and
> > Firefox builds (I think all those support WAV).

Can you say a bit more?  Can one serve up (or otherwise obtain) one .WAV file
and then use that as a timbre by ASDR techniques so you can play all 88
notes in various durations and volumes from that one file?

Thanks in advance

        - Dan

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Alessandro Warth
In reply to this post by Jay Hardesty-2
> >Chris Double wrote:
> >...
> >> Rendering MIDI data into audio (via Quicktime is the only current
> >> solution there I suppose?)

Can you say what you mean here?  Is there a simple way to call Quicktime
from JavaScript to, eg, play F sharp in an oboe timbre for 0.8 seconds?

Not sure what Chris meant, but you could generate the contents of a midi file in memory, then feed those bytes (after uuencoding) to your platform's midi player (w/ the embed tag). Here's an example: http://tinlizzie.org/ometa-js/#Etude
 
I doubt this approach would be useful for real-time stuff like making sounds come out of your keyboard morph, but it's definitely worth knowing about.

Cheers,
Alex


> > You can use HTML 5 audio via data URL's to generate sound and play it -
> > although it's a bit of a pain doing it this way.

Do you mean one data URL per note (88 of them), with no control over, eg,
duration, pitch, etc?  Or is there something better?

>There is work being
> > done on working out an HTML audio generation API. See here for some
>> discussion:
>>
>> https://bugzilla.mozilla.org/show_bug.cgi?id=490705
>>
>> If you generate the audio on the server and serve it as a WAV file then
>> you can use HTML 5 audio to play it on recent Chrome, Safari, Opera and
> > Firefox builds (I think all those support WAV).

Can you say a bit more?  Can one serve up (or otherwise obtain) one .WAV file
and then use that as a timbre by ASDR techniques so you can play all 88
notes in various durations and volumes from that one file?

Thanks in advance

       - Dan
_______________________________________________
lively-kernel mailing list
[hidden email]
http://lists.hpi.uni-potsdam.de/listinfo/lively-kernel

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Chris Double
In reply to this post by Dan Ingalls-4
On 21/04/10 12:34, Dan Ingalls wrote:
>
>>> You can use HTML 5 audio via data URL's to generate sound and play it -
>>> although it's a bit of a pain doing it this way.
>
> Do you mean one data URL per note (88 of them), with no control over, eg,
> duration, pitch, etc?  Or is there something better?

The approach I was thinking of was something like this:

<http://acko.net/blog/javascript-audio-synthesis-with-html-5>

You generate the audio data, attach a WAV header and play it with an
audio element. It's a bit hacky but does work.

Ideally there would be an actual audio API allowing generating and
playing samples - this is something a lot of people are interested in
and they're exploring possibly API's for standardisation.

> Can you say a bit more?  Can one serve up (or otherwise obtain) one .WAV file
> and then use that as a timbre by ASDR techniques so you can play all 88
> notes in various durations and volumes from that one file?

Technically you could load the .WAV file using XMLHttpRequest, get the
raw data, manipulate it and create data URL's containing the modified
data. Although not using WAV I have an example of loading a file
containing binary data and parsing it (using an Ogg file for the
example) here:

<http://www.bluishcoder.co.nz/2009/06/05/reading-ogg-files-with-javascript.html>

Another approach if you don't mind using flash is to use one of the
Flash based audio libraries:

http://www.schillmania.com/projects/soundmanager2/

Chris.
--
http://www.bluishcoder.co.nz




Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Nikolay Suslov
In reply to this post by Alessandro Warth
On Wed, Apr 21, 2010 at 4:58 AM, Alessandro Warth <[hidden email]> wrote:
> >Chris Double wrote:
> >...
> >> Rendering MIDI data into audio (via Quicktime is the only current
> >> solution there I suppose?)

Can you say what you mean here?  Is there a simple way to call Quicktime
from JavaScript to, eg, play F sharp in an oboe timbre for 0.8 seconds?

Not sure what Chris meant, but you could generate the contents of a midi file in memory, then feed those bytes (after uuencoding) to your platform's midi player (w/ the embed tag). Here's an example: http://tinlizzie.org/ometa-js/#Etude
 

It is awesome!
And is it possible to export generated by Etude midi files?

Best regards,
Nikolay

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Jay Hardesty-2
In reply to this post by Alessandro Warth
>> > >> Rendering MIDI data into audio (via Quicktime is the only current
>> > >> solution there I suppose?)
>>
>> Can you say what you mean here?  Is there a simple way to call Quicktime
>> from JavaScript to, eg, play F sharp in an oboe timbre for 0.8 seconds?
>>
>
> Not sure what Chris meant, but you could generate the contents of a midi
> file in memory, then feed those bytes (after uuencoding<http://en.wikipedia.org/wiki/Uuencoding>)
> to your platform's midi player (w/ the embed tag). Here's an example:
> http://tinlizzie.org/ometa-js/#Etude

What I meant in the remark regarding Quicktime (to which Chris replied) was just that Quicktime provides a library of samples already installed (such as oboe: program 69).  I wasn't addressing how to actually trigger playback in Javascript (hadn't got that far yet...)    

I'm guessing that in Javascript you'd typically have to generate and then play back a MIDI file if you wanted (er, somewhat) dynamic content that rendered via Quicktime.  But the link above seems to offer hope of an alternative.
Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Alessandro Warth
In reply to this post by Nikolay Suslov
Not sure what Chris meant, but you could generate the contents of a midi file in memory, then feed those bytes (after uuencoding) to your platform's midi player (w/ the embed tag). Here's an example: http://tinlizzie.org/ometa-js/#Etude
 

It is awesome!
And is it possible to export generated by Etude midi files?

That functionality isn't in there at the moment, but it should be easy to implement. After all, you've already got the uuencoded data as a JavaScript string. So all you'd have to do is save that in a file, uudecode it, and you'd have a proper midi file.

(For the record, I didn't write Etude -- it was a class project done by Amarin Phaosawasdi, a masters student that I co-advised at UCLA.)

Cheers,
Alex


Best regards,
Nikolay


Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Dan Ingalls-4
In reply to this post by Alessandro Warth
Re: [lively-kernel] Lively Audio
Hi, Alex -
Not sure what Chris meant, but you could generate the contents of a midi file in memory, then feed those bytes (after uuencoding) to your platform's midi player (w/ the embed tag). Here's an example: http://tinlizzie.org/ometa-js/#Etude
 
I doubt this approach would be useful for real-time stuff like making sounds come out of your keyboard morph, but it's definitely worth knowing about.

Definitely, and I remember getting this close once before.

The real question is :  Do you know if midi players allow a note to be started in one file and ended in another?  This would allow key-down and key-up events to come in real time from the keyboard.  I would make this work if the answer is yes.

Thanks
        - Dan
Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Jay Hardesty-2
In reply to this post by Alessandro Warth
[Sorry for the duplicate message ...forgot to edit the subject   -jh]

>The real question is :  Do you know if midi players allow a note to
>be started in one file and ended in another?  This would allow
>key-down and key-up events to come in real time from the keyboard.  I
>would make this work if the answer is yes.
>
>Thanks
>       - Dan


As far as I know a MIDI player does not care where (or from which
file) a MIDI ON, or subsequent MIDI OFF message comes from.
Each is just a pair of bytes arriving from wherever.

(That is, the first byte MIDI ON event  is 144 for channel 1, 145 for
channel 2. etc. up to 159 for channel 16. Second byte contains pitch
and velocity. MIDI OFF events have first byte set to 128 for channel 1
up to 143 for channel 16 etc)

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Jay Hardesty-2
In reply to this post by Alessandro Warth
>Second byte contains pitch and velocity
CORRECTION: meant second and third bytes contain pitch and velocity


>The real question is :  Do you know if midi players allow a note to
>be started in one file and ended in another?  This would allow
>key-down and key-up events to come in real time from the keyboard.  I
>would make this work if the answer is yes.
>
>Thanks
>       - Dan


As far as I know a MIDI player does not care where (or from which
file) a MIDI ON, or subsequent MIDI OFF message comes from.
Each is just a pair of bytes arriving from wherever.

(That is, the first byte MIDI ON event  is 144 for channel 1, 145 for
channel 2. etc. up to 159 for channel 16. Second byte contains pitch
and velocity. MIDI OFF events have first byte set to 128 for channel 1
up to 143 for channel 16 etc)

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Alessandro Warth
Hi Jay,

While the MIDI format doesn't care about matching ONs and OFFs, there's a good chance that the player tidies up (by sending an "all notes off" message to the hardware) once it's done playing a file. So this could end up working in some platforms, but not in others...

I guess we won't know until we try.

Cheers,
Alex

On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]> wrote:
>Second byte contains pitch and velocity
CORRECTION: meant second and third bytes contain pitch and velocity


>The real question is :  Do you know if midi players allow a note to
>be started in one file and ended in another?  This would allow
>key-down and key-up events to come in real time from the keyboard.  I
>would make this work if the answer is yes.
>
>Thanks
>       - Dan


As far as I know a MIDI player does not care where (or from which
file) a MIDI ON, or subsequent MIDI OFF message comes from.
Each is just a pair of bytes arriving from wherever.

(That is, the first byte MIDI ON event  is 144 for channel 1, 145 for
channel 2. etc. up to 159 for channel 16. Second byte contains pitch
and velocity. MIDI OFF events have first byte set to 128 for channel 1
up to 143 for channel 16 etc)
_______________________________________________
lively-kernel mailing list
[hidden email]
http://lists.hpi.uni-potsdam.de/listinfo/lively-kernel

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Jay Hardesty-2
On Fri, Apr 23, 2010 at 3:09 AM, Alessandro Warth <[hidden email]> wrote:
> Hi Jay,
> While the MIDI format doesn't care about matching ONs and OFFs, there's a
> good chance that the player tidies up (by sending an "all notes off" message
> to the hardware) once it's done playing a file. So this could end up working
> in some platforms, but not in others...
> I guess we won't know until we try.
> Cheers,
> Alex

Thanks, yes - I was just thinking of all those times I've had a
looping MIDI file playing, then slotted in another MIDI file to take
its place (for the next section of music), and found I had notes stuck
playing from the first MIDI cause they never got their NOTE-OFF
messages.

But I'm sure you're right - in fact can't remember the situation I
describe happening with something as self-contained as the Quicktime
player (never tried it using Javascript either).

> On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]>
>> As far as I know a MIDI player does not care where (or from which
>> file) a MIDI ON, or subsequent MIDI OFF message comes from.
>> Each is just a pair of bytes arriving from wherever.

more specifically, I should have said : a pair of data bytes following
a status byte..

Jay

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Jay Hardesty-2
In reply to this post by Alessandro Warth
On Fri, Apr 23, 2010 at 3:09 AM, Alessandro Warth <[hidden email]> wrote:
> Hi Jay,
> While the MIDI format doesn't care about matching ONs and OFFs, there's a
> good chance that the player tidies up (by sending an "all notes off" message
> to the hardware) once it's done playing a file. So this could end up working
> in some platforms, but not in others...
> I guess we won't know until we try.
> Cheers,
> Alex

Thanks, yes - I was just thinking of all those times I've had a
looping MIDI file playing, then slotted in another MIDI file to take
its place (for the next section of music), and found I had notes stuck
playing from the first MIDI cause they never got their NOTE-OFF
messages.

But I'm sure you're right - in fact can't remember the situation I
describe happening with something as self-contained as the Quicktime
player (never tried it using Javascript either).

> On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]>
>> As far as I know a MIDI player does not care where (or from which
>> file) a MIDI ON, or subsequent MIDI OFF message comes from.
>> Each is just a pair of bytes arriving from wherever.

more specifically, I should have said : a pair of data bytes following
a status byte..

Jay

Reply | Threaded
Open this post in threaded view
|

Re: Lively Audio

Jay Hardesty-2
In reply to this post by Alessandro Warth
On Fri, Apr 23, 2010 at 3:09 AM, Alessandro Warth <[hidden email]> wrote:
> Hi Jay,
> While the MIDI format doesn't care about matching ONs and OFFs, there's a
> good chance that the player tidies up (by sending an "all notes off" message
> to the hardware) once it's done playing a file. So this could end up working
> in some platforms, but not in others...
> I guess we won't know until we try.
> Cheers,
> Alex

Thanks, yes - I was just thinking of all those times I've had a
looping MIDI file playing, then slotted in another MIDI file to take
its place (for the next section of music), and found I had notes stuck
playing from the first MIDI cause they never got their NOTE-OFF
messages.

But I'm sure you're right - in fact can't remember the situation I
describe happening with something as self-contained as the Quicktime
player (never tried it using Javascript either).

> On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]>
>> As far as I know a MIDI player does not care where (or from which
>> file) a MIDI ON, or subsequent MIDI OFF message comes from.
>> Each is just a pair of bytes arriving from wherever.

more specifically, I should have said : a pair of data bytes following
a status byte..

Jay