audio / source code
Locked
 
|
>Chris Double wrote:
> >> On 13/04/10 10:32, Jay Hardesty wrote: >> Objects and apps for varying and hybridizing musical scores. >> Morphic style widgets for control and visualization of musical structure. >> Rendering MIDI data into audio (via Quicktime is the only current >> solution there I suppose?) > > You can use HTML 5 audio via data URL's to generate sound and play it - > although it's a bit of a pain doing it this way. There is work being > done on working out an HTML audio generation API. See here for some > discussion: > > https://bugzilla.mozilla.org/show_bug.cgi?id=490705 > > If you generate the audio on the server and serve it as a WAV file then > you can use HTML 5 audio to play it on recent Chrome, Safari, Opera and > Firefox builds (I think all those support WAV). > > Chris. > -- > http://bluishcoder.co.nz Thank you Chris - those developments definitely seem worth tracking. For now, my aim's to make web apps that don't require a server process generating audio (already do that for Seaside apps), and so having a sound library to render MIDI scores on the client side is important - without having to serve one sample-by-sample in html. So unfortunately Quicktime seems like the only thing (I hate general MIDI as much as anybody, but at least it works as a placeholder, for proof-of-concept apps) > Robert Krahn wrote: > >Of course you can also add source code and subdirectories elsewhere. Assuming you want to add audio support you might want add a subdirectory audio to http://www.lively->kernel.org/repository/lively-wiki/. >[...] >Important: Source code contributions MUST be MIT licensed. Please only upload your code if you are willing to accept this license. > >Best, >Robert Thanks - it's good to know that. Not sure yet if LK is an appropriate place at this time for something as domain specific as my music projects, but will keep the licensing guidelines in mind when evaluating potential paths forward. Jay |
|
|
Hi, Guys, especially Chris -
I took a run at this a couple of years back -- it's why Lively has a piano keyboard. In fact I had it in the opening screen for a few months as motivation. The most fun I had with Lively was firing it up in Java once (part of a big Java / JavaScript demo at Sun) and being able to play that keyboard through a midi synth API. So... > >Chris Double wrote: > >... > >> Rendering MIDI data into audio (via Quicktime is the only current > >> solution there I suppose?) Can you say what you mean here? Is there a simple way to call Quicktime from JavaScript to, eg, play F sharp in an oboe timbre for 0.8 seconds? > > You can use HTML 5 audio via data URL's to generate sound and play it - > > although it's a bit of a pain doing it this way. Do you mean one data URL per note (88 of them), with no control over, eg, duration, pitch, etc? Or is there something better? >There is work being > > done on working out an HTML audio generation API. See here for some >> discussion: >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=490705 >> >> If you generate the audio on the server and serve it as a WAV file then >> you can use HTML 5 audio to play it on recent Chrome, Safari, Opera and > > Firefox builds (I think all those support WAV). Can you say a bit more? Can one serve up (or otherwise obtain) one .WAV file and then use that as a timbre by ASDR techniques so you can play all 88 notes in various durations and volumes from that one file? Thanks in advance - Dan |
|
|
In reply to this post by Jay Hardesty-2
> >Chris Double wrote: Not sure what Chris meant, but you could generate the contents of a midi file in memory, then feed those bytes (after uuencoding) to your platform's midi player (w/ the embed tag). Here's an example: http://tinlizzie.org/ometa-js/#Etude
I doubt this approach would be useful for real-time stuff like making sounds come out of your keyboard morph, but it's definitely worth knowing about. Cheers, Alex
|
|
|
In reply to this post by Dan Ingalls-4
On 21/04/10 12:34, Dan Ingalls wrote:
> >>> You can use HTML 5 audio via data URL's to generate sound and play it - >>> although it's a bit of a pain doing it this way. > > Do you mean one data URL per note (88 of them), with no control over, eg, > duration, pitch, etc? Or is there something better? The approach I was thinking of was something like this: <http://acko.net/blog/javascript-audio-synthesis-with-html-5> You generate the audio data, attach a WAV header and play it with an audio element. It's a bit hacky but does work. Ideally there would be an actual audio API allowing generating and playing samples - this is something a lot of people are interested in and they're exploring possibly API's for standardisation. > Can you say a bit more? Can one serve up (or otherwise obtain) one .WAV file > and then use that as a timbre by ASDR techniques so you can play all 88 > notes in various durations and volumes from that one file? Technically you could load the .WAV file using XMLHttpRequest, get the raw data, manipulate it and create data URL's containing the modified data. Although not using WAV I have an example of loading a file containing binary data and parsing it (using an Ogg file for the example) here: <http://www.bluishcoder.co.nz/2009/06/05/reading-ogg-files-with-javascript.html> Another approach if you don't mind using flash is to use one of the Flash based audio libraries: http://www.schillmania.com/projects/soundmanager2/ Chris. -- http://www.bluishcoder.co.nz |
|
|
In reply to this post by Alessandro Warth
On Wed, Apr 21, 2010 at 4:58 AM, Alessandro Warth <[hidden email]> wrote:
It is awesome! And is it possible to export generated by Etude midi files? Best regards, Nikolay |
|
|
In reply to this post by Alessandro Warth
>> > >> Rendering MIDI data into audio (via Quicktime is the only current >> > >> solution there I suppose?) >> >> Can you say what you mean here? Is there a simple way to call Quicktime >> from JavaScript to, eg, play F sharp in an oboe timbre for 0.8 seconds? >> > > Not sure what Chris meant, but you could generate the contents of a midi > file in memory, then feed those bytes (after uuencoding<http://en.wikipedia.org/wiki/Uuencoding>) > to your platform's midi player (w/ the embed tag). Here's an example: > http://tinlizzie.org/ometa-js/#Etude What I meant in the remark regarding Quicktime (to which Chris replied) was just that Quicktime provides a library of samples already installed (such as oboe: program 69). I wasn't addressing how to actually trigger playback in Javascript (hadn't got that far yet...) I'm guessing that in Javascript you'd typically have to generate and then play back a MIDI file if you wanted (er, somewhat) dynamic content that rendered via Quicktime. But the link above seems to offer hope of an alternative. |
|
|
In reply to this post by Nikolay Suslov
That functionality isn't in there at the moment, but it should be easy to implement. After all, you've already got the uuencoded data as a JavaScript string. So all you'd have to do is save that in a file, uudecode it, and you'd have a proper midi file.
(For the record, I didn't write Etude -- it was a class project done by Amarin Phaosawasdi, a masters student that I co-advised at UCLA.) Cheers, Alex
|
|
|
In reply to this post by Alessandro Warth
Hi, Alex -
Not sure what Chris meant, but you could generate the contents of a midi file in memory, then feed those bytes (after uuencoding) to your platform's midi player (w/ the embed tag). Here's an example: http://tinlizzie.org/ometa-js/#Etude I doubt this approach would be useful for real-time stuff like making sounds come out of your keyboard morph, but it's definitely worth knowing about. Definitely, and I remember getting this close once before.
The real question is : Do you know if midi players allow a
note to be started in one file and ended in another? This would
allow key-down and key-up events to come in real time from the
keyboard. I would make this work if the answer is yes.
Thanks
|
|
|
In reply to this post by Alessandro Warth
[Sorry for the duplicate message ...forgot to edit the subject -jh]
>The real question is : Do you know if midi players allow a note to >be started in one file and ended in another? This would allow >key-down and key-up events to come in real time from the keyboard. I >would make this work if the answer is yes. > >Thanks > - Dan As far as I know a MIDI player does not care where (or from which file) a MIDI ON, or subsequent MIDI OFF message comes from. Each is just a pair of bytes arriving from wherever. (That is, the first byte MIDI ON event is 144 for channel 1, 145 for channel 2. etc. up to 159 for channel 16. Second byte contains pitch and velocity. MIDI OFF events have first byte set to 128 for channel 1 up to 143 for channel 16 etc) |
|
|
In reply to this post by Alessandro Warth
>Second byte contains pitch and velocity
CORRECTION: meant second and third bytes contain pitch and velocity >The real question is : Do you know if midi players allow a note to >be started in one file and ended in another? This would allow >key-down and key-up events to come in real time from the keyboard. I >would make this work if the answer is yes. > >Thanks > - Dan As far as I know a MIDI player does not care where (or from which file) a MIDI ON, or subsequent MIDI OFF message comes from. Each is just a pair of bytes arriving from wherever. (That is, the first byte MIDI ON event is 144 for channel 1, 145 for channel 2. etc. up to 159 for channel 16. Second byte contains pitch and velocity. MIDI OFF events have first byte set to 128 for channel 1 up to 143 for channel 16 etc) |
|
|
Hi Jay,
While the MIDI format doesn't care about matching ONs and OFFs, there's a good chance that the player tidies up (by sending an "all notes off" message to the hardware) once it's done playing a file. So this could end up working in some platforms, but not in others...
I guess we won't know until we try. Cheers, Alex On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]> wrote:
|
|
|
On Fri, Apr 23, 2010 at 3:09 AM, Alessandro Warth <[hidden email]> wrote:
> Hi Jay, > While the MIDI format doesn't care about matching ONs and OFFs, there's a > good chance that the player tidies up (by sending an "all notes off" message > to the hardware) once it's done playing a file. So this could end up working > in some platforms, but not in others... > I guess we won't know until we try. > Cheers, > Alex Thanks, yes - I was just thinking of all those times I've had a looping MIDI file playing, then slotted in another MIDI file to take its place (for the next section of music), and found I had notes stuck playing from the first MIDI cause they never got their NOTE-OFF messages. But I'm sure you're right - in fact can't remember the situation I describe happening with something as self-contained as the Quicktime player (never tried it using Javascript either). > On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]> >> As far as I know a MIDI player does not care where (or from which >> file) a MIDI ON, or subsequent MIDI OFF message comes from. >> Each is just a pair of bytes arriving from wherever. more specifically, I should have said : a pair of data bytes following a status byte.. Jay |
|
|
In reply to this post by Alessandro Warth
On Fri, Apr 23, 2010 at 3:09 AM, Alessandro Warth <[hidden email]> wrote:
> Hi Jay, > While the MIDI format doesn't care about matching ONs and OFFs, there's a > good chance that the player tidies up (by sending an "all notes off" message > to the hardware) once it's done playing a file. So this could end up working > in some platforms, but not in others... > I guess we won't know until we try. > Cheers, > Alex Thanks, yes - I was just thinking of all those times I've had a looping MIDI file playing, then slotted in another MIDI file to take its place (for the next section of music), and found I had notes stuck playing from the first MIDI cause they never got their NOTE-OFF messages. But I'm sure you're right - in fact can't remember the situation I describe happening with something as self-contained as the Quicktime player (never tried it using Javascript either). > On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]> >> As far as I know a MIDI player does not care where (or from which >> file) a MIDI ON, or subsequent MIDI OFF message comes from. >> Each is just a pair of bytes arriving from wherever. more specifically, I should have said : a pair of data bytes following a status byte.. Jay |
|
|
In reply to this post by Alessandro Warth
On Fri, Apr 23, 2010 at 3:09 AM, Alessandro Warth <[hidden email]> wrote:
> Hi Jay, > While the MIDI format doesn't care about matching ONs and OFFs, there's a > good chance that the player tidies up (by sending an "all notes off" message > to the hardware) once it's done playing a file. So this could end up working > in some platforms, but not in others... > I guess we won't know until we try. > Cheers, > Alex Thanks, yes - I was just thinking of all those times I've had a looping MIDI file playing, then slotted in another MIDI file to take its place (for the next section of music), and found I had notes stuck playing from the first MIDI cause they never got their NOTE-OFF messages. But I'm sure you're right - in fact can't remember the situation I describe happening with something as self-contained as the Quicktime player (never tried it using Javascript either). > On Thu, Apr 22, 2010 at 6:16 PM, Jay Hardesty <[hidden email]> >> As far as I know a MIDI player does not care where (or from which >> file) a MIDI ON, or subsequent MIDI OFF message comes from. >> Each is just a pair of bytes arriving from wherever. more specifically, I should have said : a pair of data bytes following a status byte.. Jay |
«
Return to Lively Kernel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |