poet

{{{ "title" : "Dancer.js, a high-level audio API", "tags" : [ "web audio", "audio data", "webgl" ], "category" : "audio", "date" : "7-30-2012", "preview" : "dancer.js is a high-level audio API, usable with both Mozilla's Audio Data API and Webkit's Web Audio API with flash fallback, designed to make sweet visualizations.\nhttp://jsantell.github.com/dancer.js\n\n_v0.2.1 (6/15/2012)_\n\nFeatures\n\n---\n* Use real-time audio waveform and frequency data and map it to any arbitrary visualization\n* Leverage beat detection into your visualizations\n* Simple API to time callbacks and events to any section of a song\n* Supports Web Audio (webkit), Audio Data (mozilla) and flash fallback (v9+)\n* Extensible framework supporting plugins and custom behaviours" }}} dancer.js is a high-level audio API, usable with both Mozilla's Audio Data API and Webkit's Web Audio API with flash fallback, designed to make sweet visualizations. <script> console.log('this is a script parse test'); </script> http://jsantell.github.com/dancer.js _v0.2.1 (6/15/2012)_ Features --- * Use real-time audio waveform and frequency data and map it to any arbitrary visualization * Leverage beat detection into your visualizations * Simple API to time callbacks and events to any section of a song * Supports Web Audio (webkit), Audio Data (mozilla) and flash fallback (v9+) * Extensible framework supporting plugins and custom behaviours Dancer Instance Methods --- ### Controls All controls return `this`. * `play()` plays the audio and begins the dance. * `stop()` stops the madness. ### Getters * `getTime()` returns the current time. * `getWaveform()` returns the waveform data array (Float32Array(1024)) * `getSpectrum()` returns the frequency data array (Float32Array(512)). * `getFrequency( freq [, endFreq ] )` returns the magnitude of a frequency or average over a range of frequencies. * `isLoaded()` returns a boolean value for the dancer instance's song load state. * `isPlaying()` returns a boolean value indicating whether the dancer instance's song is currently playing or not. ### Sections All section methods return `this` (CHAIN IT UP) and callbacks executed with `this` referencing the dancer instance. * `after( t, callback )` fires callback on every frame after time `t`. * `before( t, callback )` fires callback on every frame before time `t`. * `between( t0, t1, callback )` fires callback on every frame between time `t0` and `t1`. * `onceAt( t, callback )` fires callback once at time `t`. ### Bindings Basic pub/sub to tie into the dancer instance. `update` and `loaded` are predefined events called within the framework that are published on every frame (update) and on audio file load (loaded). All callbacks executed with `this` referencing the dancer instance. * `bind( name, callback )` subscribes a callback of `name`. Can call this method several times to bind several callbacks of the same name. * `unbind( name )` unsubscribes all callbacks of `name`. * `trigger( name )` calls all callbacks of `name`. ### Beat Beats are detected when the amplitude (normalized values between 0 and 1) of a specified frequency, or the max amplitude over a range, is greater than the minimum threshold, as well as greater than the previously registered beat's amplitude, which is decreased by the decay rate per frame. * `createBeat( options )` creates a new beat instance tied to the dancer instance, with an options object passed as an argument. Options listed below. * `frequency` the frequency (element of the spectrum) to check for a spike. Can be a single frequency (number) or a range (2 element array) that uses the frequency with highest amplitude. Default: `[ 0, 10 ]` * `threshold` the minimum amplitude of the frequency range in order for a beat to occur. Default: `0.3` * `decay` the rate that the previously registered beat's amplitude is reduced by on every frame. Default: `0.02` * `onBeat` the callback to be called when a beat is detected. * `offBeat` the callback to be called when there is no beat on the current frame. Dancer Static Methods --- * `addPlugin( name, fn )` registers a plugin of `name` with initiation function `fn` -- described in more detail below * `isSupported()` returns a string of `webaudio`, `audiodata` or `flash` indicating level of support. Returns an empty string if the browser doesn't support any of the methods. Can also return `null` when browser does not support typed arrays. * `canPlay( type )` returns either `true` or `false` indicating whether the browser supports playing back audio of type `type`, which can be a string of `'mp3'`, `'ogg'`, `'wav'`, or `'aac'`. * `setOptions( options )` takes a set of key-value pairs in an object for options. Options below. ### Dancer Options * `flashSWF` The path to soundmanager2.swf. Required for flash fallback. * `flashJS` The path to soundmanager2.js. Required for flash fallback. Dancer Constructor --- `new Dancer( source, [ codecs ] )` returns a new `Dancer` instance -- takes a string of `source` as a path to the audio file. Optionally, you may pass in array of codec extensions of the form `[ 'mp3', 'ogg' ]`, where the first supported codec is used and appended to source in the form `source + '.' + supportedCodec`. Beat Instance Methods --- These methods can be called on a beat instance to turn on and off the registered callbacks * `on()` turns on the beat instance's callbacks and detections * `off()` turns off the beat instance's callbacks and detections Example --- For simple examples, check out the `examples/` folder -- both the FFT and waveform examples are straight forward, leveraging the corresponding plugins for visualizations. ```javascript // To enable flash fallback, specify the paths for the flashSWF and flashJS Dancer.setOptions({ flashJS : '../../lib/soundmanager2.js', flashSWF : '../../lib/soundmanager2.swf' }); var dancer = new Dancer( "sickjams.ogg" ), beat = dancer.createBeat({ onBeat: function ( mag ) { console.log('Beat!'); }, offBeat: function ( mag ) { console.log('no beat :('); } }); // Let's turn this beat on right away beat.on(); dancer.onceAt( 10, function() { // Let's set up some things once at 10 seconds }).between( 10, 60, function() { // After 10s, let's do something on every frame for the first minute }).after( 60, function() { // After 60s, let's get this real and map a frequency to an object's y position // Note that the instance of dancer is bound to "this" object.y = this.getFrequency( 400 ); }).onceAt( 120, function() { // After 120s, we'll turn the beat off as another object's y position is still being mapped from the previous "after" method beat.off(); }); dancer.play(); ``` Requirements ---- **HTML5 Playback with Web Audio or Audio Data** Chrome and Firefox are both supported out of the box -- other browsers will need to leverage the flash fallback until either of these APIs are implemented. **To enable flash** You must set Dancer's defaults for `flashSWF` with the path to the `soundmanager2.swf` and `flashJS` to the path to `soundmanager2.js`, both found in `lib/`. Flash player 9 is required, and you must provide an mp3 option. Waveform data in Flash is a 1024 Float32Array, but only the first 512 elements have values due to flash's computeSpectrum method. **Uint32Array and Float32Array are required** Include a shim if you'd like to support browsers that do not have these typed arrays. Dependencies --- * [dsp.js](https://github.com/corbanbrook/dsp.js/) - A subset of dsp.js (fft) is used for Fast Fourier Transformations ( Included in packaged Dancer ) * [flash_detect](http://www.featureblend.com/javascript-flash-detection-library.html) - flash detect is used for immediate flash detection ( Included in packaged Dancer ) * [soundmanager2](https://github.com/scottschiller/SoundManager2) - soundmanager2 is used for flash fallback ( found in `lib/`, asynchronously loaded ) Extending/Plugins --- You can extend the Dancer prototype by calling the static method `addPlugin( name, fn )`, which extends the Dancer prototype. A Dancer instance then can call the function provided in its context and subscribe to a preexisting event like `update`, or make your own. Look in the `plugins/` directory for examples. Development --- This project uses [smoosh](https://github.com/fat/smoosh) to build and [jasmine](http://pivotal.github.com/jasmine/) for testing. A CLI for testing would be awesome, but Mozilla and WebKit implementations differ greatly -- go to `spec/index.html` in Mozilla/WebKit browsers to test. All tests should pass in Chrome, Firefox, Opera and Safari, _although they can get weird_. Should consistently pass. Change Logs ---- **v0.2.1 (6/16/2012)** * Added getWaveform() method and a corresponding visualization for waveforms **v0.2.0 (6/14/2012)** * Added flash support with soundmanager2 -- flash_detect now included in build * Added static methods `isSupported`, `canPlay` and `setOptions` * Added multiple audio codecs support (#7) * Added a new simple FFT examples, both examples having feature detection and controls (#10) * Fixed several Webkit bugs (#4, #8) **v0.1.0 (6/3/2012)** * Initial Web Audio/ Audio Data release