Antti Kupila

Personal Blog, Portfolio and Online playground

View over the green room at Sid Lee, Montreal

SoundFX, out-of-the-box audio filters with actionscript 3

We had a project where we would have needed to apply something like a lowpass filter in real time on an audio track. Doing some research for this quickly triggered even further interest in audio programming. I also realized it’s not as scary as it first seems and a whole lot of fun. We deployed on Flash player 9 so we ended up going with another approach. Still, this is what came out of that.

While all the audio stuff isn’t as hard as it looks when reading the docs it can still be intimidating to a lot of developers. Hopefully this post will help out those guys. The key focus here was to keep the syntax as similar to the one we’re familiar with from display objects, the SoundFX class does all the heavy lifting. Take a look at the following syntax:

[as]
var sound : SoundFX = new SoundFX( new URLRequest( “music.mp3″ ) );
sound.filters = [ new CutoffFilter( 12000 ) ];
sound.play( );
[/as]

When you don’t have to worry so much about the nitty gritty stuff of making it work you can focus on the fun creative part. You can do basically anything you can do with display object filters, animate the properties etc. It’s a lot of fun ;)

A simple demo of how this works (flash player 10 required):

Go crazy with the filters :)
A couple small shortcuts will help you too:

  • Double click: default value
  • Shift + double click: maximum value
  • Command/ctrl + double click: minimum value
  • Shift + command/ctrl + double click: middle value
  • Click on visualizer to pause/resume playback (pretty cool if you’ve added some echo)
  • Double click visualizer to randomize all knobs

An added bonus here is added control over audio streaming. As with NetStreams you can specify how many seconds should be loaded before playback starts.

Source (includes SoundFX and the demo)

The filters are currently ported from C source at musicdsp.com (awesome resource!). Once i look a bit more into the math & magic behind the effects i’ll try to post new effects. Of course i’m more than happy to include any other people’s effects too, creating new effects is pretty simple.

Pixel Bender levels example

Pixel Bender is huge and is going to be even bigger once Flash player 10 officially is out (and spreads a bit so that we can start using it in commercial projects). I just love it when Adobe constantly lower the restrictions on creativity :)

A lot of the examples we see online for Pixel Bender (previously codenamed Hydra) are crazy visual effects. While they’re amazing for certain visual effects (mostly due to the processing speed), finding real life use is harder. I’ve had the need for a similar-to-photoshop levels filter multiple times, if nothing else for preprocessing a bitmap for further analysis. Turns out creating something like this with Pixel Bender is really simple and even better; it’s blazing fast.

If you’ve read my previous articles you know that I’ve already written a prototype for the levels filter (geesh, it’s almost a year ago!). Well, back then Pixel Bender was called Hydra and the code written isn’t compatible with today’s version anymore. I addition to that we weren’t able to export stuff, it just ran in the Hydra toolkit. Today I updated the code and exported it as a .pbj that can be used in Flash.

I’ve made a small example with a couple example images with the contrast reduced. You’ll need Flash player 10 to play with it. If all you see is white download the plugin from Adobe labs.

A couple pointers on the example: Clicking a histogram will automatically balance it. Double clicking will reset it to 0-255. Clicking the image will load a new image. In photoshop you can choose to either modify the luminosity or the individual channels. Here i enabled you to modify both at the same time, but this means that if you modify all 4, the automatic mapping won’t be correct anymore (the histogram in the bottom left corner will show the result of this). I could make it understand this, but since it’s just a proof of concept i think this is fine. Also gamma isn’t supported at the moment.

The source is of course available:

Do you have other real life uses for Pixel Bender?

Nike lab: Technical case study

Nike sports marketing came to AKQA San Francisco with a request: Build a site to promote the innovative new products they are cooking in the Nike labs. These products are kind of like what concept cars are to the automotive industry and Nike wanted to have a way to tell it’s customers more about the technology behind the scenes. Also with the Beijing Olympics coming up some athletes who are using Nike’s new products would also be featured on the site.

Both because my part wasn’t so much in the creative/concept part of the project and also because i think most of my readers here are interested in the technical part anyway, this post will focus more on what the gears and cogs under the hood are doing.

View the site: http://www.nikelab.com

Nike lab homepage

Requirements into features

Nike had it’s technical requirements (the biggest ones being deep linking for flexible marketing and also support for multiple languages) but internally our goals were much higher. Fortunately we had pretty good time to build the site itself + an amazing team so we were able to put our thoughts into actions.

To support all the features we built a new as3 framework as a base and then built the site on top of it. Many (almost all) of these features are in the framework itself and developers at AKQA can easily in the future take advantage of our work in other projects as well, as these features are built in when using the new framework.

Here’s a list of some of the features right now:

* Automatic support for deeplinking to any page
* Back button functionality
* Internal sitemap
* Multiple languages/locales, each one with support for localized content (Nike Lab has 23 locales)
* Language switching on the fly without refreshing the page
* Automated support for non western characters
* Everything is driven by xml with a special workflow (more on that later)
* Special support for data models (more on that later)
* Keyboard and mouse wheel support
* several more smaller features..

Basically all the technical features are in the framework and all the features for the site are in the numerous swf components for the site. The config file config.xml defines the mapping between these.

Deeplinking and the concept of ‟pages”

People complain about flash breaking the back button. Well, that’s true, but also an animated gif doesn’t support going back to the previous frame with the back button. You may say that’s a stupid comparison but at the same time a flash site isn’t actually much more than a fancy image to the browser. If there was some standardized way for the browser to know what a ‟page” is inside a flash site it could theoretically support this automatically (Adobe?..). Still, at the time of writing this is not possible so we’ll have to build in this support ourselves.

We wanted to be able to define what a page is very easily, so that when you go to that page the framework could automatically navigate through the browser’s history. We added support for this through something we call sections. When you navigate from a section to another section a navigation event is automatically created, letting the user navigate back using the back button. Also the address bar and title are obviously updated. Of course the developer can override the browser history if it’s not suitable in some case.

Firefox address bar with a Nike Lab deeplink

Creating a site that supports the back button isn’t hard. Frankly I’m guessing that most of the readers of this have done it or at least know what goes into it. Still, you guys are a small percentage of all the developers. A part of our goal was also to spread ‟the right way”, making it so easy to do that there’s really no excuse. Unfortunately the code is proprietary so we can’t share any of it, but at least AKQA sites in the future hopefully can show the way, inspiring other people to do the same. The key is that even any junior developer can add support for this with studying some documentation and some examples for a day or so.

Since the site has an internal sitemap to keep track of the deeplinking targets the site will know if a deeplink is invalid and it will automatically fall back to the closest match (for instance: /athlete/cobe-bryant/ will fall back to /athlete/ since Kobe Bryant’s name is misspelled. The /athlete/ page will show a list of all athletes so that the user can proceed). Also because the site is aware of it’s own structure it could easily be used to generate search engine compatible sitemaps + alternate content for easy SEO..

23 custom locales, 1 custom workflow

A requirement from Nike was to have support for an at the moment undefined number of languages. They knew it was going to be many languages but didn’t know exactly how many. Because of this we had to come up with an extremely smooth and flexible workflow. One of the biggest things was the huge amount of copy localized into all these languages with several revisions. Updating stuff manually would not only be a huge waste of time but also significantly increase the risk of human errors, not to mention the added effort on QA.

What we did instead was send excel decks to the translation company. They filled in the empty gaps with the localized copy and sent the documents back to us. We then ran proprietary software to generate xml from the excel documents. After this we used an air app specifically built for this purpose by Ronnie Liew to reformat the xml into standardized xliff format. Generating ~220,000 lines of xml took about 1 minute so we could almost instantly see the updated copy in the dev build. This sped up the process a significant amount, letting us focus on the creativity instead. Awesome.

Language switcher? This is 2008, let’s do it live

Nike’s requirement was to have support for multiple languages. We wanted to go beyond that. What if you send a deeplink to a certain page to a chinese friend whose English is a bit rusty? Now he can just go to the language switcher and swap languages to Chinese. The site automatically knows what data is currently active, reloads that data from the new locale, updates the copy models and updates the site. The process takes 10 seconds or so, depending on the web connection. No need to refresh the page. Works with any combination of course, including the non-western characters. While a lot is going on under the hood, all of this is transparent to the user.

Nike Lab language switcher

Intelligent loading

This site has a lot of data, there’s no way to escape loading. We can compress assets, combine files etc but in the end there’s only so much we can do. To optimize the experience the site automatically knows what assets and data sources are required to be loaded before navigating to a certain page, meaning they can be loaded in a batch with a single preloader. If the user changes his/her mind and navigates somewhere else instead the site sets a lower priority to the files in the batch so that the files that are required now are loaded first and the other files are loaded in the background. This way the first page is instantly available if the user were to click the back button. I think there’s a lot more that still can be done with this approach but i’ll leave that for another post..

The list of files required is automatically compiled from the site’s structure. Each section can define assets (images, swf’s, fonts etc), data sources (xml files, web services etc) that are required to view the page, in addition to loading the page itself (a swf) with the possibility of having a custom loader.

A new approach to handling shared font is also built into the framework, meaning the font outlines only need to be loaded once and only when required. Automatically of course.

Put that data in the models

While the framework’s main focus is to add complex functionality with ease to the end developer, the framework also endorses use of good structure and design patterns. Loaded data is automatically linked to models that can easily be accessed. It’s simply easier to store data in consistent models than in some custom mcgyver-duct-tape-solution. These models can also easily add support for localized data (just adding $region to the path of the file is enough). Custom models are of course supported as each site’s needs differ.

The kitchen sink

Keyboard support isn’t anything new. Same goes for the mouse wheel. To be honest; I don’t think it’s a feature, i think it’s something missing if it’s not there. Super easy support for non conflicting keyboard shortcuts was built into the framework and Nikelab takes advantage of that in many places.

We couldn’t have done it alone

A lot of people worked on this project. I would mention everybody but .. to be honest I don’t even know. The people involved in the creative development part for Nike Lab (in addition to myself) were Ronnie Liew (technical manager), Tim Robles (developer), Thomas Ko (developer) and Gopi Shubhakar (quality assurance). Thanks also goes to Sallie Lippe for her project management. Other AKQA developers & tech management were also included in the concepting of the framework.

A thing to keep in mind is that this is of course the first site built on the new framework, I’m sure it’ll change and improve a lot in the coming year. This is also just a short description of what’s actually inside. Some details i can’t share, others would just make this post go on for days and days. I’m of course still open to your comments, feel free to take advantage of the box below ;)

JPGSizeExtractor multi image example

I made a quick example to demonstrate the power of the JPGSizeExtractor class i wrote about a year ago. This is a demo that came up from a brief mail exchange with Richard Bacon who was looking at the class.

The example loads 10 images (one at a time), parses their SOF0 headers to get the dimensions, stops the load and draws grey boxes as backgrounds, setting the layout. This way building the layout is very quick and you don’t have to wait for the whole images to load. It’s kind of the same idea as specifying the width and height for images in html to maintain the layout even before the image has loaded.
When all sizes have been parsed the images themselves are loaded. For the demo the images are scaled down to 25% to make them fit better in the small window here. The total filesize for all 10 images is 4169kb. All images are from sxc.hu.

Please keep in mind that this example is of the type “quick and dirty”, and is only meant as a proof of concept. The code for the example was written just to make it work, not to make it pretty or flexible.

The source is available in my fresh public svn (which for some weird reason is in Chinese in Camino?! :) ): public.anttikupila.com/svn > jpgExtractorMultipleImages. The files are also available in a zip.

PureMVC 2.0.3

An architectural framework I’ve been using a lot recently has been updated to a new version.

PureMVC framework

The version (well, not the most recent one but the jump from 1 to 2) fixes a lot of small issues that existed in 1.x, most of which were inconsistencies that caused confusion. I usually downloaded the framework before a new project, fixed the stuff i wanted to fix myself and then started working. It of course always added a bit of overhead in the beginning of a project so this update is very welcome! I haven’t had a chance to verify that all I wanted to change has been fixed but if Cliff Hall has done what he talked about before, I’m happy :)

I’ve used PureMVC for most of my recent Actionscript 3 projects. It’s primarily meant for flex/air apps so building RIA’s with it is great. Flash is not as obvious since PureMVC doesn’t do any of the stuff that you might want from a flash framework (handle loading, stage manager etc) as it merely defines the architecture for the app. The strongest side i think is the fact that it encourages you to do stuff the right way, instead of quick & dirty. If you haven’t done so yet i definitely encourage you to take a look at the framework. The docs may be a bit confusing (i know they were to me in the beginning) but you should quickly get the hang of it.

PureMVC has gotten a lot of amazing buzz already. If you haven’t caught up with the nitty gritty details yet, Cliff also gives a very good description of what PureMVC is on the flex show, episode 33. Luke Bayes and Ali Mills also did a great presentation back in -07 where PureMVC came out as the “winner”; see it here.

Of course for more info there’s always puremvc.org.

The best use of 3d so far

I’ve got to say that I got this warm fuzzy feeling when I checked my RSS feeds today and saw the Red Bull Flightlab site mentioned on a couple sites. After checking out the piece myself I’ve got to say that it is hands down the best use of 3d I’ve seen online so far.

Red Bull Flugtag

redbull.com/flightlab

The biggest thing here is that it doesn’t feel like 3d is used just because it’s 3d, but for a reason. The rest of the site holds up to the high standard it sets too. As frosting on the cake the site supports deeplinking too and has a fully functional browser history. A big hand to Lessrain.

Read their blogpost about the creation too. Good stuff :)

FDT 3.0 out — in three flavors

My tool of choice, FDT 3.0, has been officially released. No more beta! I was in the beta test program and it has been very exciting to see the progress of this wonderful tool. As I said before, if you haven’t tried it out; please do so. There’s a free 30 day trial too.

FDT Logo

http://fdt.powerflasher.com/

FDT 3.0 packs a whole bunch of features which put the older 1.5 to shame (I won’t even try to compare it to flash, or even flex builder). Features include advanced code completion, an automatic formatter, quick fixes & assists, templates, organizing imports & automatic importign, launchers, semantic highlighting and much, much more. The biggest one though is definitely AS3 support which works like a charm. In short, it has changed the way I work. If Actionscript is the language that brings the bread to your table, go give FDT a spin.

I was kind of surprised to see the split into different versions. While I understand this step as a good “excuse” to make some extra cash (which is well deserved), i find it kind of annoyed. I own a license for FDT 1.5, which I bought when I read that the upgrade will be 99€. Now, however, I’m reading this is just for the basic version which is kind of disappointing. 599€ for the enterprise version is still quite a bit more (i definitely want the debugger and advanced refactoring tools).

My take on Hydra

Loads of new stuff has been happening. As you probably know, Adobe announced loads of more goodies for us at their Adobe MAX conference.

Native 3d support. Although it won’t be (most likely — according to Justin Everett-Church) hardware accelerated it will still mean that in a year or two we won’t see sites that don’t have 3d in them, in some form. I’m guessing the behavior is pretty similar to the basic 3d we see in After Effects today and that this will be an addition—not a replacement—to Papervision.

Improved text support. This one’s a big one. I don’t really need the right-to-left text, even if i can see it’s huge in other parts of the world, but support multiple columns and inline html tables — sweet! In addition you can build your own stuff on top of the api’s, extending the capabilities even more.

Custom filters, blend modes and fills. That’s right. Astro brings support for Hydra, a new pixel shader language that lets you build your own filters that will then run at native (same speed as blur, dropshadow etc) inside flash. Wow! This is pretty amazing. The possibilities that something like this brings us are pretty remarkable.

    The best part is that we don’t even have to wait to get our hands on Hydra, as you can download it from Adobe labs right now. I tried it out today and created a levels filter in Hydra which works (as far as i know) exactly the same as in Photoshop. Here’s the result on an image i took in Yosemite:

    Hydra levels

    Levels source code

    The language itself is very simple even if it’s pretty far from Actionscript (then again if you’ve used other pixel shader languages such as HSLS for Direct3D, you should feel right at home). I think the hard part is more about getting your head around the way a pixel shader works and thinking of the technique of what you want to do, not so much how to do it.

    Check out Aral Balkan’s post from Adobe MAX 2007, along with a video of Emmy Huang’s and Justin Everett-Church’s presentation. Also make sure you get some of your questions answered by Justin in the interview by Lee Brimelow.

    FDT 3.0 public beta

    I’ve been working with FDT for quite some time. There’s no going back to any tool I’ve ever tried (then again I haven’t worked in Flashdevelop, which many developers working on Windows recommend). Still, FDT is awesome, it has changed the way I work. I simply get more error free code out in less time.

    Now FDT3 is now open beta, meaning anybody can try it right away! If eclipse is your tool of choice, go give it a test drive.

    http://fdt.powerflasher.com/beta/

    FDT3 will be 299€ or 99€ when upgrading from 1.5.

    Of course you should realize that it’s still in beta status, so judge for yourself if you want to use it for production work. The private beta has been updated quite frequently and sometimes introduced bugs that made work quite annoying. Keep this in mind ;)

    Slow motion flash for debugging

    So, here’s the deal. You’re working on something in flash, you test several simultaneous scripted (or not) animations out and notice something’s wrong. There’s some weird overlapping or something going on there, but it’s really hard to tell since it all happens so fast. Sure, you could leave it there as it’s barely noticable, but we want to make products that aren’t just good but perfect.

    So, how do we solve this? Well, by now—since you’ve read the title—you probably know where I’m heading. Me and Svante from ACNE Digital (the company i worked for before) were talking about this and i decided that i’d come up with some slick solution that’s easy and quick to use whenever this happens. So, that’s what I did. And of course to support open sourcen and help other people make better products, you’ll get it too. For free as always.

    Recorder.as

    The Recorder is a pretty simple class that let’s you specify an area to record and then simply record that into RAM, which you can then play back at the desired speed. To keep it easy to share I put everything in one class, including the recorder, the playback, and the gui with an animation engine. This of course makes it more difficult/annoying to create & modify, but it’s easier to download and use as it’s only one file.

    So, how do you use it?

    Well, it’s pretty simple. Place the class in a folder /com/anttikupila/debug/ in your classpath. Then just import the class, and call Recorder.initialize( );. That’s it.

    [as]
    import com.anttikupila.debug.Recorder;
    Recorder.initialize( );
    [/as]

    Here’s an example with some motion graphics, courtesy of Alexander Petterson (thanks Alex!):

    To try it out. Click inside it first to give it focus, press and hold space, then drag an area to record and click the area. Then press space again to stop recording and start playback. Adjust speed with the up and down keys or with the dial to the right. You can hide it again by clicking space, which obviously also frees all resources required (this thing’s really ram-hungry). If you want, you can download the source to the example too.

    There is really no limit on the length (except your ram/patience), but I’d keep it pretty short – basically just for the time you see the problem. After this it’ll start playing those frames in a loop at the original speed. You can adjust the playback speed from the dial to the right.

    Now I know some people will say that there’s no timeline etc. But people, think about it; do you really need it? I don’t think so, and it looks nicer like this :) Less is more, ‘aye. Still, if you want, feel free to add additional controls (if so, please help everybody else and share the code).

    Actionscript 2.0 only at the moment, sorry about that. Still, it’s very easy to port to AS3 if you need to do so. Also i want to mention that this is something i made in like one or two days, so please don’t use it in any production environment as it is. It’s meant as a debug tool so bug threshold isn’t the same as on production sites ;)

    I’d love to hear your comments & suggestions

    Enjoy :)