I think it was in 2004 when I started investigating, learning the maths and testing all this stuff of projecting 3d points into a 2d plane... Two years later, in 2006, I had a torus knot with flat shading with a light and a camera. Then Papervision3D came into the scene and I got distracted ;)

Two years later, in 2008, I reworked my code with the things I liked from Papervision3D but making it simpler to my mind. At that point I was calling the engine threedoob and I did a bunch of experiments with it. Later I ported the code to Javascript and also did a bunch of experiments with that version too.

Now I think I've reached a point in which I'm not ashamed of sharing the code and because doesn't seem to be much resources for rendering 3D with HTML5/Javascript I thought other people would find the javascript port useful.

So I've created a repository for it at github. It's still very early days, but feel free to take a look at it, use it, fork it, ...

42 comments

I've asked many times to Sole about the benefits of using git and every time I thought it wasn't worth the learning time. I was happy with svn, but for Harmony I thought about giving github a try as Google Code didn't seem to make collaborations and forking too easy, plus github worked better with the way I was using it (different projects in the same place).

I guess Google Code works well for big projects, but in my case I do small projects/snippets, and creating a new repository/page for each one was a bit annoying as they also check if a project with the same name exist on sourceforge, and if it does you need to email the owner for permission first (although my project wouldn't be hosted on sourceforge but on Google Code).

Anyway, here it's the link for my github page:

http://github.com/mrdoob/

So far I'm liking it, and I've started moving my projects there. First one to move was stats.js, but expect the stats.as, videoplayer and so on to be moved soon. I've found github a bit hard at the beginning (ssh, keys and so on) but once that part is done is quite simple. Oh, and it's great to see the .svn folders go away!

7 comments

Ever tried to make an online drawing application and got frustrated because you couldn't access the Wacom tablet pressure? I know the feeling :)

Although I'm a bit surprised of how little publicity this thing had (none?) seems like Wacom has had this ready for quite a while (Windows/MacOS only) and with the latest drivers it should all work smoothly.

If you have a Wacom tablet, take a look at these demos. To learn how to integrate it into your application, here it's the info but I found easier just to take a look at the sources of this demo. If you're a Flash/Flex developer, external interface is your friend, at least until Adobe implements it within the player.

Of course, I've implemented it already in Harmony ;)

7 comments

And you may be wondering, where does all this thing about drawing tools come from? Good question!

Although I've always had interest on developing a drawing application, it all started when Aaron Koblin asked me if I would be up for developing a drawing tool for a Johnny Cash homage project he was planning with Chris Milk. I think this was about 6 months ago, even before the Startbucks Love Project (where an alpha version of the drawing tool was used).

Before passing away in 2003, Johnny Cash recorded one last album which was never released, until now. The idea was to build a platform where everyone would be able to draw a frame for the musicvideo for one of the tunes.



The website (front-end and back-end) was built by @radical.media so my work on it gets simplified to the repainter and painter system.

Repainter

One of the requirements for the drawing tool was to record how the frame was being drawn so the visitor on the site could see a timelapse. For example, here is a drawing I did:



There is small aspect ratio bug on that one, but you get the idea...

Doing this took a bit of time, but luckily I had learnt a few tricks some weeks before after trying to port the Colors! Java Repainter applet to AS3.

Creating a file format turns out to be quite simple. Just a matter of deciding a few tags (size, color, opacity, position...) and deciding how many bits you need for each value. For instance, I have STROKE_START which has the id 4, we do bytearray.writeByte(4) and then we save the positions of the mouse, but because these values can be bigger than a byte (256) we need to use int bytearray.writeInt(x) and bytearray.writeInt(y). We do the same for the rest of data and at the end we use bytearray.compress() which is pretty much a zip. The end result is a very simple and optimised file format, faster to parse than a xml and much smaller.

Having a hexadecimal editor is a must here (I used GHex). Here is a screenshot of how the uncompressed file format looks like:



If you know about file formats, I'm sure you already know what's going on, otherwise there you have a quick challenge for lunch time ;)

Painter

Having the file format sorted out I "just" had to develop the drawing tool itself. This may sound an easy task but, starting with the premultiplied alpha nightmare (more info) it has a bunch of little challenges... undo/redo, custom brushes, zoom, panning, brush outline... it just takes its time.



Using the original frame of the video as onion skin we can get nice results in a few minutes.



Now I can't wait to see how the project evolves and hopefully seeing the video without missing frames soon!

42 comments

This is the result of all the weekend playing with various drawing algorithms and <canvas>.



I'm sure you prefer playing with the thing than watching a video tho... Here here!

The whole thing is quite modular so I can keep adding more brush styles whenever I get inspired. During the process I found out that, for some reason (apparently lack of hardware acceleration), Firefox and Opera do not support context.globalCompositeOperation = 'darker'. This was on the HTML5 spec before but got removed. Just so you know what I'm talking about, this is like the "multiply" blending in Photoshop. Webkit does support it tho. I hope they put it back on the specs and all browsers support it.

I went to #processing.js to ask about this and while there sephr, apart from answring this same question, suggested making the app work on Webkit mobile (that means Android and iPhone) and also pointed me to the right direction. I didn't know this was possible at all but it was quite an easy task :) I haven't implemented multitouch support yet tho... that changes a bit the way my system work but I'll put it on the list.

Take a look at the code, play with it and if you come up with a nice brush you think should be added to the default pack, please, send it over!

EDIT: Thanks to Elke for creating a Facebook Fan Page. There is also a Facebook Group :D

EDIT 2: Yet another Facebook group. RRR, with a theme everyday and really talented artists.

291 comments

Whenever I work on a html/javascript experiment I usually test it on Chrome. Once I'm done, I test it on Firefox and Opera. There is no Safari for Linux so I don't have an easy way for testing it so I usually assume it works as it's very similar to Chrome. The interesting thing is that the same code usually works directly on the other browsers. Just special cases like adding custom css properties (like -webkit-transform, MozTransform, OTransform...).

Of course, that's not the case with Internet Explorer. And to be honest I never really tried to make anything work on it. I believe that browser has made humanity lose a lot of time and money so making my experiments work will be like helping it to stay alive.

But I wasn't showing any error message. My experiments will just not work and people will mention it and complain. I really didn't want to add Javascript browser checks in each experiment either. However, there was a much simpler and efficient solution, add a rule on the .htaccess to redirect all the request from that browser to this page.

By banning all the request of that browser, not only I'm happier when developing but also I may move some of these users to a better browser, and at the same time, other developers won't have to suffer from these users.

Imagine if all the major websites did this. Wouldn't that fix the problem?

If you find the move interesting and you would like to stop supporting IE too, this is the code you'll need to put in you .htaccess file:

RewriteEngine on

RewriteCond %{REQUEST_URI} !/projects/ie/index.html$
RewriteCond %{HTTP_USER_AGENT} MSIE [NC]
RewriteRule ^(.*)$ /projects/ie/index.html [L]

Just change the /projects/ie/index.html parts with wherever you put your own message/page.

EDIT: After Microsoft released a preview of Internet Explorer 9 some people suggested that I should remove the ban so they could test my HTML5 demos on the browser. That seemed a good idea, so the ban is removed. The bad news is that the browser doesn't seem to be doing too well.

42 comments

As Quasimondo rightly pointed out history keeps repeating. Specially on the web. Over the last decade we've been repeating what off-line graphic programmers have done two decades ago. Even worst, with Javascript we're repeating what Flash was doing half a decade ago.

In the case of Flash, the reason of why history repeated is because Flash is software rendering. Meaning that it's the CPU the one processing all these 2D/3D effects. Meanwhile, all these off-line graphic programmers had been enjoying from hardware rendering where the graphic card is outperforming the CPU in many levels.

In order to avoid history repeat itself we should know history on the first place. Watching what the people have done in the Demoscene, Processing, etc worlds helps. However, it's very tempting to just remake something for the sake of showing that it's possible. And I personally don't see anything wrong with it as it's good for learning. But we should focus on creating new things.

An example of this was back on the early days of Papervision3D. People were complaining that the engine didn't have lights or shadows. Amiga computers neither had enough power to efficiently render 3D scenes with lights and shadows but they managed to fake it by prerendering the shadows in the texture. That was the reason I did the pink ball experiment. I knew the trick because I knew this Amiga demo from 2006.



Now that hardware rendering is getting closer to be default (no plugins) on the web with WebGL, the possibilities are going to increment a lot. But we don't want to have a rotating cube on the screen, we should learn from what all the off-line graphic programmers have been doing with these possibilities and continue from there.

Here it's a selection of hardware accelerated demos from the last decade. I've sorted them (more or less) in chronologic order.

Remember, all these videos run in realtime on their original form. I'm aware that presenting them here in video form devaluates the work but it's just easier to watch.































Oh my... mix all these tricks and aesthetics with interactive experiences, audio visualisation, data visualisation, ... and that's a nice looking future for the web! :D

7 comments

Yesterday night I had the idea of retrieving all my twitter follower avatars and create an image out of it. Not an amazing concept but it was playful enough. I wanted to go down into quadtrees and play with the composition behaviour a bit but that would quite a bit of time and brain energy... I may go into it later.

Long time not sharing the sources in a .zip file! ;) Nothing too interesting tho.

Anyway... Happy holidays and a great 2010!

28 comments

For a quick project I was asked to revisit my old blobs effect. The client (which, again, sells flavoured liquids) wanted a xmas greetings e-card, so the idea was to play with the liquid that would form specific shapes.

Animating was a matter of using eaze and its beziers, but making it look a bit realistic took most of the research time.

Here it's how the old version of the effect worked:

As I'm sure you had guessed already it's all based from a bunch of circles.



Then we apply a BlurFilter to the Sprite that contains the circles.



And then by copying it to a BitmapData we apply a threshold and a colorTransform.



That's it. Simple enough for a quick experiment...

This i's how the new version works:

What we had so far happened to be a bit too CPU intensive for what it was. If we were going to make it even more complex we had to optimise. We had to remove the BlurFilter step somehow.

Looking at how the Sprite wth the BlurFilter applied looked like, I though that maybe I could recreate a very similar look using pre-blurred circles.


That way, instead of doing a blur pass to a big Sprite, it would be just a copyPixels using this bitmap as source with the positions where the original circles are. The result looks like this:



Which, for the human eyes is very different, but for the computer is very similar. (I could play with brightness/contrast to make it more similar for the human eyes).

Another step for this one was to apply a colorTransform to the working bitmap instead of filling it with white every frame. By doing that we would win "trails" which would improve the liquid effect, and, at the same time doing a kind of brightness/contrast for us.



At this point we apply the threshold again.



This is pretty much like the old effect, but much faster (no fullscreen BlurFilter). Now I could make the effect a bit nicer emulating reflections. I tried many ways but in the end this 2 step process was the fastest and better looking.

First step is to apply a BevelFilter to the output of the threshold.



Doing a BevelFilter is quite slow, but I can't think of a better way.

The fact that we're using reds here is not because the final color is going to be red (which it wasn't for the client). This BevelFilter uses 0xff0000 for hightlights, 0x000000 for shadows and the threshold was 0x800000. That's basically a grayscale but using the red channel. And you may wonder why.

Well, there is this (fast) method called paletteMap that does a very interesting thing. You can map a colour channel to an array of colours. In fact, you can map all of the channels to arrays of colours but I don't know what I could do with that (yet). Lets keep it simple for now.


This is the strip of colours I'm using. Just to make it clear, what paletteMap will do is to replace all the reds with the colours in that strip... as in, 0x000000 will use the colour[0], 0x010000 will use colour[1], 0xff0000 will use colour[255]... and so on... we end up getting this:



Surprised? No? Well... I was. :)

4 comments

Over the past few weeks I've been busy working on the Starbucks Love Project website for Tool and BBDO.

I had the pleasure to work with a great team: Aaron Koblin, Hello Enjoy, Medios y Proyectos and We Are Mammoth.

During the development of my bits on the site I faced some little challenges that I though I would share so other people could consider my solutions as options for similar challenges.

The website had many release dates. This is something I though it was going to be hard to handle, but the guys at WAM managed it smoothly. However, the project had basically 2 phases.

Phase I - Love Gallery & Drawing Tool

My responsibilities for the first phase were to build the love gallery and the drawing tool.



The idea behind the love gallery was to collect love-related drawings submitted by the people, very similar to Aaron Koblin's The Sheep Market. Opposite to The Sheep Market, in this case there was no limit to the amount of submissions; that was already quite a challenge to start with.

The visualisation part wasn't decided at the start, interfaces like Google Maps (tiling) were considered, but in the end the guys at BBDO decided that it had to be similar to Cooliris. If supporting infinite amount of items wasn't challenging enough now we had to emulate what a some guys have done with OpenGL (hardware renderer) but with Flash (software renderer).

Instead of using Papervision3D or Away3D I decided to use my own engine as it was easier for me to tweak and remove computations that weren't needed. You can't have many 3d planes (specially not infinite) so the way it works is just by having 40x10 3d planes that offset the x positions depending on the camera position. Using their x,y positions I can figure out what's supposed to be their ID and then load the data from the nicely setup VO by the WAM guys. This sounds straight forward, but the code ends up being quite messy.

Loading lots of little images from the server was also another challenge. You spend more time asking the server for an image than receiving the image. To make this a bit more smooth we composed big images (Big Ass Images as the Americans liked to call them) with hundreds of little images.



So with just one request to the server we would receive lots of images inside an image. However, we did some bad decisions here. The dimensions of the big images are way too big (2800x2850) that it takes too much load time and affects the experience. And the one that pisses me off more is that I didn't realised in time (thanks anyway Mike!) that having those big images as .png or even .gif would have not only make them look better but also even have smaller filesize. Look at the images, look horrible! Oh well... next time.

Next was interaction. The people had to be able to select items on the 3d wall. That means projecting 2D points to 3D coordinates. That's something I didn't have on my engine, neither it was something for my mind to easily understand, so I took at look at how Papervision3D and others did it, and when I managed to understand and implement it, it wasn't working 100% well, there were some angle computations that had some delays and time was running out. So I went to the good old Color ID route.

What? Color ID? Yes, Color ID. What we're after here is to know what's 3D object is under the mouse. So in every frame we render the scene twice. Once using a big plane that covers the whole wall textured with a bitmap that has big pixels (that match each plane) and that has a unique color. Mapping the blue component with the X and the green component with the y... we will have... top left plane will have the color 0,0,0, second from the top 0,1,0, fist plane on the second column will have the color id 0,0,1 and so on.



This is how the first pass of the render looks. The user never sees this because as soon as it's done we copy it to a bitmap and then draw the real scene on top. If you look close you can see that the right of the wall is getting blue. That's because the Color ID is reaching 40. So having this in a bitmapData is just a matter of doing a getPixel(mouseX, mouseY) and we easily get the ID by using the coords get get and apply the camera offset. Then we know which plane we should animate/select. No complicated 3D to 2D formulas, just one getPixel per frame. Fast.

(Actually, after writing this I realise it could have been simpler by using the Plane ID as color instead of mapping the blur component to X and green component to Y... live and learn).

There was yet another little challenge. This was more a design challenge than dev, but somehow nobody managed to come up with a scrollbar that could scale to infinite amount of items. Once we started to have 15k drawings, moving the scrollbar one pixel would move the whole wall 100+ items which would mean a lot of movement and specially a lot of bandwidth because the user reached an area were more items needed to be loaded. The solution got implemented at the end of Phase II after I found this article.



The drawing tool was an easier task. Specially because for another project I had a drawing tool started and many of the challenges were already sorted. Explaining all the tricks used on the tool would require an article by itself, so maybe I'll talk about it as soon as I finish the other project.

Phase II - Map & Video Wall

For the second phase the site was going to collect videos sent from many countries where people would sing to the 'All you need is Love' track. (I wonder what's with this song, I'm hearing it in every AD campaign this Christmas, maybe it's an AD trend). My tasks for this one was to make a Map and another infinite wall (this time 2D).



Here I had a good idea. We needed to place arrows where countries are. An easy way would have been doing a little tool so whoever could place them. But if we had to change the shape of the map (which we did), we would have had to remap the arrows. Instead we used longitude/latitude values for each country (which are easy to find out on the net).

Usually there are 2 kind of world maps. One that has the poles distorted (Equirectangular projection) and one that hasn't (Mercator Projection, like Google Maps). The first one is good when mapping a sphere, but please stop using it on 2D interfaces.

Only thing to find out was the formulas to translate latitude/longitude to X,Y. The formula for the Equirectangular one is easy:

private function getPoint(lat : Number, lon : Number, mapwidth : uint, mapheight : uint) : Point
{
	return new Point(((lon+180) / 360) * mapwidth, ((90-lat) / 180) * mapheight);
}

The Mercator one took a bit more to find out, but just for the sake of having it here easy to reuse in AS3:

private function getMercatorPoint(lat : Number, lon : Number, mapwidth : uint, mapheight : uint) : Point
{
	return new Point(((lon+180) / 360) * mapwidth, ((90-GudermannianInv(lat)) / 180) * mapheight);
}

private function GudermannianInv(lat : Number) : Number
{
	var sign : Number = Math.sin(lat) > 0 ? 1 : -1;
	var sin : Number = Math.sin(lat * 0.0174532925 * sign);
	return sign * (Math.log((1 + sin) / (1 - sin)) / 2) * 28.6478898;
}

Having this sorted out, as long as the designers didn't distort the map the mapping of arrows would be always easy task.

At this point I needed to add animations. Coincidentally, eaze was just being released so I used this as a excuse to give it a try. Verdict, perfect.



Then it was yet again going back to optimising a wall that would have infinite entries. However, I didn't stress it too much this time because there were 200 walls instead of just one, so the amount of items would spread along these making it less CPU intensive.

Used again eaze to animate everything and again proven to be great.

All in all, it was quite a big project done in a very short time and I think we were lucky that we all had quite a bit of experience, otherwise we wouldn't had been able to pull this one off.

I couldn't finish this extremely-long post without sending some props to the WAM guys for their DoneDone. tool which I hope they make free for public projects at some point. It proved very efficient. Think of it like the Basecamp of Issue trackers.

24 comments