Coming soon to a photoshop near you.

March 9, 2005 2:50 PM

Easy colorization of photos and videos, with examples. Matlab code for algorithm available. [via]
posted by monju_bosatsu (29 comments total)

Wow, that is extremely impressive.

It'll be nice when there's a filter for photoshop.
posted by Vulpyne at 3:09 PM on March 9, 2005

If that's for real, that's phenomenal. Wow.
posted by vernondalhart at 3:10 PM on March 9, 2005

I, too, have trouble believing this is real. If it is, it's nothing short of mindblowing.
posted by goodnewsfortheinsane at 3:13 PM on March 9, 2005

How cool is that.

(And I'm presuming that the fact that they've made the algorithm available is a good indicator it's legit, right? Anyone have Matlab to check it out?)
posted by LairBob at 3:15 PM on March 9, 2005

I mean, come on - even if this is real they must have used desaturated photos and fed the algorithm vectors based on the (human) operator's knowledge of the original colour photos. Only the "Crater Lake" and "Birthday" videos look a little off: mostly because they look a little oversaturated (especially the birthday one).

I am skeptical or impressed, or both, not sure.
posted by goodnewsfortheinsane at 3:18 PM on March 9, 2005

Very impressive, a quick scan of their paper doesn't reveal any tomfoolery though I'm not quite clear on how they're doing it yet either, I look forward to reading it when I have the time. Thanks Monju!
posted by fvw at 3:19 PM on March 9, 2005

Sweet, nice link monju_bosatsu. I fired up MATLAB, but I think the scripts requrie the Image Processing Toolbox. Which I don't have. [on preview:] To the naysayers, it's an algorithm, folks, not magic!
posted by fatllama at 3:22 PM on March 9, 2005

I'd appreciate it if they'd also include a few colour input videos with a differing colour output like they did with the stills, just to bypass the feeling among viewers that they're trying to match an original colour spectrum.

If anyone needs me, I'll be in the uncanny valley.
posted by goodnewsfortheinsane at 3:23 PM on March 9, 2005

Amazing!
I thought it was BS at first as well. Especially with the reflected color in the glass of the red bunny.
On closer inspection, I saw that they had indeed painted in very desaturated red lines in that area to give it the color (same for the rocks and curb in that same shot).
Someone please port it to AFX!
Thanks for this.
posted by numlok at 3:30 PM on March 9, 2005

Okay, so there are colour originals shown in the pdf. I'll stop overreacting now.
posted by goodnewsfortheinsane at 3:31 PM on March 9, 2005

Or, you could just shoot your pictures and videos in color to begin with. They have that feature now. But seriously, it looks impressive. I guess it would be useful if you got crappy source material like in low light situations, etc.

neighboring pixels in space-time that have similar intensities should have similar colors.

This seems like a bold assumption to me, but unless these samples are bs, then I guess it checks out.
posted by edlundart at 3:41 PM on March 9, 2005

I've got the IP toolbox, and I'll take a look at it now - looks pretty freaking cool - and also (!) really useful for what I'm working on right now. Woo!
posted by metaculpa at 4:07 PM on March 9, 2005

If this is for real, just imagine what the movie industry's gonna do now. Colorizing old black and white films is already big buisness in Bollywood, India.
posted by chime at 4:07 PM on March 9, 2005

Well, the code checks out.

I changed some of the colours in the example photo and the output came out ok. I tried to change the shirt to red but the output was a little too bright to see the texture that was there before...

It took 12 minutes to run on my very crappy computer...
It can run faster with the Multi Grid solver, but I didn't compile it...
posted by toftflin at 4:08 PM on March 9, 2005

Back in the mid 80s, as an otherwise unemployable art school graduate, I worked for 3 years for Colorization Inc., the company that trademarked the term. The company employed dozens of people working shiftwork, keeping 20 production 'tracking' machines running 24 hours a day.

Trackers, paid a bit more than minimum wage, would sit down at a station at the beginning of their shift and pick up where the last perons left off. The scene would be advanced 2 or 4 frames, and they would carefully adjust the colour layer based on the movement from the previous painted frame. Trackers would manage to get somewhere between 1 and 10 seconds done in a shift, depending on the visual content of the scene.

Good trackers were promoted to 'First-framers' who carefully painted the first frame of scenes, based on key frames painted by art directors. After tracking, another role would combine the colour with the black and white, and send poorly painted scenes back to trackers for correction.

It would take about four months for a movie to make it through production, consuming 1000s of hours.

All of that preamble is to explain why -- even though there is still no compelling reason to colorize film -- I find the technology here both stunning, and deeply funny, for personal reasons. Back then we had to deal with the fact that our labour produced nothing of value. That labour seems even more wasted now.
posted by KS at 4:14 PM on March 9, 2005

This is great. What's especially neat is that something similar to this might go on in human vision.

The reason that black-and-white pictures work at all is because two spectrally dissimilar objects are rarely illuminated to the same degree. When we evolved a third type of cone in our eye (a pretty recent development), we didn't do so out of nowhere; we could already see red fruit among green leaves, and the third cone type just made discrimination easier. We know that the brains of colorblind people (and noncolorblind people to a lesser extent) often use luminance cues to determine colors in the absence of spectral information. This is part of the reason why colorblind people often don't know that they're colorblind - they're able to use luminance information to play proxy for the spectral information they lack. Our brains obviously don't run this exact Matlab algorithm, but there's something similar in our heads that is able to convert luminance information into phenomenal color. I'd be interested to see someone port these findings into A.I. or neurophysiology.
posted by painquale at 4:23 PM on March 9, 2005

Is every one of these run on a color original that was desaturated digitally? Why didn't they color a couple turn of the century portraits to give us the full taste?
posted by rafter at 4:44 PM on March 9, 2005

This is fascinating. I love good algorithms. But I still find colorization a little eerie.
posted by Wolfdog at 4:46 PM on March 9, 2005

Here's Lena for you geeks out there - not quite what I was expecting, but interesting nonetheless.
You can still see my original scribbles, and they kindof swamp the background color. I don't know what's up with that.


posted by metaculpa at 5:26 PM on March 9, 2005

there is still no compelling reason to colorize film
posted by KS at 7:14 PM EST on March 9

I'm glad I'm not the only one who finds this a little troublesome. Is it to correct mistakes? I can see that. Is it to colorize pics that someone else took? That's bad karma, IMO.
posted by BoringPostcards at 7:48 PM on March 9, 2005

there is still no compelling reason to colorize film

I can't disagree more here.. Looking at the color photos of WWII that were recently posted here really helped me think of the events as 'real' and not just 'history'

I know it the fictionalized information, that the colorization of period pictures is as at the same level those history channel dramatizations. The trouble is, those dramatizations really do help. I feel that it's something neural, that looking only at B&W photos changes how I process the information.

Colorization of historical photos could really help bring history to the people.
posted by PissOnYourParade at 8:13 PM on March 9, 2005

Yowch, Lena looks terrible. I think you should have coloured the light and dark parts of skin separately (and the lips), but the feathers where your scribbles can't be faulted look pretty bad too.
posted by fvw at 9:04 AM on March 10, 2005

My grandfather would have been awed by this technology (he died 20 years ago). In the 50s and 60s he, in one of his many careers, would hand colorize photographs as a work-at-home contractor for portrait studios. I remember seeing his dining room table covered with graduation photos that he'd painstakingly tinted using colored pencils.

Now there's a job you don't see much call for any more....
posted by Daddio at 11:09 AM on March 10, 2005

metaculpa: it looks like some Matlab problem there; the scribbles shouldn't be visible. With Matlab 6.5 it worked for me, technically at least - see lenatest.jpg - but it clearly needs artistic skill and practice to know what needs to be painted.
posted by raygirvan at 11:42 AM on March 10, 2005

raygirvan: Your images suggest to me that the technique is indeed legit, but that not terribly easy to use well.

Still, extremely impressive.
posted by salmacis at 12:16 PM on March 10, 2005

I'd say two things..
If it's been published in Siggraph, chances are that it's legit. Their peer review process is pretty thorough, and the reviewers are highly motivated not to let bogus stuff through.

On the other hand, if it's implemented in Matlab, chances are it's not very quick. However, a reimplementation in a faster language (say, c++) would help the speed quite a bit. Matlab is pretty general purpose, and hard to optimize.
posted by tfinniga at 2:45 PM on March 10, 2005

Second try, a considerable improvement: lenatest2.jpg (I used colours eye-dropped from another studio portrait). This has serious possibilities.
posted by raygirvan at 4:59 PM on March 10, 2005

I ran it in Matlab and got some pretty good results. Recoloring doesn't seem to be implemented in the code they provided though, only grayscale colorization.

I did a Google search for other colorization algorithms and found Fast Image and Video Colorization using Chrominance Blending. It claims faster execution than the Colorization using Optimization method, and provides a comparison page with the two algorithms side-by-side. The chrominance method seems to give better results with fewer scribbles by avoiding the color bleedoff problem seen in the other. It doesn't provide any source code for an implementation of the algorithm, which is disappointing, but there's a java applet where you can try it out.
posted by RotJ at 8:32 PM on March 10, 2005

This technology would be spectacular for making selections as well it seems. As for colorizing old B &W film, I'm not for it as the reasons are to sell it to audiences that reject B & W. It entirely destroys the original cinematographer's vision so you're simply cheapening a movie that is already not appreciated by a particular audience for a rather subjective reason.

But great for restorative work perhaps, and its simply another tool for those who wish to use it in any number of ways.
posted by juiceCake at 8:56 PM on March 10, 2005