Content Aware Image Resizing
August 21, 2007 7:29 PM Subscribe
Content Aware Image Resizing. Every year SIGGRAPH rolls around I get a reminder of how smart everyone else is, especially people who do computer graphics research. From Shai Avidan and Ariel Shamir. The algorithm resizes images non-uniformly and, well, somewhat magically.
I'm not sure if you're joking, but seriously, that's pretty much where I've gotten to.
posted by GuyZero at 7:42 PM on August 21, 2007
posted by GuyZero at 7:42 PM on August 21, 2007
Wow, that's really, really cool. How long do I have to wait for a Photoshop plugin?
posted by scottreynen at 7:44 PM on August 21, 2007
posted by scottreynen at 7:44 PM on August 21, 2007
That's bloody phenomenal.
posted by vernondalhart at 7:46 PM on August 21, 2007
posted by vernondalhart at 7:46 PM on August 21, 2007
I am not a SIGGRAPH person, but this is amazing. Definitely watch the video, and watch the end (3:43) where the show how you can seamlessly remove people from a picture.
posted by blahblahblah at 7:49 PM on August 21, 2007
posted by blahblahblah at 7:49 PM on August 21, 2007
I just showed that to my web designer roommate and broke his brain. Thanks! I hardly ever get to break people.
Also, he said what scottreynen said.
posted by darksasami at 7:49 PM on August 21, 2007 [2 favorites]
Also, he said what scottreynen said.
posted by darksasami at 7:49 PM on August 21, 2007 [2 favorites]
The part where people are erased from a digital photograph is chilling — imagine how refined this technique could be in a few years. Memory hole, anyone?
posted by Blazecock Pileon at 7:53 PM on August 21, 2007
posted by Blazecock Pileon at 7:53 PM on August 21, 2007
Hmmm, if it only resizes images....
I Kid
posted by mattoxic at 7:54 PM on August 21, 2007 [1 favorite]
I Kid
posted by mattoxic at 7:54 PM on August 21, 2007 [1 favorite]
That is some of the craziest shit I've ever seen done to a photograph.
posted by danb at 7:58 PM on August 21, 2007
posted by danb at 7:58 PM on August 21, 2007
Bwah. That's pretty stunning, especially the "deleting people from photos" bit. I wonder if it leaves any tell-tale signs, though. You'd have to look pretty closely, but there must still be some subtle seaming somewhere.
Also, I wonder if the process could be run in reverse to add people or things to photos.
posted by wanderingmind at 8:01 PM on August 21, 2007
Also, I wonder if the process could be run in reverse to add people or things to photos.
posted by wanderingmind at 8:01 PM on August 21, 2007
I wonder if it leaves any tell-tale signs, though.
If you look carefully you can see them, even at YouTube resolution. The beach scene benefits because the sand pattern looks pretty random at low resolution, but at high res, there would be artefacts I expect.
But nothing that you couldn't fix in 60 seconds with a blur or healing tool in ye olde Photoshopee.
posted by GuyZero at 8:05 PM on August 21, 2007
If you look carefully you can see them, even at YouTube resolution. The beach scene benefits because the sand pattern looks pretty random at low resolution, but at high res, there would be artefacts I expect.
But nothing that you couldn't fix in 60 seconds with a blur or healing tool in ye olde Photoshopee.
posted by GuyZero at 8:05 PM on August 21, 2007
I'm not worthy! I'm not worthy!
Hat tip to the bright one who thought "let's get rid of the least important column of pixels" and then decided to get fancy about it.
posted by five fresh fish at 8:07 PM on August 21, 2007 [1 favorite]
Hat tip to the bright one who thought "let's get rid of the least important column of pixels" and then decided to get fancy about it.
posted by five fresh fish at 8:07 PM on August 21, 2007 [1 favorite]
Wow, that's pretty freaky.
posted by puke & cry at 8:07 PM on August 21, 2007
posted by puke & cry at 8:07 PM on August 21, 2007
Siiiiick. Simple but sick.
posted by lalochezia at 8:08 PM on August 21, 2007
posted by lalochezia at 8:08 PM on August 21, 2007
I too would love to see a full-size photo example, since YouTube videos aren't know for their image fidelity. The PDF of the paper is taking ages to download.
So, here are some more papers from SIGGRAPH 2007. I'm sure there' some fascinating stuff. I wish more of these guys would put their movies on YouTube... the quality does suck, which must kill them, but downloading huge movies isn't cool either.
posted by smackfu at 8:08 PM on August 21, 2007
So, here are some more papers from SIGGRAPH 2007. I'm sure there' some fascinating stuff. I wish more of these guys would put their movies on YouTube... the quality does suck, which must kill them, but downloading huge movies isn't cool either.
posted by smackfu at 8:08 PM on August 21, 2007
Great. And here I just blew a couple grand on a widescreen.
(They want I should turn it sideways now?)
posted by hal9k at 8:14 PM on August 21, 2007
(They want I should turn it sideways now?)
posted by hal9k at 8:14 PM on August 21, 2007
That's really cool. It sounds like it wouldn't be too hard to implement, even— the only difficult bit being the importance function. Maybe I'll change my mind after I read their paper [pdf].
posted by hattifattener at 8:22 PM on August 21, 2007
posted by hattifattener at 8:22 PM on August 21, 2007
I saw "Rising Sun" the other day. A movie (and book, probably) whose entire plot hinges on it being inconceivable that the Japanese dudes could have digitally removed someone from a video.
HAHAHHAHAHAHAHAHAHA, you suck, movie.
posted by Justinian at 8:28 PM on August 21, 2007
HAHAHHAHAHAHAHAHAHA, you suck, movie.
posted by Justinian at 8:28 PM on August 21, 2007
gah! Now I remember why I used to go to SIGGRAPH each year. I've missed 2 years in a row now (after 10+). I'm there next year - no matter what my boss says. :)
posted by tomplus2 at 8:31 PM on August 21, 2007
posted by tomplus2 at 8:31 PM on August 21, 2007
I don't know if I'm thrilled or horrified.
posted by andihazelwood at 8:36 PM on August 21, 2007
posted by andihazelwood at 8:36 PM on August 21, 2007
I want it. Wonder if it's available as a torrent yet.
posted by IronLizard at 8:43 PM on August 21, 2007 [1 favorite]
posted by IronLizard at 8:43 PM on August 21, 2007 [1 favorite]
At the end of the video, they show two examples of turning couples into single people.
I mastered the bachelorize technique years ago.
posted by rob511 at 8:49 PM on August 21, 2007 [4 favorites]
I mastered the bachelorize technique years ago.
posted by rob511 at 8:49 PM on August 21, 2007 [4 favorites]
Jesus. That's pretty amazing.
posted by equalpants at 8:55 PM on August 21, 2007
posted by equalpants at 8:55 PM on August 21, 2007
Surprisingly good results for a pretty simple algorithm. Computationally it looks roughly like a tree search per row/column, which should be pretty quick if the overlapping computations are optimized out via dynamic programming. If I heard correctly, the "live preview" was using a pre-indexed image, so I think it's safe to say that their implementation was not running at video-framerate-speeds... but I would guess this should run fast enough for something like photoshop.
I also wonder how novel this is. It's about what you would expect an image processing grad student to do if you told him, "Come up with an algorithm to get rid of the unimportant parts of this image." Which is not to say the result isn't cool.
posted by Galvatron at 8:55 PM on August 21, 2007
I also wonder how novel this is. It's about what you would expect an image processing grad student to do if you told him, "Come up with an algorithm to get rid of the unimportant parts of this image." Which is not to say the result isn't cool.
posted by Galvatron at 8:55 PM on August 21, 2007
Papers? I want the plug-in, and I want it about a decade ago.
posted by ZachsMind at 8:57 PM on August 21, 2007
posted by ZachsMind at 8:57 PM on August 21, 2007
Ooo, I remember SIGGRAPH at comic-con. They had M&M's.
posted by Citizen Premier at 9:00 PM on August 21, 2007
posted by Citizen Premier at 9:00 PM on August 21, 2007
The next meaningful label: OCI - Original Camera Image.
posted by Brian B. at 9:03 PM on August 21, 2007
posted by Brian B. at 9:03 PM on August 21, 2007
I also wonder how novel this is.
Once everyone stops slamming the download site for the paper, I assume it will be in the citations.
They skim over the part where they talk about other significance functions, but I imagine they had to try a few things before they got the right one. Research always looks easy in hindsight.
posted by GuyZero at 9:04 PM on August 21, 2007
Once everyone stops slamming the download site for the paper, I assume it will be in the citations.
They skim over the part where they talk about other significance functions, but I imagine they had to try a few things before they got the right one. Research always looks easy in hindsight.
posted by GuyZero at 9:04 PM on August 21, 2007
This is doubleplusgood!
posted by ryoshu at 9:05 PM on August 21, 2007 [1 favorite]
posted by ryoshu at 9:05 PM on August 21, 2007 [1 favorite]
The next meaningful label: OCI - Original Camera Image.
And when they implement it in camera firmware?
There are no "real" digital photographs - they have all been manipulated and altered. Some just more than others.
posted by GuyZero at 9:05 PM on August 21, 2007
And when they implement it in camera firmware?
There are no "real" digital photographs - they have all been manipulated and altered. Some just more than others.
posted by GuyZero at 9:05 PM on August 21, 2007
That was astounding.
posted by OverlappingElvis at 9:18 PM on August 21, 2007
posted by OverlappingElvis at 9:18 PM on August 21, 2007
Here's one vid on YouTube, Cloth Simulation, including dropping a piece of red silk on a horse.
posted by smackfu at 9:21 PM on August 21, 2007
posted by smackfu at 9:21 PM on August 21, 2007
Also, I finally got the PDF to download, and the images from the resizing look surprisingly good at higher res. They don't necessarily stand up to close examination, but the discontinuities don't jump out at you. Like when they cut the girl out of the street photo, you can tell something's not right, but not necessarily what it is.
posted by smackfu at 9:24 PM on August 21, 2007
posted by smackfu at 9:24 PM on August 21, 2007
And when they implement it in camera firmware?
We may assume that the camera used an original image to compose an alternate image, even if discarded. The slippery slope justifications between a texturally altered image and a contextually altered image is explicitly avoided with the terms original camera image.
posted by Brian B. at 9:31 PM on August 21, 2007
We may assume that the camera used an original image to compose an alternate image, even if discarded. The slippery slope justifications between a texturally altered image and a contextually altered image is explicitly avoided with the terms original camera image.
posted by Brian B. at 9:31 PM on August 21, 2007
Interesting work, for sure. But I'm not sure if I'm looking at a web page I want a stretched or compressed version of the original image if it's important that the original remain true to its context.
posted by maxwelton at 9:34 PM on August 21, 2007
posted by maxwelton at 9:34 PM on August 21, 2007
I work damn hard on composition and lighting and any number of other factors in my photography. I wayched this with a growing sense of dread... Damn it, I *want* that negative space in there. It's there for a reason. I cropped the picture the way I did *for a reason*.
Letting a machine fuck with that is just *wrong*.
posted by pjern at 9:45 PM on August 21, 2007 [1 favorite]
Letting a machine fuck with that is just *wrong*.
posted by pjern at 9:45 PM on August 21, 2007 [1 favorite]
I'm reminded of an old Mad magazine bit about the wonders of photo retouching; where an old image was retouched to remove scratches and other unsightly artifacts. In the original scratched and dusty image a small unhappy old man is standing beside a his wife, a huge angry battleaxe of a woman. In the retouched image, the man is happy and the image has no scuffs, scratches, or frightful old battleaxe.
posted by George_Spiggott at 9:45 PM on August 21, 2007
posted by George_Spiggott at 9:45 PM on August 21, 2007
GuyZero There are no "real" digital photographs - they have all been manipulated and altered. Some just more than others.
This is true, but it's a worrisome truth to chase too much further. The same applies to analog photos, to written descriptions, to spoken descriptions, even to one's own memories. Trustworthiness must necessarily be approached as a probabilistic, rather than boolean, problem. Preponderance of evidence, each item in itself untrustworthy, will tell a story that approaches closer and closer to truth.
The trouble is this really, really isn't how people naturally think.
posted by aeschenkarnos at 9:51 PM on August 21, 2007
This is true, but it's a worrisome truth to chase too much further. The same applies to analog photos, to written descriptions, to spoken descriptions, even to one's own memories. Trustworthiness must necessarily be approached as a probabilistic, rather than boolean, problem. Preponderance of evidence, each item in itself untrustworthy, will tell a story that approaches closer and closer to truth.
The trouble is this really, really isn't how people naturally think.
posted by aeschenkarnos at 9:51 PM on August 21, 2007
I cropped the picture the way I did *for a reason*.
And when you get back to the computer, and decide you didn't quite get it perfect, this is the tool you use to fix it.
posted by smackfu at 9:58 PM on August 21, 2007 [3 favorites]
And when you get back to the computer, and decide you didn't quite get it perfect, this is the tool you use to fix it.
posted by smackfu at 9:58 PM on August 21, 2007 [3 favorites]
Re: photo retouching. This also thrills and horrifies me. The effort and talent that go into this are impressive, and the result is definitely art. The implications though...
posted by andihazelwood at 10:00 PM on August 21, 2007 [4 favorites]
posted by andihazelwood at 10:00 PM on August 21, 2007 [4 favorites]
(I should mention that I am an aspiring photo retoucher/restorer and I consider Amy Dresser, whose website I linked above, to be the best artist of this genre that I've seen. She's also a very kind person- so she inspires me more than she frightens me, but only just. I think I feel the same about the OP.)
posted by andihazelwood at 10:04 PM on August 21, 2007
posted by andihazelwood at 10:04 PM on August 21, 2007
So with the enlargement functionality, you could make pretty nice-looking large prints of low-resolution digital images, right?
posted by stammer at 10:17 PM on August 21, 2007
posted by stammer at 10:17 PM on August 21, 2007
No, the enlargement works by expanding areas without visual interest, inserting pixels that are the average of the pixels around them. Using this technique you could get larger pictures in which the interesting bits would be further apart but not any larger.
posted by George_Spiggott at 10:22 PM on August 21, 2007
posted by George_Spiggott at 10:22 PM on August 21, 2007
OK, this may sound stupid, but think about it: suppose you applied this technique to text? Clearly, some words are more important than others, so by identifying the "less energetic" words and deleting them, you can shrink the text.
Conversely, by by adding dead space to to a a paragraph, you could extend the the text without significantly modifying the the meaning!
This would revolutionize the newspaper and magazine industries! Column inches filled to order!
posted by SPrintF at 11:01 PM on August 21, 2007 [2 favorites]
Conversely, by by adding dead space to to a a paragraph, you could extend the the text without significantly modifying the the meaning!
This would revolutionize the newspaper and magazine industries! Column inches filled to order!
posted by SPrintF at 11:01 PM on August 21, 2007 [2 favorites]
I found this really creepy, for reasons that only andihazelwood and aeschenkarnos have touched on. It brings to mind recent comments by Olafur Eliasson about the importance and implications of our concept of spatiality (couldn't find a better quote in english than this). That video was a chilling whisper of how happily we'll take fucking around with what's around us completely for given/granted, indeed how grateful we are for tools that'll help us avoid percieving things the way they are.
posted by progosk at 11:21 PM on August 21, 2007 [1 favorite]
posted by progosk at 11:21 PM on August 21, 2007 [1 favorite]
SPrintF, you're probably right. I don't know how far along that technology is (language being much dicier than pictures for selective editing), but I would guess it's a matter of time.
posted by McLir at 11:25 PM on August 21, 2007
posted by McLir at 11:25 PM on August 21, 2007
SPrintF, you're right. That technology is dicier, but it's a matter of time.
posted by BrotherCaine at 11:55 PM on August 21, 2007 [4 favorites]
posted by BrotherCaine at 11:55 PM on August 21, 2007 [4 favorites]
Criminy.
Combine facial identification with this tool and a digital photo album and you could remove a person from all of your pictures with a click. Of course, you might have to explain later why you are always dancing by yourself at weddings and what exactly you were doing zooming around on the back of a bicycle built for two, but the rest is eternal sunshine.
posted by pracowity at 12:07 AM on August 22, 2007 [1 favorite]
Combine facial identification with this tool and a digital photo album and you could remove a person from all of your pictures with a click. Of course, you might have to explain later why you are always dancing by yourself at weddings and what exactly you were doing zooming around on the back of a bicycle built for two, but the rest is eternal sunshine.
posted by pracowity at 12:07 AM on August 22, 2007 [1 favorite]
I also wonder how novel this is. It's about what you would expect an image processing grad student to do if you told him, "Come up with an algorithm to get rid of the unimportant parts of this image."
See, the hard part is coming up with the question.
When questions are good enough, answers are very frequently obvious.
posted by Malor at 12:13 AM on August 22, 2007
See, the hard part is coming up with the question.
When questions are good enough, answers are very frequently obvious.
posted by Malor at 12:13 AM on August 22, 2007
There are no "real" digital photographs - they have all been manipulated and altered. Some just more than others.
There are no "real" film photographs either.
I am (faintly) chilled that people can find this algorithm chilling - as that suggests a widespread unawareness of just how manipulated almost every image in the home already is.
posted by -harlequin- at 12:38 AM on August 22, 2007 [1 favorite]
There are no "real" film photographs either.
I am (faintly) chilled that people can find this algorithm chilling - as that suggests a widespread unawareness of just how manipulated almost every image in the home already is.
posted by -harlequin- at 12:38 AM on August 22, 2007 [1 favorite]
No naïvety here, harlequin. Images are, in a fundamental sense, selections/manipulations a priori. What chills me is that as the advaned techniques become speedier/easier, even less people will stop to think about how this affects their, erm, Weltanschauung.
posted by progosk at 12:50 AM on August 22, 2007 [2 favorites]
posted by progosk at 12:50 AM on August 22, 2007 [2 favorites]
SprintF, that is dicier, but it's time.
posted by Bugbread at 1:29 AM on August 22, 2007 [1 favorite]
posted by Bugbread at 1:29 AM on August 22, 2007 [1 favorite]
I also wonder how novel this is.
If you wonder at the wonder people are displaying here in the comments, you'll have your answer. I don't know how original the basic ideas are, but the package they've built on those ideas impresses everyone. You label someone as unimportant and that unimportant person fades into the landscape. Stalin would have killed for this.
posted by pracowity at 1:30 AM on August 22, 2007
If you wonder at the wonder people are displaying here in the comments, you'll have your answer. I don't know how original the basic ideas are, but the package they've built on those ideas impresses everyone. You label someone as unimportant and that unimportant person fades into the landscape. Stalin would have killed for this.
posted by pracowity at 1:30 AM on August 22, 2007
he said semen sertion
apart from that, um, cor, wow, splur
posted by criticalbill at 1:54 AM on August 22, 2007
apart from that, um, cor, wow, splur
posted by criticalbill at 1:54 AM on August 22, 2007
Stalin would have killed for this.
In Soviet Russia, images resize you.
posted by progosk at 1:59 AM on August 22, 2007 [2 favorites]
In Soviet Russia, images resize you.
posted by progosk at 1:59 AM on August 22, 2007 [2 favorites]
One of the coolest thing about this demonstration was how the researchers were able to present the basics of how the technology worked in a way accessible to laypeople.
posted by grouse at 2:22 AM on August 22, 2007
posted by grouse at 2:22 AM on August 22, 2007
Text compression has been around for a long time -- I recall shareware for it in the early 90s -- although there is continuing interest in improving techniques for use in keyword extraction and text summarization. If you have a Mac, you can play with it right now. Open a Services-aware application (such as Safari), select a block of text, and select appname -> Services -> Summarize.
SPrintF's comment (467 characters, 75 words) pasted into SummaryService with default settings reads like this (370 characters, 62 words):
SummaryService only plucks out whole sentences or paragraphs depending on setting -- it doesn't act on words, although other applications can.
posted by ardgedee at 3:25 AM on August 22, 2007 [5 favorites]
SPrintF's comment (467 characters, 75 words) pasted into SummaryService with default settings reads like this (370 characters, 62 words):
OK, this may sound stupid, but think about it: suppose you applied this technique to text? Clearly, some words are more important than others, so by identifying the "less energetic" words and deleting them, you can shrink the text.The edits are relatively transparent. At the 1% setting, like this (140 characters, 23 words):
Conversely, by by adding dead space to to a a paragraph, you could extend the the text without significantly modifying the the meaning!
This would revolutionize the newspaper and magazine industries!
Clearly, some words are more important than others, so by identifying the "less energetic" words and deleting them, you can shrink the text.The speculative voice is gone and all that's left is a simple statement of fact. Larger text samples can take more compression before the effect is noticeable. Whether you're missing important information in what's thrown out is not necessarily noticeable.
SummaryService only plucks out whole sentences or paragraphs depending on setting -- it doesn't act on words, although other applications can.
posted by ardgedee at 3:25 AM on August 22, 2007 [5 favorites]
suppose you applied this technique to text?
This technique has been manually applied to text for years. Artistic merit of the results may vary.
posted by GuyZero at 5:37 AM on August 22, 2007
This technique has been manually applied to text for years. Artistic merit of the results may vary.
posted by GuyZero at 5:37 AM on August 22, 2007
I think the creepiness factor here is not due to the possibility of being 'fooled' by an image, but rather by the trust people seem to place in images in the first place.
Renaissance painters where already messing with people's minds like this centuries ago, and yet people still have this naive faith in images as some sort of 'truth'.
posted by signal at 6:39 AM on August 22, 2007
Renaissance painters where already messing with people's minds like this centuries ago, and yet people still have this naive faith in images as some sort of 'truth'.
posted by signal at 6:39 AM on August 22, 2007
Given careful selection of low-energy columns you could make a nice digital Mad Magazine fold-in.
posted by rlk at 7:18 AM on August 22, 2007 [1 favorite]
posted by rlk at 7:18 AM on August 22, 2007 [1 favorite]
Wow! That's amazing and pretty straightforward when you think about it. It's just the sort of thing computers are great at doing - a brute force solution that has surprisingly elegant results. With enough processing power you could do it for video, in real time. Scary.
posted by jiroczech at 7:28 AM on August 22, 2007
posted by jiroczech at 7:28 AM on August 22, 2007
I cropped the picture the way I did *for a reason*.
We'll have to get good at painting with that "leave this part alone" tool.
I'm amazed at this, though. Imagine uploading one high-res file and having it scale up and down looking it's best at every dimension and resolution? Client-side? On a webpage? On your cellphone? In outer space? With a pony?
posted by cowbellemoo at 7:30 AM on August 22, 2007
We'll have to get good at painting with that "leave this part alone" tool.
I'm amazed at this, though. Imagine uploading one high-res file and having it scale up and down looking it's best at every dimension and resolution? Client-side? On a webpage? On your cellphone? In outer space? With a pony?
posted by cowbellemoo at 7:30 AM on August 22, 2007
The most interesting thing, to me, is that fuzzy boundary between successful resizing and failure. With the lake-and-mountain-and-sky example, they get a lot of mileage out of it in both directions, but when the push it too small, bam: horrible mushed up mountain monster.
Scale it out too large and you get this bizarre impossibly minimalism.
That's the stuff. Half-measure fail states between person-there and person-gone; profiles of different classes of objective and subjective cropping failure, and how the algorithm (or some meta-algorithm designed to track its success/failure, with human input) reacts to fail states.
SprintF time.
posted by cortex at 7:36 AM on August 22, 2007
Scale it out too large and you get this bizarre impossibly minimalism.
That's the stuff. Half-measure fail states between person-there and person-gone; profiles of different classes of objective and subjective cropping failure, and how the algorithm (or some meta-algorithm designed to track its success/failure, with human input) reacts to fail states.
SprintF time.
posted by cortex at 7:36 AM on August 22, 2007
Since nobody else has linked to here, here's a coralized link to a higher quality MOV of the video. Looks a lot better than the YouTube version (and it's in the original aspect ratio, since I doubt YouTube is using Content Aware Image Resizing for their scaling/cropping needs).
posted by skynxnex at 7:37 AM on August 22, 2007 [2 favorites]
posted by skynxnex at 7:37 AM on August 22, 2007 [2 favorites]
Image based projects demonstrated at Siggraph = Voodoo.
I can't seem to access their PDF, but I would expect to find buried in the code of the resizing program a software simulation of the slaughter of a lamb and the virtual drinking of its digital blood.
posted by 6am at 9:21 AM on August 22, 2007
I can't seem to access their PDF, but I would expect to find buried in the code of the resizing program a software simulation of the slaughter of a lamb and the virtual drinking of its digital blood.
posted by 6am at 9:21 AM on August 22, 2007
Awesome tech, but let me draw your attention to the fact that the comments on youtube are actually... sane?
What hell hath these men wrought?
posted by zap rowsdower at 10:09 AM on August 22, 2007 [1 favorite]
What hell hath these men wrought?
posted by zap rowsdower at 10:09 AM on August 22, 2007 [1 favorite]
That's awesome, thank you. Such a simple basic idea, so visually intuitive. I imagine the actual implementation was a bit more complex.
posted by Nelson at 11:06 AM on August 22, 2007
posted by Nelson at 11:06 AM on August 22, 2007
Oh my god, this is jaw dropping and amazing.
Imagine putting this on a photo that's on a webpage, allowing dynamic resizing? Or what if we used the "importance finding algorithm" on compression, allowing higher compression on less
energetic areas? So many uses for this. So so many.
The "Oh Noez a persons is erasered!!" comments make me a little sad. This is so much more than that. (Believe me, people can be photoshopped out very quickly. If there's enough background to do the seam transition, there's enough for rubber stamping over them.)
posted by Brainy at 5:25 PM on August 22, 2007
Imagine putting this on a photo that's on a webpage, allowing dynamic resizing? Or what if we used the "importance finding algorithm" on compression, allowing higher compression on less
energetic areas? So many uses for this. So so many.
The "Oh Noez a persons is erasered!!" comments make me a little sad. This is so much more than that. (Believe me, people can be photoshopped out very quickly. If there's enough background to do the seam transition, there's enough for rubber stamping over them.)
posted by Brainy at 5:25 PM on August 22, 2007
allowing higher compression on less energetic areas
This is already what JPEG does. It uses DCT, which is an offshoot of a Fourier transformation and essentially it compresses different spatial frequencies differently and throws some away. Thus the artifacts on hard edges when the compression is turned way up. But it's not aware of image-specific features I suppose.
Also, I encourage everyone to look at the paper, if only to see the sample photos in high resolution. The girl that gets rerased at the end of the demo reel? Even after being erased, the guy is still holding her hand. Her disembodied, severed hand.
posted by GuyZero at 8:42 PM on August 22, 2007
This is already what JPEG does. It uses DCT, which is an offshoot of a Fourier transformation and essentially it compresses different spatial frequencies differently and throws some away. Thus the artifacts on hard edges when the compression is turned way up. But it's not aware of image-specific features I suppose.
Also, I encourage everyone to look at the paper, if only to see the sample photos in high resolution. The girl that gets rerased at the end of the demo reel? Even after being erased, the guy is still holding her hand. Her disembodied, severed hand.
posted by GuyZero at 8:42 PM on August 22, 2007
I guarantee that no newspapers are interested in this technology. If I'm wrong, you can pay off all my credit cards. But I'm planning to pay them myself.
It's alarming to me that someone would think that image composition was a problem that software should remedy.
Both photography and software coding are arts as well as sciences. Artists should respect each other.
I don't tell you how many times to iterate through your subroutine. Don't tell me you don't like my use of Rule of Thirds...
posted by bugmuncher at 9:31 PM on August 22, 2007
It's alarming to me that someone would think that image composition was a problem that software should remedy.
Both photography and software coding are arts as well as sciences. Artists should respect each other.
I don't tell you how many times to iterate through your subroutine. Don't tell me you don't like my use of Rule of Thirds...
posted by bugmuncher at 9:31 PM on August 22, 2007
But [JPEG is] not aware of image-specific features I suppose.
Not as such, no. One of the advantages of JPEG 2000 is that it has a "region of interest" capability which allows the encoder to allocate more of the bitrate to important regions; this could be paired with any number of feature-recognition algorithms to create a "content aware" compressor.
posted by Galvatron at 10:40 PM on August 22, 2007
Not as such, no. One of the advantages of JPEG 2000 is that it has a "region of interest" capability which allows the encoder to allocate more of the bitrate to important regions; this could be paired with any number of feature-recognition algorithms to create a "content aware" compressor.
posted by Galvatron at 10:40 PM on August 22, 2007
So I spent a while hacking together a program to do this, based on the narration in the video. It really is about as straightforward as the description makes it sound. Now I, too, can creepily distort peoples' faces!
(This is just the straightforward 1-dimensional resizing, without any of the refinements they discuss in the paper— the gradient-domain technique mentioned towards the end sounds especially interesting. As they note in the paper, the results really aren't all that sensitive to the specific energy function you use, though I've found that plugging in totally crackheaded energy functions produces amusing results.)
posted by hattifattener at 5:01 AM on August 23, 2007
(This is just the straightforward 1-dimensional resizing, without any of the refinements they discuss in the paper— the gradient-domain technique mentioned towards the end sounds especially interesting. As they note in the paper, the results really aren't all that sensitive to the specific energy function you use, though I've found that plugging in totally crackheaded energy functions produces amusing results.)
posted by hattifattener at 5:01 AM on August 23, 2007
Sounds fun, hattifattener. Any chance you could throw the source code out there for the more lazy among us?
posted by Nelson at 8:27 AM on August 23, 2007
posted by Nelson at 8:27 AM on August 23, 2007
I guarantee that no newspapers are interested in this technology.
AFAICT, newspapers are interested in no technology at all. They think having a "blog" is new and avant garde. So I agree with you there.
posted by smackfu at 8:39 AM on August 23, 2007
AFAICT, newspapers are interested in no technology at all. They think having a "blog" is new and avant garde. So I agree with you there.
posted by smackfu at 8:39 AM on August 23, 2007
Smackfu:
Newspapers use a lot of technology. Maybe not ones you're enthused about, but let's look at some computer techs used by newspapers:
* Wikis for internal documentation and style guides.
* Color management software and profiles to make things print nicely.
* "Ethical" dodging and burning techniques in Photoshop.
* Wireless commincations, so stories and images can be filed from anywhere.
* CMS that enable reporters to post breaking news items to a newspaper's web site without having to learn HTML.
* PHPbb so the reader community can get together and bitch about stuff.
* Various video and audio codecs for delivery of audiovisual content
* RSS feeds
Not every newspaper gets it right. That's too bad. At my paper, we don't think of blogs as avant garde, but we do think they're necessary.
posted by bugmuncher at 8:59 AM on August 23, 2007 [1 favorite]
Newspapers use a lot of technology. Maybe not ones you're enthused about, but let's look at some computer techs used by newspapers:
* Wikis for internal documentation and style guides.
* Color management software and profiles to make things print nicely.
* "Ethical" dodging and burning techniques in Photoshop.
* Wireless commincations, so stories and images can be filed from anywhere.
* CMS that enable reporters to post breaking news items to a newspaper's web site without having to learn HTML.
* PHPbb so the reader community can get together and bitch about stuff.
* Various video and audio codecs for delivery of audiovisual content
* RSS feeds
Not every newspaper gets it right. That's too bad. At my paper, we don't think of blogs as avant garde, but we do think they're necessary.
posted by bugmuncher at 8:59 AM on August 23, 2007 [1 favorite]
Anybody else reminded of the scene in blade runner where he pans & scans a still photograph to look around the corner?
posted by juv3nal at 11:08 AM on August 23, 2007
posted by juv3nal at 11:08 AM on August 23, 2007
Both photography and software coding are arts as well as sciences. Artists should respect each other.
I don't tell you how many times to iterate through your subroutine. Don't tell me you don't like my use of Rule of Thirds...
You're nuts. This is great for art. You can get the rule of thirds to work in situations where it otherwise wouldn't be possible. And the mad magazine foldout scenario rlk mentioned is something that artists should dig.
posted by juv3nal at 11:14 AM on August 23, 2007 [1 favorite]
I don't tell you how many times to iterate through your subroutine. Don't tell me you don't like my use of Rule of Thirds...
You're nuts. This is great for art. You can get the rule of thirds to work in situations where it otherwise wouldn't be possible. And the mad magazine foldout scenario rlk mentioned is something that artists should dig.
posted by juv3nal at 11:14 AM on August 23, 2007 [1 favorite]
I could be wrong but this guy agrees with me:
the Esper machine not only zooms in on Leon's photograph but actually navigates a corner, as if we were actually in the room.
posted by juv3nal at 11:25 AM on August 23, 2007
the Esper machine not only zooms in on Leon's photograph but actually navigates a corner, as if we were actually in the room.
posted by juv3nal at 11:25 AM on August 23, 2007
the bladerunner faq also mentions that the esper machine has "three-dimensional resolution capacity"
posted by juv3nal at 11:27 AM on August 23, 2007
posted by juv3nal at 11:27 AM on August 23, 2007
Nelson: Sure, I'll post it somewhere on my web site sometime in the near future. It's written for MacOS but the core algorithms ought to be portable. The logical next step would be to make a GIMP plugin or something.
cortex: Yeah, my impression of that scene has always been that Deckard is zooming in on a reflection in a window or other shiny surface and that's how he looks around the corner.
posted by hattifattener at 7:24 PM on August 23, 2007 [1 favorite]
cortex: Yeah, my impression of that scene has always been that Deckard is zooming in on a reflection in a window or other shiny surface and that's how he looks around the corner.
posted by hattifattener at 7:24 PM on August 23, 2007 [1 favorite]
You're nuts. This is great for art. You can get the rule of thirds to work in situations where it otherwise wouldn't be possible. And the mad magazine foldout scenario rlk mentioned is something that artists should dig.
It's not that I care whether the "new" photo uses rule of thirds or not; I'm just upset that it would disrespect the artistic vision I had in mind when I made the photograph. Who would think of chopping the arms off of a statue because they're not significant, and after all, so many other statues don't have arms anyway? It's the same thing as deciding which parts of a picture are insignificant.
----------------
On another note, I have to wonder about the legal implications of this technology. I think that that copyright law already protects my work from being reimagined in this way.
Suppose someone took my photo and added the metadata about significance to it, so that it could be resized. Wouldn't the resized photo be considered "a derivative work" based on the addition of data to my original work? If that's the case, as long as I don't add any significance data to my work, anyone thinking of using this technology would need to obtain my permission before creating the derivative work. (Assuming, of course, that I hold the copyright.)
Moreover, if their addition of metadata does not create a "derivative work", I'm still protected because I can choose who I sell the rights to my own copyrighted works to. So if someone uses that technology, I don't have to sell to them.
I could be way off, though. Any IP lawyers in the house?
posted by bugmuncher at 8:19 AM on August 24, 2007
It's not that I care whether the "new" photo uses rule of thirds or not; I'm just upset that it would disrespect the artistic vision I had in mind when I made the photograph. Who would think of chopping the arms off of a statue because they're not significant, and after all, so many other statues don't have arms anyway? It's the same thing as deciding which parts of a picture are insignificant.
----------------
On another note, I have to wonder about the legal implications of this technology. I think that that copyright law already protects my work from being reimagined in this way.
Suppose someone took my photo and added the metadata about significance to it, so that it could be resized. Wouldn't the resized photo be considered "a derivative work" based on the addition of data to my original work? If that's the case, as long as I don't add any significance data to my work, anyone thinking of using this technology would need to obtain my permission before creating the derivative work. (Assuming, of course, that I hold the copyright.)
Moreover, if their addition of metadata does not create a "derivative work", I'm still protected because I can choose who I sell the rights to my own copyrighted works to. So if someone uses that technology, I don't have to sell to them.
I could be way off, though. Any IP lawyers in the house?
posted by bugmuncher at 8:19 AM on August 24, 2007
bugmuncher, if you are asking if this technique provides a manner for people to legally infringe copyrighted photographs, it does not.
posted by grouse at 8:23 AM on August 24, 2007
posted by grouse at 8:23 AM on August 24, 2007
bugmuncher: You have absolutely no legal right to keep people from reinterpreting images you create. If I see a photo in a magazine, and snip it out with scissors, I am breaking no laws, even though I am destroying the magazine designer's layout. If I put a painting in a frame whose color or style upsets the balance of the composition I am breaking no laws. If something is blocking part of a photo when I am looking at it, utterly changing its impact, I am breaking no laws. If I look at your artwork and think about it in a way which you do not approve I am breaking no laws. If I clip apart a print I own and reassemble it in a collage I am breaking no laws. If I look at your artwork while I am colorblind I am breaking no laws. Copyright applies to copying and distribution: it does not control use and it definitely does not create a category of artistic thought-crime.
The only way you can keep people from reimagining your work is to keep your work secret. Keep it under your bed or, better yet, don't create it in the first place; after all your future self might decide to alter your past self's vision!
Get over it already. Once you release something into the world it will inevitably be reinterpreted and reimagined by everyone who comes into contact with it, no matter what your personal feelings on the matter, this is in the essential nature of art. But this steals nothing from you; after all, you still have your original copy and vision of the piece, untouched and unsullied by everyone else's.
posted by hattifattener at 1:22 PM on August 24, 2007 [4 favorites]
The only way you can keep people from reimagining your work is to keep your work secret. Keep it under your bed or, better yet, don't create it in the first place; after all your future self might decide to alter your past self's vision!
Get over it already. Once you release something into the world it will inevitably be reinterpreted and reimagined by everyone who comes into contact with it, no matter what your personal feelings on the matter, this is in the essential nature of art. But this steals nothing from you; after all, you still have your original copy and vision of the piece, untouched and unsullied by everyone else's.
posted by hattifattener at 1:22 PM on August 24, 2007 [4 favorites]
I wonder how long it will take before it's generally accepted that a photograph is no more trustworthy than a spoken or written account.
And how long it will take before a photograph is no more convincing than a spoken or written account.
Maybe this will speed things along a little.
posted by Many bubbles at 3:38 PM on August 24, 2007
And how long it will take before a photograph is no more convincing than a spoken or written account.
Maybe this will speed things along a little.
posted by Many bubbles at 3:38 PM on August 24, 2007
It's not that I care whether the "new" photo uses rule of thirds or not; I'm just upset that it would disrespect the artistic vision I had in mind when I made the photograph.
Pshaw. This technology is about making images fit the media, ie. browser window. You're suggesting it is preferable that, say, cellphone and UMPC users simply not see your image at all instead of seeing modified version that attempts to keep the most salient details intact.
posted by five fresh fish at 6:44 PM on August 24, 2007
Pshaw. This technology is about making images fit the media, ie. browser window. You're suggesting it is preferable that, say, cellphone and UMPC users simply not see your image at all instead of seeing modified version that attempts to keep the most salient details intact.
posted by five fresh fish at 6:44 PM on August 24, 2007
« Older the cupboard, the cupboard. | BioShock, 2nd best game ever? Newer »
This thread has been archived and is closed to new comments
posted by hammurderer at 7:38 PM on August 21, 2007