Head Tracking for iPad: Glasses-Free 3D Display
April 11, 2011 4:22 PM Subscribe
Head Tracking for iPad: Glasses-Free 3D Display - Jeremie Francone and Laurence Nigay of the Grenoble Informatics Laboratory track the user's head using an iPad's front facing camera, using the positional data to create the impression of depth without the use of specialized glasses
"The demo with the targets has been inspired by the work of Johnny Lee with a Wii remote."
"The demo with the targets has been inspired by the work of Johnny Lee with a Wii remote."
This is still just a 2D projection of a 3D environment, right? You can't have a 3D display without sending different images to each eye. Still a neat demo.
posted by knave at 4:32 PM on April 11, 2011
posted by knave at 4:32 PM on April 11, 2011
Good music choice. Really drives home how lonely you'll be soon if you don't get this immediately.
posted by hermitosis at 4:35 PM on April 11, 2011 [3 favorites]
posted by hermitosis at 4:35 PM on April 11, 2011 [3 favorites]
Knave, yeah. But I actually think that sort of technique is a lot more realistic? Easier to work with?
What might make this a non starter is how do you deal with touch input when—unless all the touch surfaces are depicted as being flush with the screen—you'll receive two different types of input?
i.e. you can tap on those icons on the sidewalls, but it's going to tell your brain that it's just a flat sheet of glass.
posted by Brainy at 4:36 PM on April 11, 2011
What might make this a non starter is how do you deal with touch input when—unless all the touch surfaces are depicted as being flush with the screen—you'll receive two different types of input?
i.e. you can tap on those icons on the sidewalls, but it's going to tell your brain that it's just a flat sheet of glass.
posted by Brainy at 4:36 PM on April 11, 2011
This is still just a 2D projection of a 3D environment, right?
Yeah, this is an implementation of that Johnny Lee Wiimote hack that spread like a brush fire awhile back and made his career. Only this time it's done just with a low-res camera and, I'm guessing, facial recognition?
Really awesome. I hope this gets app store approval, even just as a toy program.
posted by middleclasstool at 4:37 PM on April 11, 2011 [1 favorite]
Yeah, this is an implementation of that Johnny Lee Wiimote hack that spread like a brush fire awhile back and made his career. Only this time it's done just with a low-res camera and, I'm guessing, facial recognition?
Really awesome. I hope this gets app store approval, even just as a toy program.
posted by middleclasstool at 4:37 PM on April 11, 2011 [1 favorite]
I'm particularly impressed that the demo came across so well on YouTube. I expected to watch it and be all, 'yeah, i can sorta see it,' but here I am all, 'yowza, floating bullseyes!' Very cool. Thanks, BP.
posted by JohnFredra at 4:41 PM on April 11, 2011
posted by JohnFredra at 4:41 PM on April 11, 2011
See also FreeTrack (IR-based, free, Windows, requires headmount hardware) and trackIR (IR-based, not free, Windows, required headmount hardware) and Headmouse (camera-based, not free, requires HW, crossplat) and Camera Mouse (camera-based, free, windows, req camera).
I strongly expect variations on camera-based assistive inputs to become in-OS competitive features Real Soon Now. I use TrackIR in Rise of Flight and my critical analysis includes phrases such as "WHOOOOOOEEEE" and "HELL YES!!!!"
I most especially commend the FreeTrack homebrew setups to your kind attention. Gear hackers wearing funny headgear. My people!
That said, I note that this thread originates concerning an iPad app featuring similar features. I am pleased to see it on this platform, and look forward to using it in XPlane iPad.
posted by mwhybark at 4:41 PM on April 11, 2011 [1 favorite]
I strongly expect variations on camera-based assistive inputs to become in-OS competitive features Real Soon Now. I use TrackIR in Rise of Flight and my critical analysis includes phrases such as "WHOOOOOOEEEE" and "HELL YES!!!!"
I most especially commend the FreeTrack homebrew setups to your kind attention. Gear hackers wearing funny headgear. My people!
That said, I note that this thread originates concerning an iPad app featuring similar features. I am pleased to see it on this platform, and look forward to using it in XPlane iPad.
posted by mwhybark at 4:41 PM on April 11, 2011 [1 favorite]
That's really cool. Makes me think that the only way 3D will ever really work is if designers think of the viewing device like a hollow box where no objects can ever come past the screen, otherwise your image gets cut off at the edge and it ruins immersion. I'm also having a hard time imagining how you interact with anything... maybe software that can also recognize relative position of your hands, or maybe IR rings where the sensor can read where they are in 3D space?
posted by codacorolla at 5:03 PM on April 11, 2011
posted by codacorolla at 5:03 PM on April 11, 2011
I hope this gets app store approval
Ha, app store approval. I bet Apple will be giving this guy money / a job in the very near future.
posted by naju at 5:31 PM on April 11, 2011
Ha, app store approval. I bet Apple will be giving this guy money / a job in the very near future.
posted by naju at 5:31 PM on April 11, 2011
You can't have a 3D display without sending different images to each eye.
I'm no expert but it seems to me that stereoscopy isn't 3D either; that this technique of simulating depth is another solution with different constraints. And very cool. I await new implementations leveraging eye-tracking on camera-equipped devices.
posted by Jode at 5:38 PM on April 11, 2011
I'm no expert but it seems to me that stereoscopy isn't 3D either; that this technique of simulating depth is another solution with different constraints. And very cool. I await new implementations leveraging eye-tracking on camera-equipped devices.
posted by Jode at 5:38 PM on April 11, 2011
This kind of thing was done on the Nintendo DSi (not the 3DS) about a year ago. The larger iPad screen might "sell" the illusion better though.
posted by L.P. Hatecraft at 6:52 PM on April 11, 2011 [1 favorite]
posted by L.P. Hatecraft at 6:52 PM on April 11, 2011 [1 favorite]
What this might be really useful for is through-the-screen augmented reality. You could track the user's position with the rear camera and use it to transform the image from the front camera so that it matches what you can see around the edges, turning the device into a window. You apparently get 45 degrees horizontal field of view on the iPad 2, which should be more than enough to make it work.
posted by topynate at 7:28 PM on April 11, 2011
posted by topynate at 7:28 PM on April 11, 2011
I always thought the first alternative accessible interface to the iPad would be voice commands, because of the built-in but sadly under-utilized microphone. Head tracking would be a boon to many disabled people as well. Hope to see an app for that.
posted by Soliloquy at 8:28 PM on April 11, 2011
posted by Soliloquy at 8:28 PM on April 11, 2011
I've heard that one of the problems with the way the brain processes data in 3D movies is that they succeed in providing a discrete image to each eye, but fail in certain other areas. I wonder if this is one of those; a very much 3D effect that basis itself on expectations of perspective based on the movement of objects with respect to the point of observation and not actual depth of field.
Either way; Neat!
posted by quin at 9:25 PM on April 11, 2011
Either way; Neat!
posted by quin at 9:25 PM on April 11, 2011
Woah.. "3D" that actually works for me! (I'm stereoblind, stereoscopic 3D doesn't work at all for me..)
posted by vivelame at 12:55 AM on April 12, 2011 [1 favorite]
posted by vivelame at 12:55 AM on April 12, 2011 [1 favorite]
See also Face Mouse (free, Windows only, tracks your head movement for mouse control. Intended for people who can't use a conventional mouse - use it with an on-screen keyboard. Disclaimer: produced by my company.)
posted by alasdair at 2:17 AM on April 12, 2011
posted by alasdair at 2:17 AM on April 12, 2011
If you only have good vision in one eye (like me) this is then only type of 3d that works.
Peter Falk did not see Avatar in the theater and neither did vivelame or I. Nor did Fritz Lang, Big Bill Haywood, Moshe Dayan or Sammy Davis Jr, but probably for different reasons.
posted by warbaby at 6:43 AM on April 12, 2011
Peter Falk did not see Avatar in the theater and neither did vivelame or I. Nor did Fritz Lang, Big Bill Haywood, Moshe Dayan or Sammy Davis Jr, but probably for different reasons.
posted by warbaby at 6:43 AM on April 12, 2011
Very clever! And, not being actual 3D, the experience translated very well on the demo video too.
posted by GuyZero at 9:29 AM on April 12, 2011
posted by GuyZero at 9:29 AM on April 12, 2011
Separate image 3d gives me, and everybody else, headaches. This type of 3d doesn't. More of this, please.
posted by Threeway Handshake at 1:30 PM on April 12, 2011
posted by Threeway Handshake at 1:30 PM on April 12, 2011
Got an iPad? Go to the App store and download the free app "idesktopvr"
You're welcome.
posted by dbiedny at 4:54 PM on April 12, 2011 [1 favorite]
You're welcome.
posted by dbiedny at 4:54 PM on April 12, 2011 [1 favorite]
You're welcome.
Well, thank you, but the first comment on the app's youtube video advertisement (here, since you decided not to link it) says how it just simulates positional 3D by using the tilt sensor.
posted by codacorolla at 5:17 PM on April 12, 2011
Well, thank you, but the first comment on the app's youtube video advertisement (here, since you decided not to link it) says how it just simulates positional 3D by using the tilt sensor.
posted by codacorolla at 5:17 PM on April 12, 2011
A bunch of games have done the tilt-sensor 3D thing; there's a labyrinth I can't remember the name of, and the (pretty decent) Pinball HD. It's kind of seasick-inducing. The head tracking looks much more impressive.
posted by ook at 7:45 AM on April 13, 2011
posted by ook at 7:45 AM on April 13, 2011
« Older Finger painting | Free Darko goes dark. Newer »
This thread has been archived and is closed to new comments
posted by Brainy at 4:27 PM on April 11, 2011