Our Motion Control Future

Man/Machine Interface is at a crossroads. The paths being taken by the two contenders to for revolution are best summed up in the words of the marketers.

When I interviewed Josh Hutto, the product marketing manager for Kinect, his opening line was this: “Clearly a much better wand wasn’t really going to be the solution for us.”

On the flipside, When I interviewed John Kollar, director of hardware marketing for Sony, the guy behind Move, he was clear that it was all about the hardware:

“The PS3 is an HD system. As more consumers adopt HDTV, the PS3 is becoming their system of choice -- to have blue ray, to have HD gaming and now to have PlayStation Move.”

Is it possible to truly handicap the future? And if not, what’s a lowly consumer to do?

I’ve been fascinated by both Move and Kinect since they were announced. At press events, in public locations, and at conventions I took every opportunity to experience them as they developed.

Those early experiences were, at best, a mixed bag. Here’s my summation of pre-release experiences with both platforms.

Kinect: Bad lag, non-intuitive interfaces, seemingly random recognition of body location, and a limited, thin selection of "experiences".

Move: Calibrate every 10 seconds. Then go play some Wii games that are just a bit prettier, except for the random missed inputs and crazy unintended gestures.

Hardly rave preview material. But with little else to go on, I diligently filed pre-release holiday-buying-guide copy to a mainstream publication, loaded with “beta” caveats which will likely get removed in the final edit, long after I’ve cashed the check.

Fast forward to this weekend.

Both the Move and Kinect have found a way to live in my TV cabinet, wedged on either side of the Wii sensor bar.

Here was my setup experience for both.

Kinect: I plugged it in. I walked through 2 minutes of “stand here, be quiet, hold this postcard” instructions. I put Dance Central in the tray and walked away. My 10 year old daughter figured out the rest.

Move: I synced the Move controller and handed it to my 6 year old son. He pointed it at the PlayStation Eye camera (once I scraped the dust off it that's accumulated since the Eye of Judgment launch), and held down the big button in the middle.

Since those elaborate and exhaustive installation nightmares, I’ve barely had a chance to wedge my way into the living room. Both technologies simply work. But what’s interesting isn’t so much that they work, it’s that they work so much better than they did in press briefings or at PAX or on the floor of Macy’s or in the Microsoft store.

It’s as if sometime in the last 8 weeks a benevolent deity descended into both Sony’s and Microsoft’s houses and approved a billion hours of overtime for “tweaks,” where, in this case, “tweak” means “go from laughably inconsistent to incredibly refined in a month.”

My children haven’t sat still in two days. Peter, my youngest, has been playing on the floor with EyePet for the PlayStation Move to the point where my actual, real pet dog is jealous. My daughter has been working her way up the leader boards in Dance Central for Kinect, where she's majoring in Lady Gaga. I’m correlating each minute of Lady Gaga to one dollar of future therapy bills, but she’s a sweaty, giggly mess, so I let it go.

I’m not an idiot, all evidence and opinion to the contrary. Like everyone else, I read reviews and blog posts from people I genuinely trust. I see that my family’s experiences with both platforms aren’t the norm. I attribute this to having a very large, well lit living room with a white floor and movable furniture. You know all those Move and Kinect Ads? That’s what my living room looks like. My house is the ideal environment for any system that relies on machine vision.

In that near-laboratory setup, both technologies feel so much like science fiction, it's a bit unnerving.

When the dog runs in front of the EyePet on my dirty white carpet, the EyePet gets out of the way.

When I stand in front of the Kinect sensor, it aligns and centers on me perfectly, the rag-doll wireframe perfectly matching my hand motions, so that with a 3 inch flick of my wrist I can navigate menus.

And yet, I read the horror stories. The people who cannot get themselves to appear on Microsoft’s big brother eyeball because of poor lighting or bad glasses.

All this screams “wait” to me. But it doesn't scream "ignore this, it's VirtualBoy."

In their own ways, both Move and Kinect are genuinely the future of man-machine interactions. Kinect games in the right environment are like nothing I’ve ever played before, and the experience of navigating through a well designed program (meaning, the one Kinect title made by Harmonix) is like living on the deck of a starship.

The Move, on the other hand, delivers a precision that is, particularly when paired with augmented reality systems like the EyePet, simply magical.

Is all of this perfectly realized today? Nope.

But is it, in my miraculous living room conditions, something you’ve never seen before? Yep.

This time next year, I predict that Kinect will have sold like wildfire, because it delivers that shock of the new almost instantly. Move will likely be floundering, mired in poor marketing and an unclear approach. And yet both titles will be successful in their own way. That my children each latched on to a different platform is telling. My son, more a genetically predisposed gamer comfortable with a controller in his hand, found the Move’s ice-cream cone controller to be a magic wand extension of himself. There’s little doubt that when someone makes a game that lets him use it as one (perhaps in the upcoming game, Sorcery, he will blow a vein in excitement.

My daughter, a much more physical and expressive person by nature, has pronounced the Kinect is the best videogame-thing ever. The simple act of waking up the Xbox by talking to it or waving at it makes her giggle. I imagine it’s just a matter of time before someone comes along with a game which lets her rub her hands together to make a fireball out of thin air and hurl it at bad guys on the screen, and her fantasy life will be complete.

These kinds of interactions are the future. And for many gamers, or I should say Gamers, that’s probably when you’ll want to buy it. Right now, they’re beta tests for the George-Jetson-possible. For some, it may be a “Jane, stop this crazy thing” experience.

But for me, it’s a ride I like being on.

Comments

Nice expansion on what you said in the podcast. One thing I wonder is why neither Sony nor Microsoft decided to use a wider angle for their cameras, which I think would have allowed people with smaller living spaces to use the systems.

I'm a little surprised to see Kinect come out of the gate so strong. I wonder if there's any way to firmware update the existing hardware to make it more friendly to different types of living spaces. There's no doubt that many of us have far from ideal conditions for Kinect. Even if I could magically teleport my couch out of the living room (sometimes I wonder how I got it in there), 6 feet would have my back against the wall. Well, I do live in Japan, but my living room is the same as 98% of living rooms here (forget about those who don't even have what we call a living room), not boding well for Kinect in Japan.

Regardless, I'm pretty enthusiastic after hearing about people's experiences so far so I hope that MS can somehow update Kinect to get it working better in the 4-6 feet range. Or maybe I need to wait until Kinect 2?

Think about how wide angle a camera would need to be to get your feet and as high as you can reach, at 4 feet away.

The cameras are actually pretty wide angle. Try standing 4 feet away from someone with a camera and get a decent picture of their toes while aimed at their shoulders.

Yeah, too true. Hopefully, there will be another version of this technology that can adapt to smaller spaces. Maybe multiple cameras?

the cameras don't have to match the ratios of a photograph or tv screen however, and in the case of the Move it doesn't need to capture the full body. If they can make it wide angle horizontally, there should be little difficulty designing it to be equally wide vertically. Most likely culprit is cost--using something like the equivalent of a 24mm lens and an unorthodox image sensor would not be cheap. A multiple camera setup would probably be the best way to go.

On another note, I really hate that the PS3 does not have rear usb ports. It's butt ugly having the cable for the camera run in front of the system.

Favorite game so far is ping pong. I like how not only does the opponent get better going from bronze to silver, but accuracy is also increased and the game helps you less by removing the ball path guidance. When I first tried silver I found all backhands suddenly going into the net, which didn't happen ever on bronze. I realized I was rotating my wrist, and the Move controller was picking that up. I was pissed and impressed at the same time.

Great article. The extended version of the podcast, as Mao said.

When you mentioned in the podcast that navigating the menu in the Harmonix-made game was like being in Minority Report, I was sold. And I was double sold when you compared it to the first time you picked up an iPhone and said "whoa" (I had the same experience with my Android phone...it's like I'm living in the future). So the Kinect might be the one thing that pushes me over the edge to buy an XBox, however I might wait a year or so for it to work out the kinks and figure out how to work in smaller, darker spaces--like GT_Chris, I also live in Japan so my livingroom space is less than ideal, and, unlike Rabbit, my livingroom is not laboratory-white.

It really does sound like the setup in your house is a crucial factor. My "entertainment centre" is in a partially finished basement with naturally dimmer lighting. We also live in a small bungalow so I'm unsure if I even have enough room available. I think there might be 10 feet available between the TV and the wall, but that's if move the coffee table and the couch fully out of the way. I'm not sure where to though, because there's a second shorter couch turned perpendicular on the left, and my massive Roland TD12 drum kit (which I also use for Rock Band, natch) on the right.

This does reinforce the one consistent thing I've heard, which is that Dance Central is the best thing yet made for Kinect. Nicely done, Harmonix. Between this and Rock Band 3, you've got my wallet attached to a string you can yank whenever you please.

This article was helpful to me because I think it better reflects something closer to typical use when compared to the impressions of many video game journalists. Specifically, it seems to me that the living conditions of most video game journalists do not really correspond to the living conditions of the target demo for the Kinect. Many, if not most, video game website writers seem to be between 18-32, nonmarried, and live in small apartments in one of a handful of our most densely populated cities. (San Francisco/New York)

In contrast, the target demographic of the device seems to be soccer moms or gamer dads between 30-50 and their children. I would be willing to bet that a much smaller percentage of this group lives in small apartments. This demo generally has left the city core in search of more space for less money. Generally they either rent a larger kid friendly apartment or purchase/rent a house in the suburbs. Would it have been nice if MS could have adapted the camera somehow to work in a smaller space? Certainly. However, I don't think the number of customers they would pick up by doing so would outweigh the number they would lose because of the increased cost/price necessary.

To be sure this is more of a North American phenomenon and could spell trouble for MS in more densely populated countries with traditionally smaller living spaces like those an earlier poster described in Japan. However, considering their already nonexistent market share in Japan I don't think it matters much.

A couple of questions for anyone that has actually read this too long post:

1. Anyone with young kids (mine are 3 and 5 years old) have any insight on which is holding their attention more: Kinectimals or Eyepet?

2. Anyone have any comments about the exercise games compared to the Wii offerings? My wife used Wii Fit, but quickly switched to EA Active on the Wii to avoid pulling out the balance board all the time.

Chairman_Mao wrote:

Nice expansion on what you said in the podcast. One thing I wonder is why neither Sony nor Microsoft decided to use a wider angle for their cameras, which I think would have allowed people with smaller living spaces to use the systems.

I'm assuming there's a trade-off to be had here. With a wider angle lens you'd still need to capture the same pixels-per-area as the current solution, if you wanted to have the same quality of motion detection you see now. If you kept the same sensor size with a wider area, you'd lose some degree of fine tracking.

To avoid losing precision, presumably you'd need a better lens and/or a better sensor (depending on where the limitation is, currently). That increases the cost of the device, and the increased data flow will increase the processing overhead. I'm guessing they looked at all these factors and decided it just flat out wasn't worth it at release (I remember the original specs had dedicated processing in the device itself, which has since been scrapped).

This is something that can always come later. Presumably you can drop in a new/better Kinect, possibly with data processing built in, as an upgrade when the hardware reaches the price point that makes it work.

DorkmasterFlek wrote:

It really does sound like the setup in your house is a crucial factor. My "entertainment centre" is in a partially finished basement with naturally dimmer lighting. We also live in a small bungalow so I'm unsure if I even have enough room available. I think there might be 10 feet available between the TV and the wall, but that's if move the coffee table and the couch fully out of the way.

I don't think you have to get too close to the big huge living space we see in the commercials. I live in a 560 sq foot condo, managed to squeeze out about 8 feet from the sensor to the front of the couch with the coffee table slid out of the way, and with only one not very bright light on, have had no problems playing with Kinect. It is a bit of a different shaped room from normal, but it definitely works with a small amount of space and lower lighting conditions.

The biggest question is how long until the folks on the floor below come banging on my door =)

My wife is very keen on trying out the Kinect - and I have been surprised by how effective the marketing has been for her (against her?). I think you were spot on about the desire between gamers and Gamers to try out Kinect, as I have wary interest in the system (Gamer) and my peggle and PvZ playing wife (gamer) is super pumped to try out Kinectimals and Dance Central. Luckily, I am like you Rabbit, and my living room is large, mostly white, and open.

We are keeping our eyes out for holiday sales.

For those not aware, the Kinect doesn't use visible light for tracking, the RGB camera is solely for blackmail videos and photos. The actual body tracking is done with reflected IR. This can be thrown off with direct sunlight, but it doesn't require any special lighting in your room.

On another note, I really hate that the PS3 does not have rear usb ports. It's butt ugly having the cable for the camera run in front of the system.

yes, same issue here. How comes that no marketing material shows the USB cable? hm...

Hey! A well reasoned article to talk me out of getting the Kinect! Just what I need.

[Dissolve to five minutes later.]

Aww shoot.

I think eyepet is more engaging as a "toy," but my sense is kinectimals is more of a game. THe eyepet does some really astounding augmented reality stuff you've just never seen anywhere before. When you draw a picture with a pen, then hold it up to the screen, then have the picture turn into what you drew (a car with triangular wheels for instance), and then drive it around your living room ... it's really unbelievable.

As for fitness: I have your shape, wii fit, and EA Active. Your shape has a really aweful look to it (very sterile) but so far my experience is that the actual workouts are more real than either of the other two titles. I've been doing an actual mens fitness weightlifting routine which would be familiar to anyone who'se followed Mens Fitness workouts from the magazine (or P90X for that matter). I haven't done a TON with it yet, so I haven't explored every option, but so far it feels far more real.

One nice thing is that it actually corrects you pretty well. If you're doing rows, for instance, and you curl your back, it yells at you. If you're dogging your lunges, it tells you to deepen them. If you're holding your arms too wide in an overhead press, it tells you to bring your elbows in. Because it acts as both a mirror (you see your uncanny avatar without any lag), and a trainer (you also see the dude your supposed to follow) it's much more like being in a gym in front of a mirror with a weightstack.

Thanks for the additional information Rabbit.

LiquidMantis wrote:

For those not aware, the Kinect doesn't use visible light for tracking, the RGB camera is solely for blackmail videos and photos. The actual body tracking is done with reflected IR. This can be thrown off with direct sunlight, but it doesn't require any special lighting in your room.

Are you sure it's not using a combination of IR sensor and RGB camera? I can't see them including an RGB camera "for fun". I think it uses both, which is definitely going to improve reliability.

IR Sensor (with pattern) can read depth, which one RGB camera cannot. Then the RGB camera helps triangulate the image.

PyromanFO wrote:

Are you sure it's not using a combination of IR sensor and RGB camera? I can't see them including an RGB camera "for fun".

Why not? It's all but free and it lets them add the social element plus market the video chat.

Here's an article on it working in the dark. I know it works great for me in my gaming room with just light from the TV and the small corner lamp I use for background lighting.

Yeah, there has been a clear division in opinion between journalists who have obviously been able to play Kinect in the office or who actually have a suitable living space for it like Rabbit, versus those like the Joystiq bloggers trying to make it work in their tiny apartments and then bitching like crazy when it doesn't. Funnily enough, via 1UP I was reading the Famitsu (Japanese magazine) impressions of the first batch of Kinect games and they were overwhelmingly positive. Obviously they managed to do their play testing in their office :).

This sounds truly amazing.

I have never seen reaction of technology junkies go from so low to so high so quickly. I've seen reactions move from high to low, but this is truly a first. If I didn't trust this site so much I might call shenanigans.

I've gone from "maybe getting this in the future" to planning on getting it. I think I can justify waiting until my basement is finished. The wife wants a Macbook air (ok, I do as well), so this might have to get in line behind that. Besides, I'm interested in what wave 2 of games are. Will these systems get the development support that has eluded the Wii? Or are we going to get a never ending parade of mini-games and half-hearted efforts?

I haven't tried either yet, but even if I had a 360, access to an HDTV, and a large, well-lit area to play in, I don't really see myself getting either peripheral with the current game selection. The one game on either system with peripheral support that looks appealing to me is Heavy Rain, and seeing as I've beat it several times already (note: do not try this at home) it probably wouldn't really be worth it to play again but with the Move controller replacing the right analog stick.

LiquidMantis wrote:
PyromanFO wrote:

Are you sure it's not using a combination of IR sensor and RGB camera? I can't see them including an RGB camera "for fun".

Why not? It's all but free and it lets them add the social element plus market the video chat.

Here's an article on it working in the dark. I know it works great for me in my gaming room with just light from the TV and the small corner lamp I use for background lighting.

There is no such thing as "almost free" in hardware. Also, why would they want a "well lit room" if the camera was useless? It's great your setup is working but alot of people are having problems with poorly lit areas.

PyromanFO wrote:

There is no such thing as "almost free" in hardware.

Psst. Hey kid. Over here.

Whether or not lighting matters seems to depend heavily on implementation. The core body-tracking stuff is definitely based on the IR camera, and thus ambient light doesn't matter much at all. The facial recognition stuff seems almost useless in the dark, however, and programs that present a "skin" on your form, like YourShape, look unbelievably wierd in low light, as they present a monotone skin with no colormap.

Further, any titles that will do any kind of augmented reality (don't know of any on 360) obviously need a camera. Last but not least, it replaces the old USB cam for chatting stuff.