Natural (Language) Interfaces

This blog post is not about Siri, sorry.

I remember when the best way to control a games console was like this:

But over the years, controllers started to look like this:

From a single red button to a plethora of buttons, triggers, D-pads, joysticks, joysticks which act as buttons and switches, it’s no wonder that there was a bit of a “revolution” when this hit the market:

But everyone has been a little fascinated with this for the last couple of years. And not surprisingly – this is one of the interfaces that we use to control the world. It seems natural to use it for direct manipulation.

And despite the fact that the hardware is obviously capable of it, games designers haven’t been making use of one of the other obvious interfaces. One that we humans excel at.

This isn’t the same as using a headset to bark commands at team-members, but using defined commands to instruct a game element. Yes, these games exist (Shouter being one of the most well-known) but the sophistication is low.

What I’m looking for is the difference between Newton and Palm, but in terms of voice. Newton tried to recognise your handwriting while Palm made you learn a certain alphabet. For games, at this stage, we need to create a basic control set that can be easily recognised by a language processor. Whether that is in understanding actual words or whether it is mapping wave patterns – it doesn’t matter. The point is to use our voice to control games.

The instructions can be short, they can be words, they can be screams and cries. When I call “Retreat”, my units should start to retreat back to base, making a tactical withdrawal. When I order “Advance”, they should use cover and opportunity to advance upon the enemy position. And when I shout “Charge”, you get the idea.

(images not used with permission)