Augmented reality integrated in children’s books to facilitate interactive learning

• Using augmented reality (AR), the technology behind the interactive version of Giant Jimmy Jones, New Zealand author Gavin Bishop recently collaborated with Mark Billinghurst and his colleagues at the Human Interface Technology Laboratory New Zealand (HIT Lab NZ) to turn the book into not only a storytelling device, but also a storytelling experience.
• A child can flip through its pages and read it like a conventional book.
• But with a handheld display and computer vision tracking technology, the child can watch the story literally come to life.
• "You can see animated virtual characters overlaid on the real book pages and hear the voice of Gavin Bishop reading the story," says Billinghurst, director of the HIT Lab NZ..
• While Giant Jimmy Jones currently only exists in a lab setting, there are scores of others being developed at places such as Georgia Tech University's Augmented Environments Laboratory.
• Technology is not the hindrance to turning books into interactive devices whose readers can exist within them and manipulate their stories.
• The most difficult roadblock stems from the limitations of physical books, most notably the reality that embedding markers that can interact with VR-headgear is expensive and produces ugly visual results on the page.
• A wearable computer in the form of a head-mounted display (HMD) is worn like a fighter-pilot helmet, fitting over the head and eyes, and projecting images within the user's visual field.
• Improvements have led to devices that resemble glasses, but even these are unwieldy, unsanitary and limited to one user at a time.
• The HMDs also suffer tracking problems, or perfect registration, which means that the virtual overlay doesn't quite match up with the physical space upon which it is projected.