The new junaio channel works in conjunction with a popular German TV show called ‘Galileo’. Viewers watching the show are able to interact with the program by using the junaio client and by pointing their smartphone at the TV screen.

During the Galileo program questions are displayed on the screen. Unlike most interactive programs that require you to SMS an expensive number and wait several hours for the result, when the junaio enabled program asks the viewer a question the viewer simply points their smartphone at the screen. The junaio channel which uses natural feature tracking recognises the question; it is then transferred to the viewer’s smartphone. Once on the smartphone the viewer can select the answer without having to send an expensive text. In addition real-time feedback can be provided so it’s great for interactive TV quiz shows. For example, play along to who wants to be a millionaire at home and get instant feedback, even compete with other players watching the same show.

As Peter Meier, CTO of metaio points out, it’s not just about quizzes. The same technology could be used to enable viewers to receive recipes as the chief is making a dish. It’s definitely an interesting idea to enable views to interact with the TV. Take a look at the video of the solution in action. (Video Link)

What TV program would you like to see made interactive this way?

A method developed in Aalto University retrieves information to data glasses about people and objects a person is looking at. Equipment including an eye tracker implements machine learning methods and infers from context and gaze direction what the viewer may be interested in, to learn more about. Following that the machine displays additional information on viewer’s data glasses. For instance, a researcher may learn about scientific results of a researcher passing by and singles may see on their data glasses the relationship status of a person they meet. By following the eye-movement the machine also infers when additional information should be displayed.

In the test versions of the augmented reality (AR) method displayed information is retrieved from a database with information keyed in beforehand. However, the method can also be set to retrieve information from internet and social media. In this case searches – using Google among others -- can be made by looking through the data glasses only.

The equipment recognizes persons looked at. It requires that images of the persons to be recognized have been inserted into database earlier. At the moment, it is not possible to identify totally unknown persons.

Our basic research on machine learning methods has made it possible to create new advanced user interfaces in which the machine infers without direct human commands what the viewer is interested in, tells professor Samuel Kaski from Aalto University.

As a result of our research we have created a prototype that has been successfully tested in laboratories. However, there is still a way to go before this can be commercialized, reminds researcher Jorma Laaksonen from Aalto University.

Study about the new machine learning methods is to be published in the Virtual Reality journal. The research results have been reached with funding from private companies and organizations to Aalto University’s MIDE research programme.

Information about the research consortium:
http://mide.tkk.fi/en/UI-ART

Preprint version of the article to be published in Virtual Reality:
http://research.ics.tkk.fi/mi/papers/vire10.pdf

System in action

www.youtube.com/watch?v=gtuGSWDVdQU

Multi Search Topics

There are demos a plenty of taking photos of book covers to retrieve reviews and online prices. If you are new to augmented reality then it’s actually not that new. SnapTell have both an iPhone and Android application that provides this functionality. Just take a picture of the cover of a DVD, CD, Book, or Video Game and the software uploads the image to the SnapTell server and runs image recognition to compare the picture against likely matches. Once a match is found you get the usual mix of online reviews and cheaper prices.

Recognising a barcode image I would assume is a lot less complex than recognising a book cover since there are less colours and shapes to contend with. Barcode shopping is another feature of Google Goggles and again there are various products available that provide the same functionality, ShopSavvy on the Android or RedLaser on the iPhone are just a few examples. I use RedLaser from time to time to see just how obscure the product range is. GoCat Tuna and Herring cat crunchies is the most obscure product I have managed to have recognised so far. To compete with Goggle’s logo recognition there’s GetFugu for the Android, iPhone and the Blackberry so again nothing new there. Goggles will search on wine bottle logos which as far I as know is unique as I don’t know any other application that does that today, but I’m sure it’s only a small leap from recognising a book cover to recognising a wine label.

You might think that I’m having a dig at Google but far from it, just becuase they are not first does not make the application any less cool. There are some really fantastic features in Google Goggles that make me drool. I’m no art buff but Goggles will enable me to become an art critic and give me relevant information about major pieces of art, all I need to do is sneak a crafty picture when the security guard isn’t looking and Goggles will tell me the name of the picture and the artist. Fantastic if you are walking around a museum but again it’s probably not a huge step in recognising the Mona Lisa or a paperback copy of C++ for dummies, ultimately it’s a blob of shapes and colours that are compared. Nether the less it is a fantastic use of image recognition.

My personal favourite and the piece de resistance of Google Goggles is a feature that must have the CEOs of the various augmented reality browser companies laying awake at night as Google are about to change the way location aware augmented reality browsers work.

The way augmented reality browsers work today is they use LBS to figure out your current location and the compass to see which way you are facing. With that information they can determine what points of interest you might be facing. Most of the time it works pretty well but I’m sure you know that GPS is not accurate, sometimes my tests have been accurate to 30 meters and sometimes it strays to up to a mile. If you are facing a landmark the visual indicator provided by the augmented reality browser may or may not be shown near by the object you are looking to locate. The second problem is there could be a building between you and the point of interest; the application has no way of actually knowing what is in front of you and what you are really looking at.

Google Goggles however uses image recognition on the landmark. If you are standing at Big Ben in London it sends a copy of the image to the database, performs an image match and then tells you precisely what you are looking at, not what might be around you and in that general direction. The accuracy is so good you can sit at home in California and look at your holiday snaps of Big Ben and it will still recognise the landmark. Fantastic when you get back from a trip and wonder just what all those photo’s are actually of.

Google’s strength is obviously search and the vast array of seemingly endless images that have that their disposal. Properly indexed there would seem to be no limit to what can recognise, particularly if they open up the API for developers to produce our own plug-ins. What breed of cat is that? Cat recognition plug-in tells me is a British Short hair and it’s favourite food is fish. That’s a nice car what is it? Car recognition plug-in tells me it’s an Aston Martin DB7, insurance group is a 31 and it has a top speed of 186mph, but I can’t afford it because my bank plug-in is warning me based on my current salary I would have to save for 30 years.

You can’t help think that terminator vision has just got a step closer. Some six or seven months ago we got our first augmented reality browser, now at the end of the year it seems like we are about to take a huge step forward in augmented reality with Google’s arrival. It will be interesting to see how the current crop of augmented reality browser providers respond to the Google play and what is up their sleeve for the future..

The video walk-through below well worth a look at some of the features of Google Goggles.

Excellent overview of Google Goggles.

www.youtube.com/watch?v=WA7wwKIC24s

Parallel Kingdom is a massively multi-player online role playing game that takes place around you. The game uses GPS to figure out your location and places the character directly in your neighbourhood where you can explore, declare war on other players or just fight the many creatures that live there. It’s interesting to see your surroundings plotted with monsters, trees, caves and other exotic items. I had never noticed that there was a castle at the end of my street before.

I have only just installed the game and it seems to be pretty complex and loaded with options so it is going to take a while for me to figure out what the hell I am doing, but like all MMORPG’s the object is to kill, collect gold and build up your level.

Given it’s complexity it’s hard to give a full review but I guess the question is, is it really an augmented reality application? Well that depends on your point of view.

If you are a traditionalist who believes that augmented reality is defined by using the camera and blending computer graphics in the camera feed then you probably won’t see Parallel Kingdom as a true augmented reality game. If however you take the wider view and believe that augmented reality is being aware of you surroundings and enhancing them then you’ll see Parallel Kingdom as the first true augmented reality MMORPG. Either way, Parallel Kingdom is free so worth an install and making up your own mind.

Game Trailer

www.youtube.com/watch?v=3WGM37bWQzs

Chetan also talks about the choices in selecting a platform and his plans for porting applications over to Android. It’s an interesting talk, the Q&A at the end has lots of questions on where the data comes from and how to manage the presentation of lots of data on the screen. So if you are planning on building an augmented reality application and want some tips from a company that have 10 augmented reality apps already in the iPhone store,  or if you are just generally interested in learning more about the technology then its well worth a listen.

Watch the webcast

www.youtube.com/watch?v=AsTFPoK0Gbk