HoloCom – An Augmented Reality Showcase for Microsoft HoloLens

March 2, 2018

The hardware hacking team of the TNG Technology Consulting GmbH just went to another – so called – Winter Retreat into the alps to work on a new prototype. This time we had access to a Microsoft HoloLens.

What if Augmented Reality glasses allowed you to communicate with friends far away as if they were in the same room? The hardware hacking team of TNG Technology Consulting has implemented a software prototype enabling such hologpraphic telepresence. Using this software it is possible to project the 3D shape of another person into your own field of view. This resembles the holograms in the “Star Wars” movies.

The Augmented Reality concept already exists since the late 1960s. Microsoft Hololens is one of the first autonomous devices that can enrich the real world with computer generated elements and so-called holograms. Within this talk we will show and explain some ideas and concepts from the field of Mixed Reality. We will present some technical details of the device and demonstrate them within live showcases. Furthermore, we will show some simple programming examples for the Hololens using Unity 3D.

For this project a very new conference talk with live demo and live coding is available.

Enjoy!

Advertisements

Pecha Kucha video of “See like a Terminator”

March 4, 2016

On the 4th of February we held a keynote on the topic “See like a Terminator – Augmented Reality with Oculus Rift” at the OOP conference in the International Congress Center in Munich in German language.

In the evening Martin Heider (one of the organizers of the OOP conference) staged a Pechu Kucha night where many people held a talk with 20 slides using 20 seconds for each slide. Here you can watch our Pecha Kucha about a self-built Augmented Reality wearable device based on Intel RealSense and Oculus Rift with which you can see the world through the eyes of a Terminator in English language. Of course this is a very, very short version of our original talk which has been held in many different versions already.

The talk in its original length (60 minutes) was awarded with the Oracle JavaOne Rock Star award when it was held in San Francisco in 2015.

If you want our talk “See like a Terminator – Augmented Reality with Oculus Rift” at your conference as Keynote or standard session (or something else) don’t hesitate to contact us. The talk  is available in lots of time formats and updated all the time with the newest bleeding edge technology out there e.g. Atheer Air.

The Pecha Kucha session was staged by Martin Heider and is definitely worth a visit when being at the OOP conference.

Don’t miss our upcoming talks and events:

09 Mar 2016 03:00 PM – JavaLand in Brühl near Cologne
15 Mar 2016 04:40 PM – CeBIT Developer World / Hall 11, Stand D03 in Hannover
24 Mar 2016 07:00 PM – Nerd Nite München in Munich


Battle of the Drones with LeapMotion

February 26, 2016

“The Gesture Guys” Thomas and Martin made a new video where two Parrot AR.Drones 2.0 are fighting in the style of the Beat ’em up One Must Fall: 2097 video game.

We are controlling the Parrot AR.Drone 2.0 quadrocopters with gesture controls using Leap Motion. With bare hands we are able to control the drones in a very precise way so that they can do awesome fights in the air. Enjoy!

If you plan to do such fights as well download our open source software “Parroteer” from GitHub and make awesome videos and share them with us!


Keynote Talk at OOP 2016 (25th anniversary)

January 30, 2016

IMG_5381-2

At the OOP conference in Munich on the 4th of February 2016 we will have a keynote talk from 12:00 PM to 12:45 PM about “See the world like a Terminator – Augmented Reality with Oculus Rift” as TNG Technology Consulting GmbH software consultants.

As the Keynote talk is in German language you can visit the “Pecha Kucha all night long” at OOP conference instead. It takes place on the same day, but has a different time slot. “Pecha Kucha all night long” is from 6:30 PM to 8:00 PM. For more information please visit the this link.

From Tuesday, the 2nd of February to Thursday, the 4th of February you can find us at the Intel booth showing bleeding edge showcases with Augmented Reality, 3D-cameras, gesture controls, IoT a.s.o. At lunch time and coffee breaks in the afternoon you can find us next to the OOP Internet café where we are showing gesture controlled drones. Visitors are also allowed to fly the drones with bare hands utilizing the Intel RealSense technology.


TNG Augmented Rift presented at Augmented World Expo and Big Techday 8

June 15, 2015

The challenge

The hardware hacking team from TNG Technology Consulting GmbH started to work on a device which is capable to do full field of view augmented reality in March 2015 when the company’s winter retreat was taking place.

The mobile "TNG Augmented Rift" is a full field of view Augmented Reality device capable to do e.g. face tracking, face identification, emotion detection, heart rate detection, distance measurements, etc. Speech recognition for command and control is implemented as well.

The mobile “TNG Augmented Rift” is a full field of view Augmented Reality device capable to do e.g. face tracking, face identification, emotion detection, heart rate detection, distance measurements, etc. Speech recognition for command and control is implemented as well.

The goal

The goal was to built a device made with off-the-shelf hardware in one working day. Within this short period of time eleven TNG software consultants were able to implement a prototype which is based on an Oculus Rift DK2 and an Intel RealSense F200 3D-camera. With some additional days and a core team of about six TNG software developers we were able to add some more features for the Augmented Rift.

The TNG Augmented Rift consists of off-the-shelf hardware like the Oculus Rift and the Intel RealSense F200 3D-camera.

The TNG Augmented Rift consists of off-the-shelf hardware like the Oculus Rift and the Intel RealSense F200 3D-camera.

The features

The TNG Augmented Rift has the following features:

  • See through” Augmented Reality head mounted display with Oculus Rift DK2 (in contradiction to the HoloLense the whole field of view is augmented)
  • You can see a lot of real time elements inside the Augmented Rift in 3D

The following visible elements are supported:

  • Real environment mixed with augmented information and 3D objects
  • Face detection elements
  • Face tracking information
  • Face landmarks
  • Emotions
  • Heart rate as text
  • (in progress) Heart rate as a graph (“Electrocardiography mode”)
  • Speech recognition for command and control (“Okay Rift, this is Martin.” – “Is this really Martin?” – “Yes!” – “From now on I will remember that user 100 is Martin.”)
  • “Terminator mode” (world augmented with red-shaded textures)

The setup

We need two cameras to record the real world environment. The video streams of the two cameras are merged to a stereo (3D) capture. The webcam processor itself consists of different parts. We use the OpenCV library to do long distance measuring and face recognition of people which are not in the nearer distance.

The Augmented Rift consists of two cameras for recording the real world. The information gets streamed into the Oculus Rift DK2. The Intel RealSense F200 is used to add features like face tracking, face identification, pulse detection and much more.

The Augmented Rift consists of two cameras for recording the real world. The information gets streamed into the Oculus Rift DK2. The Intel RealSense F200 is used to add features like face tracking, face identification, pulse detection and much more.

The Intel RealSense detector uses an F200 3D camera that can be used for realizing the features mentioned above (near field face tracking, face identification, face landmark detection and tracking, emotion detection a.s.o.). All the Augmented Reality elements we gather are added to our stereo webcam capture to enrich the real world environment capture with useful information.

Wait but why?

Of course this device based on an Oculus Rift DK2 is a little bit awkward. But imagine what would happen if the computers and cameras used are getting smaller and smaller within the next few years? It doesn’t need a Oculus Rift DK2 to realize such projects. What we need is glasses with an integrated full field of view transparent display to enrich the world with Augmented Reality elements in a progressive way.

Nowadays devices are only augmenting a small fraction of the field of view of the user (see Epson BT200 or Microsoft Hololens). You need to focus on a special direction to see Augmented Reality elements on the display. With the Augmented Rift this is not problem but the device is huge. If there will be glasses with transparent displays available we could transfer our solution to these new devices.

In the front view of the Augmented Rift you can see two web cameras for streaming a stereo capture image into the Augmented Rift. This would not be necessary when using normal glasses with transparent displays inside the glasses and an R200 camera.

In the front view of the Augmented Rift you can see two web cameras for streaming a stereo capture image into the Augmented Rift. This would not be necessary when using normal glasses with transparent displays inside the glasses and an R200 camera.

At least you could argue that the 3D camera used is too huge for such a field of application. But we could also use the very new Intel RealSense R200. It’s such a tiny device and so powerful that you can easily attach it to glasses. The weight is negligibly small.

Intel RealSense R200

Intel RealSense R200

We want to make a video of the TNG Augmented Rift as soon as possible. See the world like a real T-800! The future is here!

The talks and demonstrations

The Augmented Rift was presented at the Augmented World Expo in Santa Clara, California, USA at the Intel booth from the 8th to the 10th of June 2015. On the 12th of June we held a talk at the Big Techday in Munich with more than 100 developers. After the talk hundreds of people wanted to try the Augmented Rift with its Terminator mode on their own!

Disclaimer

The TNG Augmented Rift is developed by TNG Technology Consulting GmbH software consultants. The software parts used by the Intel RealSense are developed by ParrotsOnJava.com

Hasta la vista!


First pictures of the Intel RealSense R200 development kit

May 29, 2015

This week we’ve received an interesting package by Intel USA with the very new Intel RealSense F200 camera. We are really surpised how small the camera is. The camera is a longer range peripheral 3D camera, perfect for sensing the environment (for Windows and Android tablets, 2-in-1s, and more). The “R” in the cameras model name stands for “Rear” since it is best suitable using it while the camera is facing away from you (instead of the F200 camera which is facing frontally).

The camera has a range of 3-4 metres (inside a room) and a larger range out of doors. The key features of the camera are

  • 3D recording (faces, people, environment)
  • Depth camera
  • Face tracking and face recognition
  • Measuring in general

We are now waiting for the SDK to start our first projects with this tiny piece of high-tech.

Intel RealSense R200

Intel RealSense R200


ParrotsOn JavaLand Conference

April 1, 2015

Last week we’ve attented the JavaLand Conference in the  theme park Phantasialand in Brühl near Cologne. We’ve demonstrated our showcases at the “Java Innovation Lab” starting from flying drones with bare hands up to our very new web based Intel RealSense HTML5 game “Parrots On Target”.

TNG Technology Consulting GmbH – the company we are working for as Software Consultants – was supporting us in doing this. We also showed another really cool showcase which was built during the so called “TNG Winterretreat”. This TNG showcase is about an Oculus Rift DK2 enhanced with two cameras to build an “Augmented Rift”. The idea is to view the real world through the Oculus Rift DK2 enhanced by Augmented Reality elements. The Terminator vision is real!

ParrotsOn JavaLand 2015