Research Issue: Sensing

Fachgebiet Augmented Reality (FAR)

General Description

To track users and mobile objects in the real world, suitable sensors need to be registered, calibrated and combined in real-time (i.e.: at video rate). We use, test, evaluate and enhance sensing technology, placing particular emphasis on the establishment of sound characteristics of measurement noise.

Associated People

• Permanent Staff: Martin Bauer, Gudrun Klinker, Daniel Pustka
• External Collaborators: Rudi Lindl (BMW), Katharina Pentenrieder (metaio), Stephan Thoma (BMW), Leonhard Walchshäusl (BMW)
• Former Team Members: Martin Wagner

Associated Projects

DySenNetz (Dynamic Sensor Networks)

This project provides the foundations and formal basis towards building dynamic systems to fuse multiple sensors online. To this end, it a) lays the groundwork for building ad-hoc tracker networks for Augmented Reality, and b) formulates and analyzes accumulation of measurement errors and sensor noise. (Deutsche Forschungsgesellschaft (DFG))

PRESENCCIA

This project undertakes a research programme that has as its major goal the delivery of presence in wide area distributed mixted reality environments. The environment will include a physical installation that people can visit both physicall and virtually. The installation will be the embodiment of an artificial intelligent entity that understands and learns from its interaction with people. People who inhabit the installation will at any one time be physically there, virtually there but remote, or entirely virtual beings with their own goals and capabilities for interacting with one another and with embodiments of real people. (EU 6th Framework, Future and Emerging Technologies (FET)): The Presenccia project

trackframe

Use of AR technology in wide-range industry plants requires a tracking framework which supports a fusion of various tracking technologies based on careful evaluations of tradeoffs between technical requirements and costs. It is the goal of the trackframe project to systematically develop the formal basis of a tracking framework. The concept is expected to consider algorithmic and data-related standards via which tracking systems can be dynamically merged. Furthermore, trackframe aims at providing concepts towards the analysis of quality criteria (such as error statistics for individual sensors) and towards supporting AR-engineers in planning improvements to their tracker configuration. Three demonstrators will exhibit trackframe concepts in industrial and academic setups. (Bayerische Forschungsstiftung (BFS))

COMPOSE

COMPOSE develops and evaluates collision mitigation and vulnerable road user protection systems for trucks and cars. COMPOSE focuses on pre-crash measures to be taken during the last second before a collision. Various application measures are used: mitigation through autonomous braking / brake assist, and also utilization of protective structural actuators. Various innovative sensor data fusion methods will be pursued to achieve the high degree of sensor reliability, which is mandatory for such applications. (BMW rather than TU Munich is involved in this project.)

Associated Publications

	M. Huber, B. Becker, G. Klinker Location aware computing using RFID infrastructure Int. J. Autonomous and Adaptive Communications Systems, Vol. 3, No. 1, 2010, pp. 23 - 38 (bib)
	C. Waechter, D. Pustka, G. Klinker Vision based People Tracking for Ubiquitous Augmented Reality Applications 8th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Orlando, FL, USA, Oct. 19-22, 2009, pp. 221-222. (bib)
	M. Huber, M. Schlegel, G. Klinker Temporal Calibration in Multisensor Tracking Setups 8th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2009), Orlando, USA, October 2009, pp. 177-178 (bib)
	M. Tönnis, R. Lindl, L. Walchshäusl, G. Klinker Visualization of Spatial Sensor Data in the Context of Automotive Environment Perception Systems The Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 13 - 16, 2007. (bib)
	D. Pustka, P. Keitler, M. Huber, F. Echtler, G. Klinker Ubiquitous Tracking for Quickly Solving Multi-Sensor Calibration and Tracking Problems Demonstration at the Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 14 - 16, 2007 (bib)
	T. Tatschke, F. Färber, E. Fuchs, L. Walchshäusl, R. Lindl Semi-autonomous reference data generation for perception performance evaluation Proceedings of the 10th International Conference on Information Fusion (FUSION), Quebec, Canada, July 9-12, 2007 (bib)
	R. Lindl, L. Walchshäusl, V. Paraschoudis Autonomous Braking for Collision Mitigation Purposes by Means of Multi Sensor Perception 6th European Congress and Exhibition on Intelligent Transport Systems and Services, Aalborg, Denmark, June 18-20, 2007 (bib)
	L. Walchshäusl, R. Lindl Multi-Sensor Classification using a boosted Cascade Detector IEEE Intelligent Vehicles Symposium, Istanbul, Turkey, June 13-15, 2007 (bib)
	M. Bauer Tracking Errors in Augmented Reality PhD Thesis (bib)
	D. Pustka, M. Huber, M. Bauer, G. Klinker Spatial Relationship Patterns: Elements of Reusable Tracking and Calibration Systems The Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, Santa Barbara, CA, USA, Oct. 22 - 25, 2006. (bib)
	M. Bauer, M. Schlegel, D. Pustka, N. Navab, G. Klinker Interactive Visualization of the Predicted Accuracy of Marker-Based Optical Tracking Systems Demonstration at the Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, Santa Barbara, CA, USA, Oct. 22 - 25, 2006. (bib)
	M. Bauer, M. Schlegel, D. Pustka, N. Navab, G. Klinker Predicting and Estimating the Accuracy of n-occular Optical Tracking Systems The Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, Santa Barbara, CA, USA, Oct. 22 - 25, 2006. (bib)
	G. Hillebrand, M. Bauer, K. Achatz, G. Klinker Inverse Kinematic Infrared Optical Finger Tracking 9th International Conference on Humans and Computers (HC 2006), Aizu, Japan (bib)
	R. Lindl, L. Walchshäusl Three-Level Early Fusion for Road User Detection PReVENT Fusion Forum e-Journal Volume 1, Sep. 2006 (bib)
	L. Walchshäusl, R. Lindl, K. Vogel, T. Tatschke Detection of Road Users in Fused Sensor Data Streams for Collision Mitigation 10th International Forum on Advanced Microsystems and Automotive Applications, Berlin, Germany, Apr. 25-27 2006 (bib)
	F. Althoff, R. Lindl, L. Walchshäusl Robust Multimodal Hand- and Head Gesture Recognition for controlling Automotive Infotainment Systems VDI-Tagung - Der Fahrer im 21. Jahrhundert, Braunschweig, Germany, 21.11.2005 (bib)
	M. Wagner Tracking With Multiple Sensors PHD Thesis (bib)
	L. Walchshäusl Klassifizierung dynamischer Gesten im Kontext multimodaler Infotainmentsysteme Diploma Thesis, Dezember 15, 2004 (bib)
	M. Wagner, S. Hennauer, G. Klinker Easing the Transition Between Multiple Trackers International Symposium on Mixed and Augmented Reality (ISMAR) 2004, Arlington, VA, USA, Nov 2-4, 2004 (bib)
	H. Najafi, N. Navab, G. Klinker Automated Initialization for Marker-less Tracking: A Sensor Fusion Approach International Symposium on Mixed and Augmented Reality (ISMAR) 2004, Arlington, VA, USA, Nov 2-4, 2004 (bib)
	R. Bauernschmitt, M. Feuerstein, E.U. Schirmbeck, J. Traub, G. Klinker, S.M. Wildhirt, R. Lange Improved Preoperative Planning in Robotic Heart Surgery IEEE Proceedings of Computers in Cardiology (CinC 2004), pp. 773 - 776, Chicago, USA, September 2004 (bib)
	J. Traub, M. Feuerstein, M. Bauer, E.U. Schirmbeck, H. Najafi, R. Bauernschmitt, G. Klinker Augmented Reality for Port Placement and Navigation in Robotically Assisted Minimally Invasive Cardiovascular Surgery Proceedings of Computer Assisted Radiology and Surgery (CARS 2004), pp. 735 - 740, Chicago, USA, June 2004 (bib)
	G. Klinker, M. Wagner 2nd Joint Advanced Summer School 2004, Course 3: Ubiquitous Tracking for Augmented Reality Technischer Bericht der TU München, TUM-I0417, June 2004 (bib)
	R. Bauernschmitt, E.U. Schirmbeck, M. Groher, P. Keitler, M. Bauer, H. Najafi, G. Klinker, R. Lange Navigierte Platzierung endovaskulärer Aortenstents Z Kardiologie 2004 93 S3: 116 (bib)
	G. Klinker, H. Najafi, T. Sielhorst, F. Sturm, F. Echtler, M. Isik, W. Wein, C. Truebswetter FixIt: An Approach towards Assisting Workers in Diagnosing Machine Malfunctions Proc. of the International Workshop on Design and Engineering of Mixed Reality Systems - MIXER 2004, Funchal, Madeira, CEUR Workshop Proceedings (bib)
	F. Echtler, F. Sturm, K. Kindermann, G. Klinker, J. Stilla, J. Trilk, H. Najafi The Intelligent Welding Gun: Augmented Reality for Experimental Vehicle Construction Chapter 17 in Virtual and Augmented Reality Applications in Manufacturing (VARAM), Ong S.K and Nee A.Y.C eds, Springer Verlag, 2003. (bib)
	M. Wagner, G. Klinker An Architecture for Distributed Spatial Configuration of Context Aware Applications 2nd International Conference on Mobile and Ubiquitous Multimedia, Norrköping, Sweden, Dec. 11-12 2003 (bib)
	M. Wagner, F. Löw Configuration Strategies of an AR Toolkit-based Wide Area Tracker The Second IEEE International Augmented Reality Toolkit Workshop, Tokyo, Japan, Oct. 7 2003. (bib)
	M. Wagner Building Wide-Area Applications with the AR Toolkit The First IEEE International Augmented Reality Toolkit Workshop, Darmstadt, Germany, Sept. 29 2002. (bib)
	H. Kruppa, M. Bauer, B. Schiele Skin Patch Detection in Real-World Images Annual Symposium for Pattern Recognition of the DAGM 2002, Springer LNCS 2449, p. 109f (bib)
	M. Wagner DWARF - Design, Prototypical Implementation and Testing of a Real-Time Optical Feature Tracker Diplomarbeit (bib)
	G. Klinker, T. Reicher, B. Bruegge Distributed User Tracking Concepts for Augmented Reality Applications International Symposium on Aumgented Reality (ISAR 2000), Munich, Germany 2000 (bib)
	G. Klinker, D. Stricker, D. Reiners An Optically Based Direct Manipulation Interface for Human-Computer Interaction in an Augmented World Computers and Graphics 23(6), 1999. (Also in Proc. Workshop EGVE'99, Austria, 1999.) (bib)
	G. Klinker Augmented Reality: A problem in need of many computer vision-based solutions NATO Advanced Research Workshop at the 8. International Conference on the Computer Analysis of Images and Patterns (CAIP-99), org. by A. Leonardis and R. Bajcsy. Ljubljana Slowenia, Aug. 29-31, 1999. (bib)
	G. Klinker, D. Stricker, D. Reiners Augmented Reality: A Balance Act between High Quality and Real-Time Constraints Proc. 1rst International Symposium on Mixed Reality (ISMR'99): Y. Ohta and H. Tamura (eds.), "Mixed Reality - Merging Real and Virtual Worlds", 1999. (bib)
	G. Klinker, D. Stricker, D. Reiners The Use of Reality Models in Augmented Reality Applications Proc. European Workshop on 3D Structure from Multiple Images of Large-scale Environments (SMILE), in combination with ECCV'98, 1998. (bib)
	D. Koller, G. Klinker, E. Rose, D. Breen, R. Whitaker, M. Tuceryan Real-time vision-based camera tracking for augmented reality applications Proc. ACM Symp on Virtual Reality, Software and Technology (VRST'97), 1997. (bib)
	D. Koller, G. Klinker, E. Rose, D. Breen, R. Whitaker, M. Tuceryan Automated camera calibration and 3D egomotion estimation for augmented reality applications Proc. 7th Int. Conf. on Computer Analysis of Images and Patterns (CAIP'97), 1997. (bib)
	G. Klinker, K. Ahlers, D. Breen, P.-Y. Chevalier, C. Crampton, D. Greer, A. Kramer, E. Rose, M. Tuceryan, R. Whitaker Confluence of Computer Vision and Interactive Graphics for Augmented Reality PRESENCE - Teleoperators and Virtual Environments, Special Issue on Augmented Reality 6(4), 1997. (bib)
	G. Klinker A Physical Approach to Color Image Understanding Book published by AK Peters, Ltd., Wellesley MA, March 1993. (Based on Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, May, 1988) (bib)
	I. Carlbom, W. Hsu, G. Klinker, R. Szeliski, K. Waters, M. Doyle, J. Gettys, K.M. Harris, T.M. Levergood, R. Palmer, L. Palmer, M. Picart, D. Terzopoulos, D. Tonnesen, M. Vannier, G. Wallace Modeling and Analysis of Empirical Data in Collaborative Environments Communications of the ACM (CACM), Special Issue on the Siggraph'92 Showcase, Vol. 35, Number 6, June, 1992. (bib)
	G. Klinker, S. Shafer, T. Kanade A Physical Approach to Color Image Understanding International Journal on Computer Vision (IJCV), 4(1), 1990, pp. 7-38. (bib)
	G. Klinker, S. Shafer, T. Kanade Color Image Analysis with an Intrinsic Reflection Model International Conference on Computer Vision (ICCV), Tarpon Springs, FL, Dec. 1988. (bib)
	G. Klinker A Physical Approach to Color Image Understanding Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, May, 1988 (bib)
	G. Klinker, S. Shafer, T. Kanade The Measurement of Highlights in Color Images International Journal on Computer Vision (IJCV), 2(1), 1988, pp. 7-32. (bib)
	G. Klinker, S. Shafer, T. Kanade Using a Color Reflection Model to Separate Highlights from Object Color International Conference on Computer Vision (ICCV'87), London, June 1987. (See also proc. DARPA Image Understanding Workshop, CA, Feb. 1987). (bib)
	G. Klinker, S. Shafer, T. Kanade Measurement of Gloss from Color Images Conference on APPEARANCE of the Inter-Society Color Council (ISCC), Colonial Williamsburg, Feb 8-11, 1987. (bib)
	Y.Z. Hsu, H.-H. Nagel, G. Rekers (=Klinker) New Likelihood Test Methods for Change Detection in Image Sequences Computer Vision, Graphics and Image Processing (CVGIP), 26, 1984. (bib)
	H.-H. Nagel, G. Rekers (=Klinker) Moving Object Masks based on an Improved Likelihood Test International Conference on Pattern Recognition (ICPR), Munich, 1982. (bib)