(as of 2004)
Recognition of human actions and related topics
Recognition of human actions is an important research topic which has many
application areas. One of our research goals is to establish a
library of computational procedures for the recognition of a wide
variety of human actions .
Examples of such actions are the "walking" and
"running" actions, which are entire body actions,
manipulative actions such as
"grasping a cup", or the "reading action" which can only be judged
by jointly evaluating several sub-activities over an extended period
Our projects can be divided into two categories according to the sensing
techniques being used:
1. One of the projects is based on solely
monocular color image sequences as
input data. The research objective here is to develop techniques that are
able to recover enough 3D information from an image sequence to be able to
estimate the 3D trajectories and
velocities of moving persons and their major
body parts, such as hands and legs.
Example of "walking" action:
Left image: Comprehensive view of the action scene;
Center image: Walker trajectory in 3D space, estimated with our method;
Right image: Walker velocity, estimated with our method.
2. In another project, monocular color image sequences
in conjunction with (synchronously taken)
thermal image sequences (from an infrared camera) and
3D object surface point measurements from a stereo camera
are used as input data. The objective here is to develop first the most
appropriate image acquisition system and
then develop reliable data fusion techniques
which work both at the signal level
and decision-making level in order to (hopefully) obtain
robust action recognition results.
Another topic involving the automatic sensing and recognition of human
actions we have been persuing in our laboratory
is the recognition of objects which are being pointed at by a person in
front of the camera system. First, the system detects a finger pointing action
in real-time, then tracks the hands of the person and determines whether the
hand motion represents a finger pointing action. If it does, the system
estimates the direction of the line-of-sight of the person and intersects it
with the stored environment model of the room. As a result of this intersection,
objects pointed to by the person can be identified. The following photo shows
such a finger pointing scene.
Human face recognition
In this area we proceed along three different lines of
research which we are planning to eventually merge into one coherent
method for general--purpose, unconstrained face recognition .
(1) Identification of faces by applying
Neural Networks and Support Vector Machine
learning and recognition methods to 2-dimensional face image sets which are
obtained with image sensors working in the visual wavelengths and infrared
(2) We investigate the possibility of using 3-dimensional face surface
point sets obtained with a 3D laser scanner for face identification.
(3) We develop methods for the on-line acquisition of human face images of
persons moving in a room; for this purpose we use the combination of a
laser scanner with a line scan characteristic and a color CCD camera,
both mounted on the same platform and pointed in the same direction.
The following images show the system and the laser scanner.
Development of color and texture similarity measures
Color plays an important role in our daily lifes, and likewise it is an
important source of information in the fields of image processing and
computer vision. Perhaps the most fundamental operation involving colors
is their judgment as to whether two colors are the same, or similar, or
completely different. If they are similar, we would like to know how similar
they are. For this purpose we need color similarity measures; these are
mathematical formulae which assign to two test colors a real number in the
range between 0 and 1: 1 for equal and 0 for completely different, and all
numbers in between give the degree of similarity. In our laboratory, we
evaluate existing color similarity measures and develop new ones according
to various criteria that make these measures useful for various applications.
For example, it is sometimes necessary to judge by computer whether a given
color similar to the color of human skin (within the same ethnic group), and it
is desirable to have the best color similarity measure available that can
be used for this task. The following image shows several color similarity
images which were computed with several different color similarity measures.
Note the differences in their performance.
Development of wearable machine vision systems