 |
|
|
|
|
|
|
|
|
| |
Generic Visual Categorizer
Automatically classify images based on visual content.
Technology Description
Generic visual categorization (GVC) assigns one or multiple labels to an image
based on its semantic content. "Generic" highlights the goal of classifying a
wide range of objects and scenes.
Xerox has developed a technique that is sufficiently generic to work with
several object types simultaneously and which can be readily extended to new
object types. It can also handle variations in view, imaging, lighting and
occlusion (partial visibility), typical of the real world, as well as the
intra-class variations typical of semantic classes of everyday objects (e.g.
size, shape, form, color).
Xerox’s breakthrough GVC technology is the result of combining Xerox
scientists' expertise in image processing, computer vision and machine learning.
The Xerox visual categorisation method first extracts and describes patches
found in an image. It then maps patch descriptors to "visual vocabularies"
which are sets of predetermined clusters of patches called "visual words".
Visual vocabularies are learned automatically from training sets and provide an
intermediate representation (hidden layer) bridging the semantic gap between
the low-level features extracted from an image and the high-level concepts to
be categorized.
Since one universal vocabulary made of the most frequent visual words across
all the considered classes is not sufficient, Xerox borrowed a technique from
speech recognition known as "vocabulary adaptation" to derive class-specific
vocabularies from the universal vocabulary.
For each class, an image is characterized by a histogram of visual word
occurences, which determines whether the image content is best modeled by the
universal vocabulary or the corresponding class vocabulary.
Performance
Advantages of the Xerox method include simplicity, computational efficiency,
scalability, robustness to variations, and applicability to all types of
classes and objects. Presently, GVC has been trained for about 100 categories.
Rigorous tests involving more than 30 simultaneous categories have demonstrated
state-of-the-art categorization performance:
- Classification run-time of 0.2 to 0.5 sec per image (depending on processor
performance)
- ~0.1 msec computational increment per added class
- Equal error rate ranging from 2% to 10% (depending on class, independent of
number of classes)
Applications
- Automatic tagging of images (e.g., images in documents, photographic
archives, consumer photo albums, online shopping catalogues)
- Content-based image retrieval
Generic Visual Categorizer Training Tool
Train your own Visual Categorizer from a collection of tagged images
Presently, GVC has been trained for about 100 categories. To meet customers'
needs beyond the current coverage, Xerox has developed a beta version GVC
Training Tool with a graphical user interface which is simple enough to be used
by any holder of tagged images, while sufficiently sophisticated to give
performance feedback and offer an iterative training process.
Tagging of the Training Material
The training material is a collection of tagged images. There is no need to
associate tags with corresponding regions in the image (no segmentation). The
collection should be representative of what the visual categorizer is expected
to recognize at run time. It should also be diverse. An image in the training
set can be labelled with multiple tags. Training iterations are likely to
detect the wrong and the missing labels, thus giving the user a chance to
improve performance.
Training Iteration Settings
Starting from such a tagged collection, the user can define the following
settings:
- Subset: models can be trained for a subset of all categories
- Class aggregation: classes can be aggregated at training time. For example,
if the training dataset includes distinct labels for "cats" and "dogs", the
user may however prefer to combine them as "pets"
- Class-to-class neutrality: when two categories are semantically close to
each other (e.g. Forest / Trees), the user can instruct the Training Tool to
neutralize some tags during training. By default, images tagged anything else
than "Forest" are negative examples of the "Forest" class. It is wise to handle
"Trees" as neutral examples vis a vis the "Forest" category
Training Process
A progress bar helps visualizing the process steps. Some intermediate results
are saved for later reuse. Training time is dependent on the computer hardware
(processor, memory, hard disk). Training 40 categories from 30,000 labelled
pictures is completed in a few hours on a PentiumŪ 4 or Athlon™ 64 processor
computer.
Interactive Performance Feedback
The Training Tool can measure the performance of the models just trained, by
using the technique of N-fold cross-validation, where N is set by the user.
This randomly splits the training set in N subsets of equal size, and
iteratively subtracts one subset from the training dataset and uses it as test
data. The performance can be visualized in several ways incl. precision,
recall, F1, accuracy, confusion matrix.
Performance Measure (percision & recall)
The user can easily correct gaps in the tagging. Color coding indicates how
each image was tagged, while the scores result from the models just trained.
Ranking by scores and looking at the highest and lowest ones help to identify
mismatches in the tagging, which can be easily corrected.
Assistance in Tagging
Based on the performance measure the user can decide on appropriate next steps,
which may include:
- Remove, aggregate or redefine certain categories
- Add, remove or edit some tags or images
- Neutralize certain tags vis a vis certain categories
- Launch a new training iteration
- Import the trained models into the GVC run-time
Intellectual Property Summary
Xerox Intellectual Property includes patents, patent applications, and know-how.
For Licensing Information
To learn more about licensing the Generic Visual Categorizer technology.
|
