Transcript Pr cis

May 4, 2005
Computer Science Department, Duke University
PhD Defense Talk
Fast Extraction of Feature Salience Maps
for Rapid Video Data Analysis
Nikos P. Pitsianis and Xiaobai Sun
Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis
Feature Salience Maps (FSMs)
• FSMs are used in
– separation or integration
– automatic or assisted visual search tasks
– target indication, object recognition, tracking
• Salient information in multiple feature dimensions
– color, edge orientation, shape, texture, motion
– selective tuning, feedback, attentional or intentional guidance
• High volume and rate of video data, frame by frame
• Involves many filtering steps at multiple spatial scales
MUNDHENK, T. N., ITTI, L. Computational modeling and exploration of contour integration for visual saliency. Biological
Cybernetics (2005).
HPEC 2010
Sept 15, 2010
2
Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis
Processing of feature maps on GPU
NVIDIA Tesla C1060 @1.3GHz, CUDA v3.1
2D Convolutions of 10 512x512 frames and 16 filters/frame
NVIDIA Tesla C1060 @1.3GHz, CUDA v3.1
2D Convolution of 512x512 frames and 16 filters/frame
140
0.3
Spatial domain
Fourier domain
Spatial domain
Fourier domain
120
100
Frames Per Sec
Execution Wallclock Time (sec)
0.25
0.2
0.15
80
60
0.1
0.05
40
5x5
10x10
15x15
Template Size
20x20
• Direct domain
– Filter-centric
– Image-centric
• Fourier domain
– Based on the convolution
theorem
20
5x5
10x10
15x15
Template Size
20x20
• Using CUDA SDK 3.1
– NVIDIA Tesla C1060
• 240 processing cores @
1.3GHz
• 4GB or GDDR3
– CUFFT CUDA FFT library
– Asynchronous I/O and
Streaming
HPEC 2010
Sept 15, 2010
3
Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis
Discussion
• The extraction and use of salient information from static or
dynamic images are recent and active research topics
• The computation based on an extraction model serves two
purposes
– Test and validate the underlying neurobiological model for certain
visual function in the visual system of the primate brain
– Exploit the new understanding and model(s) for developing and
improving artificial vision systems.
• Challenges :
– generation of motion features, which are much more computation
intensive
– visual tasks at the higher levels
• segmentation, object recognition, tracking of moving targets.
• data representation at higher levels, sparse and irregular, but still
structured
– Efficiency of high-level processing steps on GPUs
HPEC 2010
Sept 15, 2010
4