Transcript 08-NATO-talk - Video Recognition Systems

Automated video surveillance:
challenges and solutions.
ACE Surveillance (Annotated Critical Evidence)
case study.
Dmitry Gorodnichy and Tony Mungham
Laboratory & Scientific Services Directorate
Canada Border Services Agency
www.videorecognition.com/ACE
Outline

Problems with status-quo Video Surveillance



Next generation solution - Video Analytics based




2
“Motion detection” myth and problem
“Object detection” as example of real intelligence
ACE Surveillance – first fully-functional objectdetection-based prototype


Real-time and archival problems
Operational considerations
Year long tests with different levels of complexity
What that means for future of Video Surveillance
Conclusions
Role of Video Technology (VT)

In the context of enhancing security, Video
Technology (VT) is one of the most demanded
technologies of the 21st century



Multi-million funding in Canada and worldwide:


3
It is publicly acceptable
It provides rich in content data
CBSA Port Runner project invested 10s of Millions in
CCTV upgrades
Transport Canada opens $35M of funding towards
procurement of CCTV
VT at CBSA

CBSA is a major user of CCTV systems at POEs
 Most major CCTV installations start to leverage VT
 Current task: to lead applied R&D to push VT to help
CBSA apply S&T innovative approaches to border
management:




4
Event detection and notification to provide effective response
to events
Traffic trends analysis to assist with border management
Video storage management to manage the cost of storage
and meet obligations under the privacy act
Data integration/fusion of contextualised video information
Problem with status quo use of CCTV
surveillance
Modes of operation:
1. Active - personnel watch video at all times
2. Passive - in conjunction with other duties
3. Archival - for post-event analysis
Current systems and protocols are not efficient
in either mode!
Problem in real-time modes: an event may easily
pass unnoticed .


5
due to false or simultaneous alarms,
lack of time needed to rewind and analyse all video
streams.
Problems in Archival mode:
Due to temporal nature of data:
1. Storage space consumption problem
• Typical assignment:
2-16 cameras, 7 or 30 days of recording, 2-10 Mb / min.
1.5 GB per day per camera / 20 - 700 GB total !
2.
Data management and retrieval problem
• London bombing video backtracking experience:
“Manual browsing of millions of hours of digitized video from
thousands of cameras proved impossible within timesensed period”
[by the Scotland Yard trying to back-track the suspects]
6
Operational considerations




7
Lots of CCTV infrastructure: Many local initiatives, not
coordinated
Most video technology decisions are influenced by
vendors - short-term solutions
Over 30 different video systems within the same dept. (at
RCMP)
A national program with proper benchmark-based planning
and evaluation of VT is required
 Leveraging advances recently made in S&T
 Technical standards for capturing /saving video data.
 Policies in when, where and how VT should be used.
Video Technology today
Video Analytics (Video Recognition)
21st century
Wireless, Network Connected (IP)
Digital
Analog
First video recording
20th century
8
Next generation Video Technology
Is Video Analytics based
also identified as:
 Video Recognition,
 Intelligent Video,
 Smart Video / Smart Camera
 Video Analysis & Content Extraction
 Perceptual Vision
 is not much about capturing better data (better
lenses, grabbers, coders, transmitters)
 but about understanding captured data (better
theory)
9
Status-quo “video intelligence”
Transport Canada CCTV Reference Manual for
Security Application .
Australian Government National code of practice for
CCTV applications in urban transport
USA Government :recommended security Guidelines
for Airport Planning, Design and Construction.
…. refer to “Motion-based” capture as Intelligent
Surveillance Technology, and make their
recommendations based on thereon.
10
“Motion-detection” is not intelligent!


Term “Motion-based” is coined to make people believe that
video recognition is happening, which is not!
It’s actually illumination-change-based, as it uses simple
point brightness comparison:

Which often happens not because of motion!
Changing light / weather (esp. in 24/7 monitoring)
 Against sun/light, out of focus, blurred, thru glass
 Reflections, diffraction, optical interferences
 Image transmission, compression losses

11
“Object-detection” is intelligent …
… but few can do it, since necessary advances in video
recognition theory became possible only recently (>2002).
In 2002 National Research Council of Canada (NRC) starts
developing Video Recognition Systems to leverage its
scientific Video Recognition expertise for the industry.
In 2005, it develops ACE Surveillance:
an object-detection-based Automated surveillanCE
prototype capable of automatically extracting
Annotated Critical Evidence from live video.
NRC becomes also the organizer of the first Canadian academic workshops
dedicated to Video Processing for Security (since 2004)
12
What is ACE Surveillance?


A Windows software that performs real-time video analytics by
integrating best object detection and tracking algorithms.
Replaces video clips with annotated still images:
 Compresses 1 Gb of video into 2 Mb of easy to browse and
analyze still images
ACE Surveillance output:
A 7-hour activity from day to
night (17:00 - 24:00) is
summarized into 2 minutes
(600Kb) of Annotated Critical
Evidence snapshots.
13
Note illumination changes! - Watch tree
shadows and sun light.
ACE Surveillance architecture

Works with ordinary USB cameras or CCTV cameras with USB
video converters.
Adds on top of existing infrastructure using an ordinary desktop
computer.

ALARM!
..
.
Video clips (Tb)
C.E.S. (Gb)
Last captured
CES
14
Real-time mode
of operation
Archival mode
of operation
ACE Capture
ACE Browser
Adds on top of existing infrastructure
15
Status quo “Motion-based” capture
(Courtesy: NRC-IIT Video Recognition Systems project)
1. Many captured snapshots are
useless: either noise or
redundant
2. Without visual annotation,
motion information is lost.
16
3. Hourly distribution of
snapshots is not useful
ACE Surveillance “Object-based” capture
(Courtesy: NRC Video Recognition Systems project)
17
1. Each captured shot is useful.
2. Object location and velocity
shown augmentent.
3. Hourly distribution of shots is
indicative of what happened in
each hour, provides good
summarization of activities over
long period of time.
ACE Surveillance testing benchmarks
Tested in different levels of complexity:
 lighting conditions,
 object motion patterns,
 camera location
 environmental constraints.
most difficult - outdoors in unconstrained environments with
little or no object motion consistency (as around a private
house in a regular neighbourhood).
most easy - in controlled indoor environment where minimal
direct sunlight is present and where all objects are of
approximately the same size and exhibit similar motion
pattern (as at access gate inside the business building).
18
Outdoor, wireless, eye-level
Outdoor, webcam, overview
Indoor with sunlight, CCTV
VT within CBSA
Indoor w/o sunlight, CCTV
19
On week-end
On week-day
Enables efficient detection of abnormal
activities
Back Door Entry
Delivery Entry
20
More than usual
ACE Surveillance results
In real-time mode: alarm sounds & last captured evidence
(time-stamped) is shown.
In archival mode: “Zoom on the evidence” browsing of captured
evidences – zoom on a day, on hour, then on event - point
and click (for high res as needed)
Made Commissioners much more aware of activities.
21
Conclusions

Affordable automated (intelligent) video surveillance
(AVS) is possible!


To replace traditional DVR
OR to supplement them: DVR for 1 month + AVS for 1 year
However:
 Requires extra training from security officers.
 Requires new protocols to handle automatically extracted
evidence.


Requires new privacy policies.

22
- From forensic prospective, data that are not original and have
been processed by a computer can not be considered as evidence.
- Surveillance data are normally not kept for a long period of time
(<1 month), due to their size. AVS allows to store on local machine
many months (even years) of evidence data.
ACE surveillance case study outcome


ACE Surveillance (which is developed by a research lab)
provides a reference standard against which can be
measured solutions coming from industry.
It deals with common misconceptions related to deploying
intelligent video surveillance systems (IVS):




23
“motion detection” myth vs real object detection and tracking.
The “one-fit-all” myth. - Extra video analytics expertise is required to
set and operate IVS.
better video data (better resolution or compression) do not imply
better video intelligence. - ACE Surveillance is shown to work with
regular TV quality data (320 by 240 pixels).
 However better quality of video image is needed for forensic
purposes as evidence
Due to closing of the project by NRC, CBSA takes lead on it.