Until recently, all wet film and digital images of license plates were interpreted by
humans. However, much technology that was developed in the cold war has now found
it's way into the photo-enforcement industry. Companies and consultants such as those
listed below are now aggressively providing products to automate this function.
License plate identification/recognition (LPI/R) is one form of ITS technology that not
only recognizes and counts vehicles, but distinguishes each as unique. For some
applications, such as electronic toll collection and red-light violation enforcement,
LPI/R records a license plate's alphanumerics so the vehicle owner can be assessed the
appropriate toll or fine. In others, like commercial vehicle operations or secure-access
control, a vehicle's license plate is checked against a database of acceptable ones to
determine whether a truck can bypass a weigh station or a car can enter a gated community
or parking lot.
LPI/R can be used to issue violations to speeders or simply to offer speeding drivers a
reminder by displaying a plate number with the vehicle's speed on a variable messaging
sign. It can facilitate emissions testing by recording a plate's alphanumeric sequence
while automatically analyzing tailpipe effluents, or help identify and fine violators.
LPI/R also can monitor the time it takes vehicles to travel from one point to another,
keeping traffic management centers apprised of transit times along busy streets and
highways.
At international border crossings, license plates-- the only universal vehicle
identifier-- can be checked against a database of "hot" cars to locate stolen
vehicles and plates or those registered to fugitives, criminals, or smuggling suspects.
System Components & How They Work
A typical LPI/R system is comprised of a video image-acquisition subsystem, a CPU for
image processing and control, a hardware- or software-based character recognition engine,
and a storage or transmission subsystem for electronically recording plate contents and
data such as date, time, and location.
At any LPI/R system's heart is its recognition engine and the embedded algorithm. It is
important to understand that the system supplier decides how to code the recognition
engine. The user, OEM, or integrator takes that algorithm along with the system. A basic
understanding of how the recognition engine interprets image content is central to
confirming that the overall "solution" can handle a given application.
The correlation or template matching approach to character recognition is
straight-forward and can be reliable, provided the target is "cooperative" and
the application remains invariant. As the name implies, once each character is isolated,
the recognition engine attempts to match it against a set of predefined standards. Any
condition-- lighting, viewing angle, obscuration, plate size, font-- that causes a
character to vary from the standard is likely to confuse the engine and return a
questionable result.
Structural analysis uses a decision-tree to assess the geometric features of
each character's contour. The technique can be somewhat tolerant of variations in size,
tilt, and perspective. As a simple example, the characters B, D, 6, and 9. Features that
might be used to distinguish them are the number of loops in each character-- one or two--
and the vertical position of the loop-- top, central, or bottom. Two loops point to a B,
and one loop leads to the next branch of the tree. A loop at the top indicates a 9; if the
loop is central, it's a D, and a loop at the bottom means a 6. Characters without loops
(E, M, N, hyphen, etc.) require additionally complex, time-consuming analysis.
Neural networks are trained by example instead of being programmed in a
conventional sense. While learning to recognize a recurring pattern, the network
constructs statistical models that adapt to individual characters' distinctive features.
Therefore, neural networks tend to be resilient to noise, and performance usually is not
compromised under changing operational conditions. However, each modification (e.g. a new
font) that is presented to the neural network may require a significant investment in
retraining.
What to Look For
Evaluating a system's capabilities can require that one look beyond the recognition
engine's proficiency with a standard character set under nearly ideal test conditions. It
is prudent to determine all factors that can influence operations and to learn the
effects of those variables that cannot be held constant. They include:
- vehicle speed
- volume of vehicle flow
- ambient illumination (day, night, sun, shadow)
- weather
- vehicle type (passenger car, truck, tractor-trailer, etc.)
- plate mounting (rear only or front and rear)
- plate variety
- plate jurisdiction (and attendant fonts)
- camera-to-plate distance
- plate tilt, rotation, skew
- intervehicular spacing
- presence of a trailer hitch, decorative frame, or other obscuring factors
Applications differ in their requirements. Some mandate that information from the plate
be reported to the issuing jurisdiction, others need additional I/O, such as interfacing
to a gate controller, an off-site database, or remote archival facility. In those cases,
it becomes essential to make sure the recognition engine and its host computer can handle
the added computational load. Even if that degree of complexity is not contemplated
immediately, it is wise to consider whether future enhancements will be possible without a
major system reconfiguration.
Buyers should distinguish between absolute identification of each and every character,
as for a traffic violation application-- an example that requires doing so accurately and
reproducibly-- versus recognizing a plate that the system has seen before; as in inventory
control where the universe of plates is finite and known. The degree of computer
processing, the recognition engine's precision, and internal accuracy monitoring are of
greater import in the former scenario.
Presently, LPI/R systems excel at inventory control. Many already are being used for
vehicle surveillance, monitoring, and origin-destination surveying. In those situations, a
single character error is less significant than in enforcement applications. It's easier
for a system to recognize an entire plate it has already seen-- whether or not it was
interpreted accurately-- than to properly identify each and every character, time and
again, under all operating conditions. Clearly, a system designed to identify and fine
violators automatically must be sufficiently accurate to prevent law-abiding drivers from
receiving undeserved tickets.
A Word on Accuracy
Determining the accuracy of an AVI system is complex and depends on the application,
operating conditions, and assumptions made during testing. When evaluating a system, it is
important to use those criteria to examine manufacturers' claims carefully.
System performance is difficult to quantify. It's tempting to expect a machine to be
perfect and to assume one hundred percent accuracy in interpreting license plate content.
However, some plates cannot be read at all-- neither by machine nor by eye-- due to dirt,
poor lighting, damage, or obscuration. An automated system, therefore, should not be
expected to achieve perfection, even under ideal conditions.
One way to measure success is as the percentage of license plates correctly identified
by the machine that can be verified by a person looking at the raw video signal on a
monitor. If a person must guess from a poor video representation, it is probable the
machine also will produce a lower-confidence answer. Looking at clear transmissions, the
human is less likely to err, and an automated system similarly will return a higher degree
of accuracy.
Of course, a machine can only identify a license plate's alphanumeric content after it
properly recognizes that a plate is present in the field-of-view. The success rate of each
step thus must be figured into the overall calculation:
- A = (T * I) * 100 (1), where
- A = total system accuracy, expressed as a percent
- T = rate of successful plate recognition, expressed as a decimal number
- I = rate of successful interpretation of entire plate content.
If the interpretation of every character is assessed individually, the equation
becomes:
- A = (T * I1 * I2 * . . . In) * 100 (2), where
- A = total system accuracy
- T = rate of successful plate recognition
- I1 = rate of successful interpretation of first character
- I2 = rate of successful interpretation of second character
- In = rate of successful interpretation of nth character.
But a system's overall accuracy cannot be extrapolated directly from its individual
character accuracy. For example, let us assume a system recognizes and identifies ten
thousand license plates with seven characters on each plate for a total of seventy
thousand characters. If the system reads the first six characters correctly on each plate
but misses the last character on every plate, one might be inclined to state the overall
accuracy as (60,000 ÷ 70,000) * 100, or 85.7 percent. However, using Equation 2, the true
system accuracy in this case is zero.
- A = (T * I1 * I2 * I3 * I4 * I5 *
I6 * I7) * 100 (3)
- A = (1 * 1 * 1 * 1 * 1 * 1 * 1 * 0) * 100 (4)
- A = 0 percent (5)
Performance attributes must be specified carefully to be meaningful. Simple statements
like "eighty-five to ninety percent accurate" can hide important assumptions and
often engender unrealistic expectations. Avoiding the pitfalls of operator bias while
truthfully demonstrating statistically significant results about correct interpretations
requires a large sample size, studied under varying conditions of illumination, speed,
camera offset angle, precipitation, etc. Usually time and money are too limited for
systems developers to perform such rigorous testing.
The supplier should clearly and precisely define the conditions under which the system
achieved the stated accuracy rate. A system may correctly identify plates ninety-five
percent of the time under controlled conditions, but only fifty percent of the time under
less ideal conditions. The supplier also must explicitly state the definition of failure:
a failure could signify a missed plate (no recognition) or an error in the interpretation
(identification) of one or more characters.
The importance of a failed identification depends on the application. For example, in
secure-access control, any system failure would be unacceptable because that could admit
someone not authorized for entry (false positive) or deny admission to an authorized
person (false negative). Different threshold settings may predispose a system to one or
the other type of error. Operational constraints determine which type of error to favor,
allowing designers to set thresholds appropriately.
Using the access control example, postprocessing may compensate for imperfect
interpretation. Following the recognition step, each plate's sequence of characters is
submitted for matching against a database of known sets. A single character error would be
insignificant if the system were instructed to grant admission under the following
conditions.
The plate is ABV 123 (Fig.4):
The database entry is ABV 123 (Fig.5):
The plate is read as ABW 123 (Fig.6):
Given a finite database populated with plate ABV 123, designers must consider the
likelihood of encountering the true plate ABW 123 and the implications of admitting an
unauthorized vehicle.
What's In Store
Computer-based plate recognition emerged in the 1980's. In 1993, LPI/R technology made
a successful transition from the research bench to the commercial marketplace. Recently,
as supply and demand dynamics took hold, off-the-shelf components started to become
available from a greater number of vendors. Now, with over three dozen suppliers offering
commercial LPI/R products, the technology is finding its way into progressively more
solutions-oriented ITS systems.
As LPI/R enjoys heightened visibility among law enforcement, commercial, and private
sector communities, potential users will be exposed to the wealth of AVI (automatic
vehicle identification), AVL (automatic vehicle location), ETTM (electronic tolling and
traffic management), ITS, and video violation enforcement (VES) applications that can be
addressed with the technology. Likewise, new challenges are sure to be found in
ever-increasing plate varieties, smaller-sized alphanumeric characters, decorative fonts,
and a more prevalent call for reporting not only plates' letters and numbers but the
issuing jurisdiction as well. (Note that not all U.S. States, for example, have the
complete State name on the plate. Even when the name is present, it's usually too
small to be captured by the image acquisition subsystem.)
Those observations not withstanding, several vendors are rising to the occasion with
innovative and creative outcomes:
For plates where
foreground/background contrast makes even visual identification difficult, Racal Messenger
Ltd. is employing a dual-camera configuration using visible and infrared regions of the
electro-magnetic spectrum. Filtering appropriate wavelengths significantly improves
character readability for plates such as the one from the State of Illinois, shown in
Figure 1.
The United States federal
government-issued plate's (Fig.2) leading half-height characters are unreadable at
typical CCIR resolution. Computer Recognition Systems, Inc. integrates an ultra-high
definition image capture "front-end". At 1300 * 1030-pixels, those important
alpha-numerics can be imaged with sufficient resolution to allow automatic recognition.
AutoVu Technologies Inc.'s
pattern matching methodology not only reads a plate's alphanumerics, the company's
application software analyzes the entire image of the plate and queries a database for
potential matches. Under that scenario, it may become less vital to report the issuing
jurisdiction which-- as on the State of Tennessee sample (Fig.3)-- is not always indicated
on the plate.
As transportation and law enforcement organizations tap into near real-time access to
who (or which vehicle) is where, we all may benefit from heightened safety and violation
enforcement, faster transit times, and guidance to less congested routes. Prospects seem
positive for the growth of an international market to help realize that potential.
- Lee J. Nelson, Electro-Optical Technologies, Inc.
E-OTEK®
Electro-Optical Technologies, Inc. (E-OTEK)
helps clients assess emerging technologies while developing commercial
applications for them in the high-performance electronic imaging and graphics
arena.
Principal Systems Consultant, Lee J.
Nelson, is an independent analyst with expertise in all technologies
pertaining to raster-based digital image processing and display.
Electro-Optical Technologies, Inc. (E-OTEK) helps clients assess emerging technologies
while developing commercial applications for them in the high-performance electronic
imaging and graphics arena.
Principal Systems Consultant, Lee J. Nelson, is an independent analyst
with expertise in all technologies pertaining to raster-based digital image processing and
display.
Send E-mail
