Objects recognition and tracking system

FIELD: systems for recognizing and tracking objects.

SUBSTANCE: system has matrix sensors, each of which is meant for performing functions of first type sensor, providing for possible detection of object presence in working zone of sensor and determining position thereof, and second type sensor, providing for possible use of this object position, determined by first type sensor, for identification or recognition of object, and possible focusing or operation with greater resolution then first type sensor.

EFFECT: higher efficiency.

16 cl, 12 dwg

The technical field

The invention relates to a system for object recognition and tracking them.

Review of the known technical solutions

A known system for counting objects, such as people passing through the corridor, for example, in a shopping center, but such systems are not adapted to recognize or distinguish objects of a specific type or different types and their position, and to track such objects over time.

The purpose of this invention is to provide a system for object recognition and tracking them.

The invention

In accordance with one form of implementation, which may not be the only or broadest, the invention relates to a system for object recognition and tracking them containing at least one matrix sensor, and specified or each matrix sensor contains a sensor of the first type and the second sensor type, the sensor of the first type configured to detect the presence of the object in the working area of the sensor and determine its position, and the sensor of the second type is made with the possibility of using this particular first position sensor object for identification or recognition of the object or to the counting or registering the presence of the object, if it is the object of some of the selected type.

Presets the ance object, which is determined by the first sensor, can be detected by changes in high contrast edges or changes in the picture image or movements. Thus it is possible to detect objects that move in the working area of the sensor.

The sensors can be made with the possibility of using part of the electromagnetic spectrum selected from radar, microwave, radio frequency, infrared, millimeter and optical ranges or use of sound location or other detection system.

The sensors of the first and second types can be devices of the same type or they may be devices of various types. Thus, the sensor of the first type may be a TV camera with low resolution, and the sensor of the second type may be a radio frequency detection system.

If the first sensor and the second sensor are in fact one and the same device, the second sensor can operate with a higher resolution than the first, or may be a sensor identifies the specific type.

In the preferred form of the invention, the sensor of the first type may be a device type of insect eyes. Namely, it may be a sensor that is designed to detect changes in contrast between related the diversified pixels in the field of view, instead for the final identification of objects. In this form of the invention, the sensor of the second type may be a sensor with digital signal processing, such as digital television sensor. Thus, the sensor of the first type is used to identify in the working area of the sensor specific area of interest, and then the sensor with digital signal processing only needs to identify the object by looking at this specific area of the working zone, and does not handle all of the television image of the monitored space. Because it requires much less computing power and can be achieved with higher efficiency.

The first sensor and the second sensor may be a single device, where the second sensor provides a second stage of data processing.

Thus, in an alternative form of the invention, the sensors of the first and second types can be made in the form of a single TV sensor, observed which the image is processed in two different ways. First, the observed image can be transformed into digital form, to enroll in consecutive memory area, and then a sequence of images can be analyzed to detect changes in consecutive frames digital is the first image, which will point to moving contrasting border. There is determined by the moving object in the working area of the sensor, and the same digital image can then be analyzed in this specific area of the working area sensor of the second type. To improve detection, or result in the object of interest, the sensor of the second type can use the next or previous frames.

If the recognition process does not reach threshold levels of reliability, predetermined or obtained as a result of artificial learning, then additional stored or loadable algorithms can be used in an iterative process until the results do not improve.

While the sensor of the first type is used to identify objects of interest, the sensor of the second type is used for the purposes of recognition. Each object has a number of characteristics such as weight, height, shape, contrast, brightness, color, patterns, speed, heat reflectivity, signature and many others. Each feature by itself is not necessarily suitable as a characteristic that can be used separately for recognition of the object, and some features may be more easily identifiable than others. Preferably the, the invention provides that the analysis of the most easily recognizable characteristics is performed first, and then using other features and successive time frames, the system at each step in the iterative process of analysis improves the probability of detection by the object with greater accuracy. The definition of what characteristics are analyzed first, and the order in which they are analyzed, is the most important part of this invention.

In another form of the invention, the system can be built with artificial intelligence to make these decisions without external interference. In this case, the system determines what characteristics will be analyzed first. Over time the system can identify a new feature or a new sequence of characteristics that can better identify the object of interest, and therefore becomes a self-enhancing. In this case samosoversenstvovanie procedures or algorithms stored in memory in the current memory-resident library for future reference. The system will use the analysis using artificial intelligence to solve, using probabilistic analysis, which algorithm is most appropriate for its success and speed whic is navane.

Thus, the invention generally describes a system that can be broken down into several stages. The first step of treatment is to enable the machine to detect an object that is designed to capture images and download sequences of digital images of the hardware in the appropriate digital format for the machine to recognize the object. The sensor of the first type may identify the boundary of the object, determining its on successive frames when the object is moved through the working area of the sensor. The sensor of the second type, which may form the second stage of the system is a machine for recognition of the object, which, firstly, is used to convert certain characteristics in suitable mathematical or algorithms and, secondly, can be used to view and record the results in the time required to analyze specific characteristics of the considered objects. Characteristics under analysis may include, but are not limited to, one or more characteristics, such as weight, height, shape, color, pattern, contrast, texture, heat reflectivity, frequency, speed, radiation signal, signature, etc. Each characteristic of the detected object is assigned probability that the detected object, represent the work of interest. To obtain higher levels of precision serial digital images can be analyzed over time using stored procedures, stored libraries, algorithms, analysis, and iterative process of analysis, which can provide surveillance of many characteristics of many objects in real time.

Each iteration of the analysis increases (or decreases, as appropriate) the probability for the detected object to be an object of interest. If the confidence level reaches a predetermined level of recognition or threshold level, the object is recognized as the object of interest. To improve probabilistic analysis and increase the probability of recognition of the object, additional types of sensors of the second type can be included in one work area that are used for monitoring various characteristics of the object of interest.

Sensors additional varieties can be located in different places and are directed at different angles, but to view the same working area, to improve the probability of distinguishing objects positioning objects in three-dimensional space or location of the object, otherwise, when using only one sensor t is a, would be shaded.

The sensor of the second type can be made with the possibility of retrieving the requested device associated with the object. In one form of the invention, the first sensor can be performed with the opportunity to observe that the object moves within the working area, and the requesting device of the second sensor is used for the subsequent request of the radio frequency signature of the integrated circuit, such as integrated circuits smart cards or transponder to provide a confident identification of the object and passing through the working area of the sensor.

Recognition and tracking of objects may include a few matrix of sensors, the sensors of several types and several working areas of the sensors.

Matrix sensor can be made interoperable with neighboring matrix sensors in several work locations of sensors to transmit or receive information, which is accepted or obtained through training. This information transfer can be carried out in a closed network such as Intranet or on an open network such as the Internet. Solutions or information will be passed on to the following sensors or other devices of third parties that require such information to carry out, when necessary, pre-programmed actions. The sensors can take the right, to give commands or to initiate some other form of action without human intervention.

The third stage of the system can be represented in a machine for tracking. Recognizing the object of interest, the machine for tracking can perform such tasks as initializing the grid, two-dimensional or three-dimensional, a label, a tracking label and send the labels in the following working areas of the sensors. This requires significantly less processing power than machines for object recognition, when identified and marked as the object moves from the coverage area of one sensor in the working area of the other. If in the following the working area of the sensor object will appear, which previously was not detected due to the fact that was obscured by another object, the process may be traced in the reverse direction through all the possible recorded images images and recordings can be appropriately adjusted. The whole system is updated with this new information. This greatly increases the accuracy of the system. From this point, the object can be tracked forward in time until such time as he shall go out from all working areas of the sensors. After recognizing a type of object that you want to track the position of each object of interest, over BP is like sequentially displayed on the universal grid. Because the computational power required for tracking is significantly reduced in successive work areas of sensors, multiple objects can be tracked with high accuracy and reliability.

In the preferred form of the invention can be used several matrix of sensors, the sensors of several types and several working areas of sensors in a two-dimensional or three-dimensional space, and the system may include a means of interaction between adjacent matrix sensors for transmission tracking the selected object from one working area of the sensor adjacent the working area of the sensor and record the movement of an object over time.

Using these tools, the system may be able to track moving objects of the selected type in a much larger area than the working area of one sensor. This can be used, for example, if you want to track objects throughout the Mall and in the working areas of the individual sensors, such as exhibition space.

It should be noted that because the objects are not moved to the working area of the sensor, for example furniture or other stationary objects, are not observed by the sensor of the first type, which observes only moving contrasting border or change, for example, between neighbouring the mi personnel of the television image, they are not tracked or not observed and the second sensor. This will save considerable computing power.

It should be clear that if a matrix sensor, installed on the ceiling of the building, looking for the creature down, for example in the corridor, then the object when it is directly under the sensor, will have a certain path, and when the object moves through the working zone of the sensor circuit, the observed stationary matrix sensor will vary with viewing angle. Therefore, in one preferred form of the invention the processing unit to identify moving objects in the working area of the sensor can contain a means of accounting for the angle at which the object is located relative to the matrix of the sensor, or offset of the object relative to the center within this zone.

In the system of object recognition and tracking them some or all of the matrix of sensors can be installed on the side of their working area, for example on a wall defining the boundary of the working area, or in the corners of this area.

System object recognition and tracking them in accordance with this invention can be used to count people in supermarkets or other places or for vehicles moving along the road, for surveillance and control, as well as to determine the preferences of the people during their movement through the store or exhibit space or to count the number of people in three-dimensional space such as a room or a certain area.

The system can be used for tracking, seemingly random, to determine their laws or trends over time.

The system can be used as a surveillance system with a user interface to generate human-readable image that observes the sensor, such as a television camera.

It should be noted that the matrix sensor can be installed on a movable object, such as a car, and his observations will relate to stationary objects such as road signs or stop lights. It can be used to detect the presence of pedestrians.

For a more complete understanding below is the description of the invention with reference to the attached drawings showing the preferred form of embodiment of the invention.

In the drawings:

Figure 1 shows the working area of the sensor with some objects in it.

Figure 2 shows the objects and the field observed by the sensor of the first type.

Figure 3 shows the area observed by the sensor of the second type, and find them objects.

Figure 4 shows an expanded work area sensors with multiple matrix sensors and objects moving through it.

Figure 5 shows the more realistic the picture of the objects the observed sensor of the first type in its working area.

Figure 6 shows the objects in figure 5, the observed sensor of the second type.

7 shows the diagram of the system of recognition and tracking in accordance with this invention.

On Fig shows a diagram of a detection system according to another form of the present invention.

Figure 9 shows a diagram of a detection system according to another form of the present invention.

Figure 10 shows a diagram of a detection system according to another form of the present invention.

Figure 11 shows a diagram of a detection system according to another form of the present invention.

On Fig shows a diagram of a detection system according to another form of the present invention that uses a transponder signature authentication device.

Detailed description of preferred embodiments of the invention

Refer to the drawings the device is shown in Fig.1-3, where in the working area (control area) 1 sensor has multiple objects. The first object is person 2, pushing the trolley 3 and other objects are two-man 4 and 5, moving close to each other.

As you can see in figure 2, the details of the objects detected by the sensor 7 of the first type are the only circuit 8 people the century 2, the contour 9 of the trolley 3 and the circuits 10 and 11 men 4 and 5.

Information about these contours, defining the area that you want to view in detail, is transmitted to the sensor 15 of the second type, and the observed image, such as digitized video images, is observed only in the areas shown by dashed lines. Region 16 is observed to identify the object 2, the region 17 is observed to identify the cart 3, but since the truck is not the object that is to be calculated, the processing unit of the tracking system does not accept the carriage 3 into account and not count it. Two people 4 and 5 are observed when viewing areas 18 and 19 and are identified as two people walking together.

Thus, the system made according to this invention, viewed in this it space 3 counts people moving in this space, and ignores the fourth object of the cart.

In figure 4 you can see the six working areas 20, 21, 22, 23, 24 and 25 sensors. Man 28 enters the zone 21, passes through this zone to the zone 23 and then exits from the zone 23 in its corner, which is located adjacent to area 25. Man pushing a cart, enters the area 20 of the sensor moves diagonally through it into the working area 22 of the other of the sensor and then to the zone 24.

The matrix sensor (matrix feelings is positive elements) 31 in the working area 21 of the sensor observes and conditionally notes man included in the working area of the sensor, monitors the movement of people through his work area and then passes on the matrix sensor 32 in the working area 23 the information that people came to this area in a certain place, and a matrix sensor 32 continues to monitor marked man as long as he does not leave the working area of the sensor. For continuous movement of this man's watch so that observations can be traced to determine why this person turned in the work area 23 of the sensor, and not chose to go right. When monitoring many people moving in different working areas of sensors, it is possible to observe some trends that may give some indication as to why such a action happens. However, person 29, pushing the cart 30, moved in a straight line from one working area of the sensor in the following. Transfer of function to another zone made so that man 28 counted only once, even though he was in two working zones, sensors, and human 29 counted only once, although he had been in three business areas. The cart 30 is not counted at all, because the detection devices are intended only for surveillance of moving people.

Of course, it should be clear that other devices found what I objects, you want to monitor can be moving truck or vehicle, and the people can be ignored.

Figure 5 and 6 shows a somewhat more realistic picture of where your working area 40 matrix sensor 41 of the first type observes the object 42 having an irregular shape, and another object 43, having an irregular shape. This irregularity is a consequence of the fact that the observed objects are not directly under the matrix sensor 41, and, when there are only moving a contrasting border, in each case determined by a few rough accent border (outline). The details of the boundary of the object is transmitted to the second matrix sensor 44, as shown in Fig.6, and they observed only a limited area in which at a given observation angle is determined that the moving object 42 is actually a man, and the object 43 - truck. Similarly, figure 5 two objects 47 and 48 have a few rough shape, but if their angle relative to the sensor 41, they are identified as people 49 and 50.

As can be seen in Fig.7, the matrix sensor 60 includes a sensor 61 of the first type and the sensor 62 of the second type, observing the working area 63 of the sensor. The primary signal processor 64, associated with a matrix sensor, receives and processes signals from the sensor 61 of the first type and submits to the sensor 62 is which type of command about what to observe in the working area of the sensor. Details of the relevant observations are transmitted to a Central point 65, which also receives information from the adjacent matrix of sensors on the lines 66 and 67 and allows you to track objects moving on the working zones of neighboring sensors. Relevant information, such as the results of the counting, served on other matrix sensors or devices connected by a communication network or device 68 count.

On Fig matrix sensor 70 is located on one side of the working area and covers an area of 71, which includes a working area, a four sectors 72, 73, 74 and 75. In this form of the invention, the matrix sensor 70 serves as a sensor of the first type and the second sensor type and can detect objects in several working areas represented by these four sectors 72, 73, 74 and 75.

Figure 9 matrix sensor 76 is essentially the same location as on Fig, area 71 of coverage, which includes a working area, a four sectors 72, 73, 74 and 75. In this form of the invention, the matrix sensor 70 includes a sensor 77 of the first type and a separate sensor 78 of the second type. Here, the sensor 77 of the first type can detect moving objects in multiple work areas represented by these four sectors 72, 73, 74 and 75, and the sensor 78 Deut what type can identify each object.

Figure 10 shows the location of the matrix of sensors, in which there are two matrix sensor 80 and 81, each side of the working area of sensors, consisting of four sectors 72, 73, 74 and 75. Matrix sensor 80 has an area of 83 coverage, which allows for the detection and identification of objects in the sectors 72 and 73 of the working area. Matrix sensor 81 has an area of 84 scope, which allows detection and identification of objects in the sectors 74 and 75 of the working area.

Figure 11 matrix sensors 80 and 81 are essentially the same location as in Fig. 10, with the corresponding regions 85 and 86 of coverage, but they allow you to monitor only the sectors 73 and 75, respectively, while the observation in the sectors 72 and 74 are provided corresponding matrix sensors 87 and 88 located at the top.

On Fig shows highway 90, which moved two cars 91 and 92. Cars 91 and 92 are moving along the highway in different directions. Car 91 has an integrated circuit transponder (defendant) 93 installed on it, and the car 92 has an integrated circuit transponder 93 installed on it. On the one hand from highway 90 is a matrix sensor, containing the first sensor, for example a television sensor 95, operating in the infrared region of the electromagnetic spectrum, and the second microwave sensor is 97, designed to interrogate transponders 93 and 94. Region 96 of the television coverage of the sensor 95 includes both lanes of the highway and then it can detect moving cars 91 and 92. When each vehicle is detected, is driven by a second microwave sensor 97 to interview focused beams 99 and 98 transponders 93 and 94, respectively. Through this registers not only the fact that a certain vehicle has passed a given point, but also the direction of its movement. Without the first stage of the observation matrix sensor can register that the transponder has passed a given point, but the direction of movement cannot be registered.

In this description have been given different instructions regarding the scope of this invention, but the invention is not limited to any one of them, and may consist of two or more joined together. Examples are given for illustration only and not for limitation.

In this description, unless the context indicates otherwise, the words "include", "include" and their derivatives, such as "includes" and "including"shall be understood to imply the inclusion of a whole or group, but not the exclusion of any other integer or group.

1. System object recognition and tracking them containing at least one matrix Yes the chick, moreover specified or each matrix sensor is designed to perform the functions of the sensor of the first type, providing the ability to detect presence of an object in the working area of the sensor and determine the position of the object, and the sensor of the second type, making use of this provision, the object specified by the sensor of the first type, for identification and recognition of the object, and the ability to focus or work with a higher resolution than the sensor of the first type.

2. The system under item 1, characterized in that the presence of the object defined by the sensor of the first type can be detected by changes in high contrast edges or changes in the picture image or displacements, and thus the system can detect objects that move in the working area of the sensor.

3. The system under item 1, characterized in that the sensor of the second type is made with the possibility of counting or registering the presence of the object if it is an object of some of the selected type.

4. The system under item 1, characterized in that the matrix sensor is made with the possibility of using part of the electromagnetic spectrum selected from radar, microwave, radio frequency, infrared, millimeter and optical ranges, or use of audible the location or other detection system.

5. The system under item 1, characterized in that the sensors of the first and the second type is made in the form of a single TV sensor, observed which the image is processed in two different ways, the first of which is the observed image can azithromyacin and consistent image can be analyzed to detect changes in consecutive frames of a digitized image, which indicate the presence of the object, and the second one the same digitized image in a specific area of the working zone of the sensor can then be analyzed in more detail.

6. The system under item 5, characterized in that the sensor of the second type is configured to use the previous or subsequent frames to improve detection, or result in the object of interest.

7. The system under item 1, characterized in that for the detection and identification of objects using one or more characteristics such as weight, height, shape, contrast, brightness, color, patterns, speed, heat reflectivity, the signature.

8. The system under item 1, characterized in that it further comprises a machine for tracking made with the possibility of tracking a particular object.

9. The system under item 1, characterized in that it contains several what about the matrix of sensors and several working areas of the sensors.

10. The system under item 9, characterized in that it includes a means of interaction between matrix sensors for transmission tracking the selected object from one working area in the neighboring work area.

11. The system under item 10, characterized in that the means of interaction between matrix sensors comprise means of transmitting information on the following sensors or other devices of third parties that require such information to carry out, when necessary, pre-programmed actions without human intervention.

12. The system under item 1, characterized in that it includes means for metering the angle at which the object is located relative to the matrix of the sensor, or offset of the object relative to the center within a two-dimensional or three-dimensional area.

13. The system under item 1, characterized in that the or each matrix sensor has essentially over your work area.

14. The system under item 1, characterized in that the or each matrix sensor is installed at the side of your work area.

15. The system under item 1, characterized in that it is used for counting people in supermarkets or other places or for vehicles moving along the road, for surveillance and control and to determine the laws or tendencies seemingly random movements within some of the CSOs time for example, when the movement of people and / or exhibit space.

16. The system under item 1, characterized in that it is used on a moving vehicle to detect objects that are compared with the vehicle are relatively immobile.