Method and device for identification and localization in real-time zones with relative movement in the scene and to determine the speed and direction of movement
The invention relates to computing and can be used for processing digital video surveillance system. The technical result is the extension of the class of tasks. The invention is based on the temporal processing of video signals for each position of the image elements, which consists in determining the difference between the amplitude of the signal component image in the current frame and the signal amplitude of the picture element in the previous frame, and spatial processing, which for each frame of the digital video signal in its matrix distribution. When tracking a state of wakefulness of the driver of the vehicle processes the video signal representing a sequence of images of the eyes of the driver. 3 S. and 11 C.p. f-crystals, 22 ill. The invention relates to a method and apparatus for identification and localization of areas with relative movement in the scene and to determine the speed and oriented direction of this relative movement in real time.Relative movement means moving the said zone (which may consist of "object" in the broadest sense terminate or more or less complete immobility of the specified area (or object) in the environment, which at least partially is in motion.The invention relates to processing digital video output of the surveillance system containing an optical input system (or lens) is arranged to form images of the observed scene, and optoelectronic converting system or sensor, configured to convert the received image into a digital output signal.The monitoring system consists mainly of the camera (or camescope), which observes controlled stage (and then the specified digital output signal is a digital signal generated by the camera with a digital output, or the output signal of the analog-to-digital Converter, an input connected to the output of the camera that outputs an analog video signal).The monitoring system may also include a lens, an optical device (binoculars, visirule refracting device, the viewfinder), from the output optical signal of which shall sample and using a photoelectric sensor, for example, charge-coupled device (CCD) or a complementary structure of metal-oxide-conductor (CMOS structure) with the corresponding electronic system of the GSM system in a digital video output.The invention relates mainly to the processing of the digital video signal output from the monitoring system, in particular camcorder with digital output for determining signal indicating the presence and localization of areas with relative movement in a given scene, and the speed and oriented direction of movement, if the specified area is really moving in this scene about the nearly stationary environment, in real time.The best system for the identification and localization of an object in relative motion, and to determine its speed and oriented direction of movement are the eyes of the animal or human, such as hunter, determining from his hiding place, the movement of the animal and the direction and rate of movement.The known device type observation of artificial retinas, or analog (see Giocomo Indivery, MicroNeuro'96, p.p. 15-22) or digital (Pierre-Francois Ruedi, MicroNeuro'96, p. R. 23-29). The first work we are talking about detectors and analog modules with complex structures, as in the second work - on means of identification edges of the object.In the described known devices are very bistromath is correspondingly limited information about moving areas or objects.Were asked to memorize the output signal of the camera frame or similar device, consisting of a sequence of data associated with the image elements representing the scene observed by the camera at time t0in the first two-dimensional memory, and then the video signal for the next appropriate frame representing the specified scene at time t1in the second two-dimensional memory. If the object has moved in the interval between t0and t1, there are, on the one hand, the distance d, which moved the object in the scene between t0and t1and on the other hand the time T=t1-t0between the beginnings of two consecutive frames associated with the same image elements. Then the moving speed is equal to d/T.A system of this type requires a very large total capacity, if you want to get accurate data speed and oriented direction, characterizing the move. In addition, there is some latency data retrieval speed and direction of movement; this information is not available until t1+R, where R is the time required for calculations over a period of t1-t0. These two disadvantages (think themes of this type.In addition, in the French patent 2611063, one of the authors is the author of the present invention, describes a method and apparatus for processing real-time serial data stream, comprising, in particular, from the output signal of camescope to implement data compression. According to this known patent, form a histogram of signal levels with the help of the law classification for the first sequence, remember the characteristic function of the Gaussian associated with this histogram, and produce from it the maximum and minimum levels, compare the levels of the following sequence or the second sequence with the specified signal levels for the first sequence, saved with the same time constant, the same for each picture element, generate a binary signal classification, which is characterized by the following specified sequence with reference to the law classification, generate an auxiliary signal from this binary signal representing the duration and position of the range of significant values, and finally, use the specified auxiliary signal to generate a signal, confining the range with the greatest long-term is atacandolo signal. This method and this device classification provide data compression, preserving only useful parameters in the processed stream of ordered data. In particular, this method enables the processing of the digital video signal representing a video image, for isolation and localization of at least one characteristic of at least one zone in the specified image. Thus, it is possible to classify the levels of the luminance and/or chrominance signal and to characterize and localize the object in the image.In U.S. patent 5488430 disclosed detection and estimation of moving by determining separately the horizontal and vertical changes in the image of the observed area. To detect movements from right to left or left to right in horizontal and vertical directions, respectively, using differential signals, first, by performing a logical EXCLUSIVE OR operation on the differential signals of the vertical/horizontal change and difference signals of frames, and secondly through the use of ratios of sums of signals vertical/horizontal changes and amounts differential signal frame with respect to the window CH. In this paten the areas are identical duplicate the difference in these two orthogonal directions, moreover, this difference To be determined as a function of the determined velocities.The device corresponding to this U.S. patent, determines the direction of movement along each of two orthogonal directions by applying a set of computational operations specified in columns 12 (beginning and end) and 13 (in the beginning), to differential signals, which requires a very complex (and, therefore, difficult-to-implement) electronic computing modules performing, in particular, division, multiplication and summation. In addition, more complex computational modules for the speed and oriented direction of movement (taking the square root to obtain the amplitude of the velocity and calculating the arctangent function to obtain the oriented direction), since the projections on the horizontal and vertical axes. And finally, in U.S. patent 5488430 no funds smoothing values of image elements using a time constant that is variable for each picture element, to compensate for overly rapid changes of these values.In contrast, the method corresponding to the invention is carried out by use of the digital type, yay capacity, with the help of which you can quickly get the required information with a wide range of results and applications (respectively full polyisobutenyl for frames interleaved or full image, depending on the specific conditions of use).In the work of Alberto Tomita (Rokuya Ishii "Hand Shape Extraction from a Sequence of Digitized Gray-Scale Images"), JEEE, vol. 3, 1994, p. R. 1925-1930, the described method of motion detection by subtracting consecutive images with the subsequent formation of histograms based on the human hand, to highlight the shape of the human hand in the scene, converted into digital form. Analysis of histograms based on the range of tones inherent in the human hand. It does not use any means of histograms in the coordinate plane. Essentially in this work solves the problem of detecting movements of the human hand, for example, to replace the conventional manipulator "mouse" hand movements which are identified for data entry into the computer.In contrast, the present invention is not limited to detection of hand movement, and allows detection of the relative movement of any object (in the broadest sense of the term) in skretny digital variables, representing the movement (if any), and the histogram in the flat coordinates.In accordance with the present invention: - process the digital video output of the surveillance system, and this signal is composed in a known manner from a sequence of frames (corresponding to polyisoprene in the case of two alternating frames on the image or full image of one frame in the image), each of which consists of a given number of consecutive rows and a given number of picture elements or pixels of the image in each of these rows, to obtain using a relatively small memory capacity signal, providing an indication of areas where the relative movement in the monitored scene, and, in the presence of such a zone, to determine the location, the speed and oriented direction of this zone, if it really moves relative to its environment, by generating two digital signals, one of which is a characteristic of a significant change or absence of change of the signal of the picture element for the location of the same picture element respectively to the two successive frames, and other Avenia these two signals for picture elements in a portion of the frame at the same time.The present invention is to provide a method of identification and localization in real-time zones with relative movement in the scene observed by the monitoring system with an output signal consisting of a digital video signal of a type that contains a sequence of the respective frames, each of which consists of a sequence of rows, each of which consists of a sequence of image elements, and to determine the speed and oriented direction of movement, and this method differs in that it consists in performing the following sequence of operations on the digital video output: - processing smoothing the specified digital video output using a digital time constant, a numerical value which can be changed independently for each picture element of the specified output signal - storing, on the one hand, a frame of the specified output signal after smoothing and, on the other hand, the time constant of the smoothing associated with the specified frame - time processing for each position of the picture element, which consists in determining, first, whether, as in the directly previous smoothed and stored by the frame, and in generating two digital signals, the first signal is a binary, or bit, a signal with two possible values, one of which is the presence, and the other is the lack of meaningful changes between two successive frames, and the value of the specified binary signal changes the stored value specified time constant to reduce it, if the specified signal is a meaningful change, and to increase it if the specified signal is not such a change, with an increase or reduction is carried out quantitatively, while the second digital signal, i.e. the signal amplitude, is mnogopudovymi signal with a limited number of bits, quantitatively determining the amplitude of this change; and - spatial processing, comprising the following steps for each frame of the digital video output: distribute only the values of the elements of the image in the frame at a given point of observation (the part that scans for the specified matrix for the entire frame duration), first, the specified binary signal, and secondly the specified digital signal amplitude, in a matrix with number of rows and columns, NGOs, to characterize the values of the picture elements are defined in this instant double matrix representation of the particular zone in which the specified binary signal has a desired value, representing the presence or absence of significant changes, and the specified digital signal amplitude changes or does not change to a significant value for adjacent picture elements in the matrix along the oriented direction, starting with the original image element, in the same part of the frame and, consequently, in the same moment of observation, and
generate signals representing the presence and position of the zone relative displacement and the relative frame-to-frame speed and oriented direction of this move, if it is, relative to its environment, based on the instantaneous matrix of the distribution of these two digital signals is a binary signal and signal amplitude.The method corresponding to the invention is preferably characterized in that it also includes:
forming first, histograms of the values of the signals distributed in the matrix, and secondly, the histogram of the slopes of the two coordinate axes with adjustable slope in the plane,
identification IU, and
for each identified region, establishing the presence of a zone of relative movement and, if available, its location, speed and oriented direction.In a specific implementation options:
- this matrix is a square matrix with the same odd number of rows and columns (2l+1), and consider the sub-matrix containing 3x3, 5x5, 7x7,... (2l+1)x(2l+1) elements, centered on the center of this square matrix to determine the smallest sub-matrix, in which the specified digital signal changes along the oriented direction, starting from the specified center, and the value of the specified binary signal indicates that exceeded a threshold limit along this direction,
this matrix is hexagonal matrix, and consider nested hexagonal matrices of increasing size, centered on the center of the hexagonal matrix, to determine the smallest sub-matrix, in which the specified digital signal changes along the oriented direction,
this matrix is inverted L-shaped matrix with one row and one column, and consider the sub-matrix containing the Oia, for one row and one column to determine the smallest sub-matrix, in which the specified digital signal changes along the oriented direction, namely, the line with the greatest slope and constant quantification.Specified time constant is mainly present in the form 2pwhere R is a number less than 16, and, therefore, can be expressed in no more than 4 bits, and this time constant can be reduced or increased by subtracting or adding one unit to the R.In this case, if necessary, consider successively smaller part of the full frame, using the algorithm temporal scale Mallet (Mallat), and selects the largest of these parts, for which data from the displacement, velocity and orientation compatible with the value of the R.Another object of the invention is a device for identification and localization in real-time zones with relative movement in the scene observed by the monitoring system with an output signal consisting of a digital video signal comprising a sequence of the respective frames, consecutive lines each corresponding frame and a successive image elements in each of the sliding way moreover, the specified device receives the video output signal as an input signal and characterized in that it comprises:
a tool for smoothing a digital video output using a digital time constant, the numerical value of which may vary independently for each of the picture elements of the output signal;
means for storing, in the first frame of the output signal after smoothing and, secondly, the time constant of the smoothing associated with the specified frame,
module time processing for analysis of changes over time of the amplitude signal of the picture element, to position one and the same element of the image between the current frame and the immediately preceding smoothed and stored by the frame of the digital signal, while the specified module contains a storage device which has a capability of receiving, storing and searching information on relevant previous smoothed frame, together with a comparison tool to determine, exceeds or does not exceed the absolute value of the difference between the current signal of the picture element and a characteristic value of the signal of the picture element to the provisions of the same picture element is the position in the frame of the input video signal, by generating binary, or bit, of a signal with two values, one of these two values shows that the threshold is exceeded, and the other shows that he is not exceeded, and the computing means is arranged to determine a multi-bit digital signal amplitude with a small number of bits whose value depends on the amplitude of changes in the value of one and the same element of the image between the current frame and the immediately preceding, smoothed and stored by the frame of the digital video signal; and
module spatial processing, in which the inputted outputs the specified serial binary signal and the digital signal amplitude to picture elements in a given frame, generated by the module time processing, while the specified module spatial processing provides a means to characterize the amplitude values of the image elements, and this tool distributes only the binary signal and the digital signal amplitude associated with one and the same moment, i.e., one frame in a matrix with number of rows and columns is less than the number of rows and the number of picture elements in the row, respectively, in the frame of the specified digital videosignal for determining the area of the image elements within said matrix, which at the moment the binary signal has a desired value, and means for determining the area of the image elements in the given matrix, in which at the same time, the digital signal amplitude to the neighboring picture elements is changed to a meaningful value, and means which, in response to the results of the two previous funds, generates signals representing the area of the image elements, and, consequently, the presence and location of zones with relative movement within the observed scene and the relative frame-to-frame speed and oriented direction of this zone, when it really moves relative to its environment.The spatial processing module preferably includes an output for issuing the delayed digital video signal consisting of an input digital video signal with a delay equal to the duration of the rows of the matrix minus the duration of a single number, to ensure the delivery of the output signal simultaneously with the analysis of the matrix in a specified unit time of processing.Module spatial processing device corresponding to the invention preferably comprises cascade connected delay means, each of United funds delay for each row, each of which introduces a delay equal to the time difference between two successive picture elements in the row, and the outputs of all registers and input in the first registers in each row give the specified torque values specified binary signal and the digital signal amplitude at the same time in the specified identification tool.The device corresponding to the invention preferably also includes means for forming a histogram of output values from the module spatial processing and histogram of the slopes of the two having a variable inclination of the coordinate axes in the plane tool to identify areas on each histogram, in which there is a significant change in the processed value to confirm this area at its output and to determine for all histograms of the output signals that identify and localize the area of the relative movement in the observed scene, if it is, and the speed and oriented direction of this movement, if the specified area really moves relative to its environment.If you want to detect movement of an object in practically stationary environment, it is necessary on the digital signal amplitude changes by a significant amount between adjacent picture elements in the frame.On the contrary, if you want to detect a stationary object in the environment that really moves, you need to identify the part of the matrix in which the value of the binary signal indicates that the limit is not exceeded, and at the same time, the digital signal amplitude does not change between adjacent picture elements in the frame.Preferably, the device for identifying, locating and determining the speed and oriented direction of movement of the zones with relative movement in a scene that implements the aforementioned method:
- specified smoothing tool includes an input for receiving a digital video signal and provides the production for each subsequent picture element in the frame of the specified signal smoothed signal in which the temporal change of the input digital video signal is reduced by using the threshold signal on the other input, and the time constant associated with the position of each picture element in the frame, the value of which varies consistently, so the anti-aliasing support, although with the decline, the trend in the changes of the incoming digital video signal, with this tool the smooth operation of the values of the smoothed signal and the time constant for each of the elements of the image in the frame and gives the position of each picture element, at least, the sequence of values updated time constant and the values of the binary signal, which indicates that exceeded the specified threshold the absolute value of the difference between the value of the item image and its smoothed value; however, the specified node performs spatial processing matrix distribution along the rows and columns in the reduced number of output signals from the means of smoothing, namely the successive values of the time constant and the binary signal;
- provides a tool to identify when the specified matrix distribution area of the image elements, in which at the same time:
either the value of the binary signal corresponds to the exceeded threshold, and the time constant is changed between adjacent picture elements to a meaningful value in the same direction and at the time of receiving output signals indicating the location of the specified area and the speed and oriented direction of movement in this area;
either the value of the binary signal corresponds to neprevyshenie limit, and the specified time constant does not change between adjacent picture elements and at the time of receiving the output signals indicate the first means of smoothing includes, in combination with the storage device for storing video data or storage device with the sample magnetic field, which stores successive values for each picture element in the frame specified time constant and smooth digital video signal, means for calculating, for each picture element of the absolute value of the difference between the value of the digital video output of the camera and the value of the previously smoothed digital video signal, means for comparing the specified difference threshold and to generate a binary signal, one of two values, which indicates that the specified limit is exceeded, and the other value indicates that it is not exceeded; means for updating the time constant, the host immediately previous value of the time constant of the specified storage device and decrease it if it takes a binary signal, indicating that the limit has been exceeded, or increases it, if the value of the binary signal indicates that the limit is not exceeded, the reduction or increase, however, is not carried out if it may lead, respectively, to the negative value or a value exceeding the threshold value is the previous value of the smoothed signal, received from the storage device, the quotient of the difference between the digital video output of the camera and the previous smoothed digital video signal from the storage device by a factor equal to the value of the previous time constant obtained from the storage device;
- this time constant is presented in the form 2pwhere R is a whole number less than 16, and, therefore, can provide no more than four bits, and the increase or decrease of this time constant can be done by subtracting or adding one unit to p;
- the tool matrix distribution includes, in combination with means to delay introducing sequential delay equal to the duration of one line of the video signal into a digital video signal from the output of the camera, for delivery to the serial outputs this video signal is delayed by one delay period, two periods of delay, and so forth up to the number of periods of delay, equal to the number of rows in the matrix of the spatial distribution minus one tool for the implementation of the matrix distribution along the respective rows of the matrix, receiving, first, the digital video signal from the camera without the constant time and the specified binary signal, taken from a means of smoothing in order to obtain a matrix distribution in the rows and columns of the indicated values in the moment, time constant and a binary signal for the image elements in the portion of the frame of the digital video signal with the same size as the matrix;
the tool matrix distribution contains a group of conductors of the digital signal, one for each row in the matrix distribution, each with a cascade connected shift registers, each of which introduces a delay equal to the time difference between two successive picture elements in the line digital video signal, while the position of the picture element, distributed in the matrix, is determined by the point in the matrix in front of the register latency, which number in the string is equal to the number of columns in the matrix minus one, and the point below the register, the latter in the direction of the traversal;
- the specified detecting a moving area in the specified matrix by detecting the simultaneous existence of the value of the binary signal, indicating that the limit has been exceeded, and by changes in the value of time constant includes means for determining along discrete convert the om image element in the center of said matrix, forming a starting point for these directions, and means for selecting the largest of the slope of this change near this center, the starting point, and determining the oriented direction with regard to the selection criteria to select the direction, if there is more than one direction with the same maximum slope of this change, and this latest tool generates signals characterizing the speed and oriented direction of displacement in the moving area, with confirmation signal indicating that these signals speed and direction is valid, together with the value of the time constant.Preferably, the means for forming a histogram, comprising the device according to the invention, includes:
inputs for receiving the signal from which is formed the histogram, and the confirmation signal from the matrix distribution, and
means for defining two linear one-dimensional histograms for two flat coordinates for the unification of these two linear histograms in the surface histogram describing a zone in which there is a significant change in the input signal, and
the output signal representing the specified area.In addition to the, at the input of which the signal sequence of rows, the signal sequence of the columns and the synchronization signal of the picture element, and the output signal of which represents the reference change,
two tools to generate histograms for the two axes for receiving the two reference signals and generate histograms of these axes, and
tool zone, in which is entered the output signals of the two means of histograms for the two axes, and which produces a signal that contains global information about the inclination of the two axes.In some applications, the specified time constant may consist of a serial number of intervals in limited quantities and with a gradually increasing value in the absolute value of the difference between the value of the current picture element and the immediately preceding value of the same element of the image after smoothing, the position of each picture element.To determine the binary signal in the presence or absence of a threshold value, the absolute value of the difference between the value of the current input picture element and directly previous smoothed value of the same picture element of the C image.The digital signal amplitude, it is preferably formed in the form of an integer representing a convergence trend values of the current picture element directly with the previous smoothed value of the picture element, the position of each picture element.One of the characteristics of the invention lies in the fact that to determine the binary signal and the digital signal amplitude as the value immediately preceding the picture element to use its saved smoothed value to reduce any excessive change of the signal of the picture element in time that may exist in the digital input video signal from a camcorder or any other device monitoring with digital output.It is known that the effect of the smoothing operation is the gradual replacement of the digital signal with large amplitude changes in time of the signal with smaller changes and, therefore, quantumm fewer steps and, consequently, of bits, to obtain the smoothed digital signal amplitude.The following describes the preferred implementation of the device corresponding to the invention, and implement estoya the invention, with its inputs and outputs and the input signal for this system.Fig. 2 is a view of the functional blocks of the core modules in the device corresponding to the invention, forming the site of temporal and spatial processing.Fig. 3 and 4 is a functional diagram of the temporal processing node computing node spatial processing matrix of the distribution, respectively, forming part of the device shown in Fig.2.Fig.5 is a schematic representation of temporal processing and spatial processing in the system corresponding to the invention.Fig. 6 is a numeric value corresponding to the code of Freeman (Freeman), discrete directions, starting from the original centre in the matrix shown in Fig.4.Fig. 7 - two small sub-matrix within a matrix of a temporary allocation.Fig. 8 and 9, a matrix of the other two types, namely, hexagonal, and inverted L-shaped matrix, respectively.Fig. 10 is a schematic representation of the node device shown in Fig. 2, along the line Z-Z1with an additional node in accordance with a preferred embodiment of the invention.Fig. 11 is a schematic representation of an additional node, shown in Fig.10, in the form of BLG.2 and 10.Fig.12 is an illustration of the formation of two one-dimensional histograms and on the basis of these two-dimensional histograms histograms moving area of the input signal.Fig. 13 is a more detailed image forming unit histograms and processing associated with block linear Association.Fig.14 is a one - dimensional histogram.Fig.15 and 16 is an illustration of the changes in the observed tilt of the scene.Fig. 17, 18 and 19 illustrate other possible applications of the device corresponding to the invention, and Fig. 17 relates to the teleconference, Fig.18 - to the operational control of highways, freeways and major roads, and Fig.19 - to the operator running the machine hand motions.Fig. 20 and 21 is a schematic representation of application of the invention to operational monitoring of the sleeping-car driver.Fig.22 - transformed image corresponding to the scheme Mallet.According Fig.1, in a preferred embodiment, the device 11 corresponding to the invention, contains the input 12, which receives the digital video signal S from a video camera or camescope 13 with one or more CCD sensors, CMOS type with direct digital output or analog output converted analog of preobrazitb, for example - TR1and TR'1TR2and TR'2, each of which consists of a sequence of horizontal scan lines, with each line (for example, l1.1, l1.2, ... l1.17in TR1and l2.1in TR2) consists of a sequence of elementary signals of picture elements, or pixels PI, representing the point (for example a1.1, a1.2and a1.3for a line l1.1;2.1and2.2for a line l1.2; al17.1and a17.2for a line l1.17; al1.1and a1.2for a line l2.1) scenes 13A controlled by the camera 13. The drawing shows the signal S(PI) representing the signal S consisting of elements of the image PI.The signal S(PI) contains signals (ST) synchronization frame scan at the beginning of each frame and signals (SL) synchronization horizontal at the beginning of each line.You may notice that the signal S(PI) consists of:
sequences sequences (consecutive frames) within the framework of some temporary area and
in each sequence (in each frame) - from a number of suppositionally (rows of picture elements) within the framework of a certain spatial region.In temporarily the aqueous species (e.g., such odd frames as TR1or even frames such as TR'1in such pairs of frames as TR1-TR'1forming a sequence of images of the digital video signal S(PI), and the expression "sequential picture elements in the same position" will denote the successive values of the image elements (PI) in the same location in consecutive frames of the same type, for example a1.1line l1.1in the frame TR1and a1.1line l2.1in the following the corresponding frame TR2.In addition, the device 11 also includes outputs 14, issuing a number produced by them to digital signals, which are used to indicate the presence of a zone or "object" (described above in a more General sense), making the relative movement, and its location, along with its speed and direction of movement, if it is significant, indeed stationary environment, in particular, issuing a complex signal ZH, conditionally grouping signals indicating the presence and localization of this area or object, the velocity V and oriented direction DI move and possibly the input digital video signal S delayed to synchronize it with the RP is the position of the image, perceived by the camera 13 on the monitor or television screen 10 when there is information relating to the zone with relative movement (if there is one), namely the signal ZH (V, DI), which can be used in the node 10A processing and testing.With reference to Fig.2 below describes the design of the first part of the device 11 shown in Fig.1, and this part presents the block 11a in Fig.2.The main components of the node 11a are, first, the module 15 time processing with an associated storage device 16, and secondly, the module 17 spatial processing related delay module 18 and a regulating module 19, and a clock generator 20 of the image elements, setting the rhythm for module 15 time processing and ordering module 19.Module 15 temporal processing, which, among other functions, smoothes video:
- uses a digital video signal S from the camera 13, consisting of a sequence of values PI of the image elements, and impulses HP generated by the clock generator 20 (starting from S) with the frequency of the picture elements in the frame (in particular, 13.5 MHz), to generate a number of signals, as a description of the NGO - smoothed values L digital video signal and values With a time constant of smoothing, and the values of L and C follows the letter " O " for the values stored in the storage device 16 of the module 15, or the letter I for the values retrieved from the storage device 16 and supplied to the module 15, and
- generates a binary output signal DP setting, exceeded the limit, and the digital signal indicating the updated calculated value of the time constant, namely the value WITH reports sent to the storage device 16.Blocks calculations and/or comparisons, forming module 15 time processing, shown in detail in Fig.3; module 15 contains four blocks 15A, 15b, 15C, 15d.The first block 15A module 15 using:
- input digital video signal S consisting of the sequence of signals PI image elements, and
- smoothed value LI of the signal S for the immediately preceding corresponding frame calculated in advance by the module 15 as L and temporarily stored in the storage device 16 (as explained below),
calculates the absolute value of AV difference between the input values of PI and LI to the provisions of the same picture element (for example, a1.1line l1 is nerator 20 added.The second block 15b is testing unit:
- this unit adopts the above-mentioned digital signal AV from the block 15A and the digital threshold signal SE, which should be constant, but that usually depends on the value of the picture element, so it is changing in the same sense as the specified value, so that creates a gamma correction (known tool change SE to implement gamma correction presents additional block of 15th shown in dotted lines);
he compares the two digital signal representing AB and SE to determine a binary signal DP, in other words, the signal, which can be equal to one of two values 1 and 0, to indicate, respectively, the AV exceeds or does not exceed a specified threshold SE:
if AB is greater than SE, the signal DP is assigned a value of 1 in block 15b to show that the threshold is exceeded;
if AB is less than or equal SE, the signal DP is assigned a value of 0 in block 15b, to show that the threshold is not exceeded.In fact, when DP= 1, the difference between PI and LI, in other words, between the input digital video signal and the previous smoothed digital video signal is too large, and this difference must be reduced by the mind>/p>The third block 15C makes the desired change in the value of the time constant depending on the values of DP:
- if DP=1, block 15C reduces the time constant at the value U of the unit: (new value of this constant)=CI (old value constant)-U;
- if DP=0, block 15C increases the time constant at the same value of U units: CO-CI+U.Therefore, block 15C receives the above-mentioned binary signal DP from the block 15b on one input and the signal CI, which is the value of the previous time constant stored in the storage device 16, the other input of and increases or decreases the U-value unit input time constant CI, which becomes the value of the WITH sent in the specified storage device 16 instead of CI.The time constant, which determines the convergence of the smoothing (as a function of time required to achieve a smoothed value of the input values of the digital video signal), mainly present in the form of a power of two, namely the values of 2pand it is an integer, p is a number that will decrease or increase by one unit, in other words - 1, in block 15C; in this case U=1 for p in Fig.3,
- if DP= 1, block 15C subtracts edinica 15C adds one (1) to the exponent p in the time constant of the 2pwhich is equal to 2p+1.The choice of the time constant of type 2phas two advantages, first, it corresponds to the physiology of the organs of vision, and secondly, it facilitates the calculation and thus simplifies the design of the unit 15C.Block 15C must also satisfy two conditions, namely, must be maintained between the following two values: must not become negative (WITH0) and it should not exceed the limit of N (WITHN). In the special case where CI and CO are presented in the form 2pthe upper limit N is an integer n which is the maximum value for the R.The upper limit of N (or n) can be either constant or variable. If it is AC, then an additional block 15f (shown in dashed lines) does this change the N (or n), for example in the order specified by the user. The sequence of increasing N is to increase the sensitivity of detection of movement, whereas the decrease of N improves the detection of high speeds.And finally, the fourth block 15d receives on the first input value WITH the new time constant generated by the unit 15C, and the second Ode - the smoothed value of the previous incoming digital video signal, namely LI, from the storage device 16, and this block calculates the value
served on his way out.In fact, component (PI-LI) represents the change in the smoothed value of the digital video signal, taking into account the changed values WITH a time constant proportional to the algebraic difference between the current value of the current picture element PI coming from the camera 13, and his previous smoothing value LI and inversely proportional WITH.If=2pthen
using p0the p value, calculated in block 15C, which replaces the previous value of pifor p in storage device 16.Therefore, the module 15 time processing with four blocks 15A, 15b, 15C, 15d:
- accepts S(PI) of the video camera 13, the synchronizing pulse HP to control the frequency of operations and the signals SE and N (or n) threshold values;
starting with the incoming signals LI and CI from the associated storage device 16 determines the updated signals L and sent to the specified storage device to replace the LI and CI, respectively, and that the public, and
- delivers the above-mentioned signal WITH the binary signal DP threshold, calculated on the basis of PI, LI and SE, module 17 spatial processing through the delay module 18.The purpose of the smoothing operation is to normalize the change of the digital value of the input video signal for each picture element or pixel in the image, namely the change of each PI by minimizing the differences of the changes and replacement for each picture element of the serial PI values of the real variable in this point of the image smoothed values L that change less than PI.Thus, for each incoming PI module 15 temporal processing in conjunction with the storage device 16 replaces the smoothed value L reduced changes by using binary signal DP of exceeding or not exceeding the threshold, and the signal FROM the time constant, which is updated and transmitted to the spatial processing module shown in Fig.4.Each element of the image inside the area frame can be identified by two coordinates (usually orthogonal) on the abscissa and the ordinate, namely x and y, giving two index element images, and online images with indices i and j has videomachine (amplitude signal) PIij.Now if we consider the variation with time t of the values PIijfor successive respective frames in the moments of t0, t1, t2, t3,..., separated by a period T corresponding to the period of the image (usually equal to two frame periods), which may be 0.04, if the frequency of the video signal is 25 Hz, or 0,0333 with, if the frequency of the video signal is 33 Hz, or 50 Hz for sensors with incremental image (1 frame 1 image), signal element video index localization of i and j is consistent values, denoted as PIijt0PIijt1PIijt2PIijt3... in these moments of t0, t1, t2, t3...Within the scope of the invention PIijreplaced by successive smoothing values Lijtnamely , Lijt0, Lijt1, Lijt2, Lijt3module 15 time processing.For each of the successive image elements or image point with coordinates i, j in time t, in other words - PIijtthe actual value of PIijtis replaced by the smoothed value defined by the formula
Lijt=LIijt(t+1)+(PIijt-LIij(t-l)1from t0, t2from t1, t3from t2and so on , ensures the convergence values Lijtto the value of PIijtwith speed-dependent time constant, which varies in space (and therefore depends on i and j) and in time (and therefore depends on t) and which can be written in the form of COijt.In all cases, the convergence Lijtslower when COijtmore. If COijt=1, no smoothing no more.It is obvious that in the above formula could be replaced by Cartesian coordinates x and y indices i and j that specifies the ordinal number of rows and the number of the picture element in the string.Unit 15A calculates AB = |PI-LI|, while the indices i, j and t are not defined explicitly, and the magnitude of the AB is instant changeability digital video signal S(PI) relative to the smoothed signal LI for each triplet i, j, t.One of the characteristics of the device corresponding to the invention, consists in the exercise in module 15 time processing smoothing values of image elements to determine for each image element sequence values of permanent frosty between the value of the item image and the smoothed value of the picture element for two successive respective frames, to distribute in a flat matrix of numerical values in the same moment, time constant and a binary signal for the image elements in a limited part of the frame, and this part scans the frame to localize based on local changes of these two values in this time zone with relative movement and to determine the speed and (oriented) direction of the actual displacement as a function of this distribution, and this is done by the module spatial processing described below with reference to Fig.4.The convergence of the smoothing is provided by computing blocks 15b and 15C, which determine the change in the value of the new incoming time constant (in fact, COijtto accelerate convergence. This is done by comparing AB (actually, ABijt) with the threshold value SE, which may be constant or, preferably, is a variable, and if it is variable, it may be a function of the values of the picture element to perform gamma correction: if the value of the picture element increases, the limit is increased, and Vice versa. The result of the comparison of AV with the threshold value SE (in fact, SEijt in the optional computing unit 15th) provides generating unit 15b binary signal DP move with two possible values 1 and 0, as explained above.The binary signal DP is supplied in unit 15C to change the value of the time constant. Consequently, the computing unit 15C also takes the value CI of the incoming time constant of the storage device 16, and updates it using the new value of CI time constant, which (among other things) is sent to the storage device 16, where it replaces the previous value CI; in fact, for two values of CI and CO are used: CIijand COijin two consecutive moments, such as t0and t1separated by an interval T between two successive respective frames (either odd or even).Unit 15C, which receives signals DP and CI, adds or subtracts the value (U) units to the value of the time constant CI or one unit to R, when presented in the form ofp, depending on whether the binary signal DP, which threshold value is exceeded (DP=1) or not exceeded (DP=0).If the threshold is exceeded, the value of this constant is too large, and it is reduced, and Vice versa.Block 15C also ensures that the new value WITH the time constant obtained from CI by adding or subtracting units supported between 0 (WITH Neot is that the value of CI (which actually lies in the interval from 0 to N, including both of these limit), and then WITH a=CI.The upper boundary value of N (or n) can be constant or variable; if it is variable, it should not exceed a limit value N max (n max); changes, if any, to be paid by the unit 15f under the action of the control available to the user.Alternatively, N or n can be made dependent on PI (where N, n and PI is actually assigned to the indices i, j, t) to ensure that the regulation changes LO (which is computed in block 15d) as a function of the level of PI that can be expressed as follows: "Nijtor nijtis a function of PIijtmoreover, the definition of Nijtor nijt=F(PIijt) is carried out in the computing unit, replaced by shows a block 15f, in which, in addition to N, the video camera 13 takes the value of PI.Time constant (in fact, every Cijt) can mainly be set to satisfy the condition of equal numerical value or a multiple of two, or equal to powers of two, regardless of the values of i, j and t; in this particular case, Cijt=2p(ijt)where R is a small integer, which is a function of i, j and t and can be represented by a small number of digits. This condition obsolescence the laws of the organs of vision of the person;
- electronic circuit units 15C and 15d is simplified; in particular, in block 15d, which is designed to determine changes in the value of time constant for each pair i, j by the following formula
calculations are simplified, if presented in the form 2p(where p is a small integer), and the value of the threshold value n, the number p is a small integer that is represented by a limited number of bits.In all cases, the new value of the smoothing LO incoming digital video signal S is supplied in a storage device 16, where it replaces LI (for each pair of indices U).According Fig. 2 and 3, the module 15 temporal processing, which includes computing units 15A, 15b, 15C and 15d and possibly block of 15th and/or block 15f, and which communicates with the storage device 16 determines and provides as output signals, as explained above, the following values for each triple of indices i, j, t:
firstly, updated smoothed value of LO, which is passed to the storage device 16 to replace the previous smoothed value LI;
- second, two digital signal, namely:
the binary signal DP, which indicates that the specified threshold or exceeded (DP= 1) or not exceeded (DP=0), paradidomi smoothed signal of the picture element to the same point; and
digital signal amplitude, consisting of values WITH the updated time constant;
these digital signals DP and accepted through the delay module 18 module 17 spatial processing, which will be described later, and the signal is also entered in the storage device 16, where it replaces the value WITH the previous value of CI for the same picture element.Consequently, it is possible to notice that the capacity of the storage device 16 for storing, firstly, successive values of the smoothed signal of the picture element, and secondly, the time constant, assuming that the frame has R elements of the image, and therefore 2R picture elements on the full image should be at least 2R(e+f) bits, where e and f - the number of bits allotted to the signal of the picture element and time constant, respectively. In fact, the storage capacity should not be much larger; it just has to exceed 2R(e+f) by the number of bits required for the proper operation of the storage device, in particular for addressing and retrieving bits from the smoothed signals of picture elements and bits time constant as a function of the indices i and j. If each movie is j and COijcoming out of the node 15 temporal processing at time t, evaluated and used in node spatial processing shown in Fig.4, and the node that is the Assembly shown in Fig.3 and 4, is shown in Fig.2.In fact, the node 15 time processing processes the signals of the frames, while the node 17 spatial processing shown in Fig.4, processes suppositionally lines and picture elements in the frame.Fig.5 schematically depicts a temporary sequential processing of the corresponding human sequences TR1TR2TR3and spatial processing in the first of these frames, in other words - TR1illustrating the Cartesian coordinates x and y, and the picture element PI with coordinates y, x, in other words, with the indices i, j at time t; successive image elements with the same indices i, j in the three frames TR1TR2TR3indicated by the indexes ijt1, ijt2, ijt3, respectively, and have values PIijt1, PIijt2, PIijt3picture elements, respectively. The plane of Fig.5 corresponds to the spatial processing of the frame, whereas the superposition of frames corresponds to temporal processing (DG is which is also shown in Fig.4), interacts with the control module 19, controlled by the clock generator 20, which generates the sync pulse HP on each successive signal of the picture element (see Fig.2 for illustration of the node).The signals DPijand COijcoming out of the module 15 time processing, distributed node 17 in the matrix 21 is reduced in size, containing a number of rows and columns is less than the number L of rows and the number M of picture elements in the row, respectively, DPijand COijat a given time t. In particular, the matrix may include 2l+1 rows along the x-axis and 2m+1 columns along the y-axis (in Cartesian coordinates), where l and m are small integers. Mainly, l, and m is chosen equal to powers of two, where, for example, l is equal to 2aand m is equal to 2band a and b are integers, for example from 2 to 5. To simplify the drawing and explanation, we assume m is equal to l as an example (although it may be different), and m=l=23=8; in this case, the matrix 21 will be 2x8+1=17 rows and 17 columns.In Fig. 4 shows part of the 17 rows of Y0, Y1The...Y15, Y16and part of 17 columns X0X1,..., X15X16forming a matrix 21 of node 17.The task is to distribute in mA who sub> and COijtin other words - the binary signals DP threshold and digital signals amplitude representing the time constant, which comes from the module 15 time processing, providing them with a wide matrix distribution for the frame consisting of L (in particular - 312,5) rows and M picture elements in the row, in particular about 250-800, depending on the standard television signals.To distinguish the two matrices, i.e., the matrix video size L and matrix size lm module 17, which is assigned to the position 21, will be used indices i and j for the two coordinates of the first matrix (which will be visible only when displaying the digital video signal on a television screen or monitor) and the indices x and y for the two coordinates of the last matrix (shown in Fig. 4); at a given point in time, the picture element of an instant value of PIijtcharacterized by the input module 17 spatial processing two digital signals DPijtand COijt. Matrix L these two signals is moved during scanning, a much smaller matrix 21 size (2l+1)x(2m+1), as explained with reference to Fig.4, the matrix 21 materializes (2l+1)x(2m+1) elements in the picture, match the DN of 0 and 16 (inclusive) for series Y0-Y16, respectively, and the ordinal column number between 0 and 16 (inclusive) for the columns of X0-X16accordingly, in the case when l=m=8, namely 2l+1=2m+1=17. In this case, the matrix 21 will create a view h=289 image elements in the plane, whereas the matrix video will include several tens or hundreds of thousands of picture elements, or even more.In Fig. 4 extra long horizontal rectangles Y0-Y16(of which presents only four named Y0, Y1, Y15and Y16and the vertical lines X0-X16(of which presents only four named X0X1X15and X16) used to illustrate this matrix 21 (site 17) with h of the pixels or image elements with indexes defined in the intersection of row coordinates and column of the abscissa. For example, the position of the P88the picture element is located at the intersection of column 8 and row 8, as shown in the drawing at the pointthat is the center of the matrix 21.To undertake a spatial distribution of the parts of this matrix size LM within this matrix 21 RA is ivory indexes ijt), secondly, the incoming signal S of picture element, in other words - PI (with assigned indices ijt), and the signal HP from the generator 20 clock pulses and signals SL sequence of lines and signals SC sequence of the columns (Fig.2 and 4).As shown in Fig.1, the signal S(PI) includes not only the signal values of image elements, for example - a1.1, a1.2comprising a temporal sequence of consecutive frames) and spatial sequence (picture elements in the line in each frame), but the signals ST, SL, based on which the clock generator 20 sets the synchronization signal, for example at a frequency of 13.5 MHz, namely one marker pulse for each picture element in the frame and the blanking signals BL, translating module 19 in the off state during the above-mentioned signals.In response to these signals HP and BL from the generator 20 signals (Fig.2) module 19 speed control produces a signal SL sequence of lines with a frequency equal to the private from division 13.5 MHz on the number of columns in the frame (such as 400), together with the signal SC of a sequence of frames, the frequency of which is equal to the above-mentioned astrosuit these signals SL and SC and HP synchronization signal to control line spatial distribution in the matrix 21.As a result of this successive rows of Y0-Y16receive signals DP and WITH:
- nezagarionnyje (the number of Y0);
is delayed by one period TR, is equal to the length of lines in the frame (the number of Y1);
- detained on 2 TR (number of Y2); and so on up to
- detained on 16 TR (number of Y16).Serial delay having a duration of several frames, namely, TR, implemented in a cascade connection of sixteen delay circuits r1, r2,... r16that cater to the rows of Y1, Y2The... Y16and the number of Y0served directly by the signals DP and received without delay from node 15.All schema r1, r2,... r16can be built on a single delay line with sixteen outputs, the delay imposed by any section of this line between two consecutive outputs is constant and equal to TR.Module 19 speed control controls the scanning of the whole matrix frames LM the matrix 21 size (2l+1)x(2m+1), while turning to a sequential series of consecutive frames using signals SL sequence alignments using the following procedure.A cyclic shift of the picture elements in im connection sixteen shift registers d in each of the 17 rows of Y0to Y16(which gives only h=272 of the shift register) is placed in each row between two successive positions of the image elements, namely the case d01between positions PI00and PI01the case d02between positions PI01and P102and so on, Each register introduces a delay TS, is equal to the difference in time between two successive image elements in a row or line signal SC sequence of columns.Note that due to the fact that the rows of l1, l2...l17in the frame TR1(Fig.1) to S(PI) and for DP and reach of module 18 shifted TR (full duration of the series one after the other, and because the module 18 distributes them with gradually increasing delay value TR in rows of Y0, Y1... Y17these rows show the signals DP and at a given time series l1, l2... l17in the same part of the frame.Similarly, in a given row, e.g. - l1consecutive signals a1.1, a1.2. . . image elements come close together on TS, and the shift registers d impose a delay equal to TS; therefore, the result is that the signals DP and for picture elements in a specified number of words, they correspond to the same part of the frame.Therefore, and for the lines and picture elements in these rows in some part of the frame the task is to perform a purely spatial processing because the matrix contains 21 in their x positions of image values DP and for 17 of the image elements in each of the 17 rows in the matrix of the same digital video signal S(PI), although these image elements, for example - a1.1do sequentially line by line and the picture element for an image element in each of the successive rows (Fig.1) in the module 18 with the corresponding signals DP and displayed.Signals representing the simultaneous signals FROM and DP matrix 21 are present at a given point on h=272 outputs of shift registers, and before 17 registers, before 17 rows, i.e. registers d0.1d1.1... d16.1that gives a total of h+17= h outputs for h provisions of R0.0, R0.1... R8.8,..., P16.16.Inside the matrix 21, around the center of thehaving the coordinates x=8, y=8 (therefore the number of rows and number of columns in the matrix 21 is preferably odd, 2l+1 and 2m+1, respectively), m is its 9 elements is indeed the image elementwith coordinates x=8, y=8. Let this small matrix matrix
in which a Central elementunderlined.This matrix of 3x3 elements, containing provisions a, b, C, d, f, g, h and i around the Central element, orcorrespond to the eight oriented areas, each of which starts from a Central positionand passes to one of the other eight positions.To this end the eight directions can be identified using code Freeman, is shown in Fig.6, and direction are encoded digits from 0 to 7, starting from the x-axis, with steps 45o. In the code Freeman 8 possible oriented areas numbered from 0 to 7 can be represented by a three-digit number, as 23=8, which gives eight possible options.For the above minor matrix M3 8 directions corresponding to the Freeman code, starting from the Central positiontake the following form:
as shown in Fig.6.Considering shown in Fig.4 matrix 21 with h of the pixels or image elements is odbijenou the environment in the scene, observed by the video camera 13 and, therefore, presented in the form of a digital video signal S consisting of picture elements PIijtand how to determine the speed and oriented direction of the actual movement about an almost stationary environment.Between two successive frames, for example - TR1and TR2(Fig.5), the picture elements PIijin the signal S will be characterized with respect to their change between time t1(the first frame) and the time t2(the second frame) two signals DPijand COij(distributed by scanning in the matrix 21).A significant change in the value of the picture element at some point in this matrix exists if DP=1 for this point. Therefore, the area that really moves, identified by matrix area in which DP=1 for each point.In fact, the computational module 17A is used to verify at the same time various nested square matrices with the center in theand with dimensions 15x15, 13x13, 11x11, 9x9, 7x7 5x5 and 3x3 inside the matrix 21 with h provisions, and the 3x3 matrix is the above matrix M3. The device determines which of these Mat is UPRAVLENIE traffic zone, in which values of DP equal to 1 and which defines a change of +1 and -1 around WITH. To change the +1 and -1 around WITH DP must be set to 1 whenever the value if the test should provide a satisfactory result. Selects the smallest matrix that was used for testing (the main line).Then inside this moving zone in one of the sub-matrices, for example in a small matrix M3 with 3x3 elements, the device determines whether changes FROM on each side of the Central position in a given direction, from +1 oriented in a direction and to -1 along the oppositely oriented directions. For example, if we have -1, 0, +1 in the (oriented) direction 1, in other words - in positions g, e, C, respectively, in the small matrix M3 in this matrix there is a movement from right to left in (oriented) direction 1 in the code of Freeman (Fig.6). Obviously, at the same time in this direction in the small matrix DP=1. The movement speed is higher when the matrix in the nested matrices from 3x3 to 15x15, which varies from +1 or -1 between two adjacent positions along a certain direction, and more. For example, if you have -1, 0, +1 oriented in the direction 1, in other words, g, e, s - matrix Noi as M3 (Fig.7).Because it is a power of two and it seems this degree in preferred embodiments implemented, it is possible to identify a wider range of speeds, using only a few bits for the power of two, and at the same time to identify relatively low speed (which you can choose by increasing the changes or boundaries for the test, for example, -2, 0, +2 at positions g, e, in the matrix M3 3x3 would indicate half the speed that corresponds to -1, 0, +1 for the same positions in the matrix M3).You can also provide two tests to exclude uncertainties:
using the first test selects the largest deviation, in other words, the largest time constant, if there are changes along several directions in one of the sub-matrices, for example in a small matrix M3 with 3x3 elements;
using the second test arbitrarily choose one of two (or more) directions along which variations of the identical, for example, by selecting the smallest value code Freeman; this case usually occurs when the actual (oriented) direction of movement is approximately between two consecutive coded directions in the code FR is nachit of 1.5 (Fig.6), forming an angle of about 67,5owith the direction of the x axis (the direction of 0 in the code Freeman).Definition oriented direction and speed zone with a valid move, as described above, is performed by a compute module 17A (Fig.4), which is associated with the module 17 and which receives the above-mentioned h output signals from the matrix 21 for FROM and to DP (shows two output signals from the matrix 21, namely S0.1and S0.16). Block 17A handles values WITH and DP for sequential nested matrices and uses these values to determine the oriented direction (in code Freeman) and speed (depending on which you want to select a sub-matrix), it is possible, by applying the above tests.Scan the whole frame of the digital video signal through the matrix 21 is:
first group of the first 17 rows or rows (rows 1-17) in the frame: the left frame to the right, considering the relative movement, as shown for the frame TR2in Fig.5, i.e., on the part of TM1on the edge on the left, then shift to TM2one column is relatively TM1and before TMM(where M is the number of picture elements in a row or a range of frame) at the edge of the right;
- then go down one line, from line 3 to line 19 -- until then, until it reaches the last group at the bottom of the frame, namely, the line L - 16,...,L (where L is the number of lines per frame).Considering Fig.2 and 4, you can see that the signals generated by the modules 17, 18, 19, in other words, the node spatial processing are the following:
signal V representing the speed of movement on the basis of the maximum amplitude changes in the identified area, the value of which may be, for example, represented by a scale of eight integers from 0 to 7, if the speed is represented as powers of two, and therefore, it contains 3 digits;
signal DI, representing the direction of the movement, defined on the basis of the direction of maximum change, and the value of DI can be represented one of eight values from 0 to 7 in the code of Freeman, and hence it contains 3 bits;
- confirming signal VL, indicating that the result for the velocity and (oriented) direction is valid, in other words, enables us to distinguish valid output signal with V=0 and DI=0 from the absence of the output signal due to chance, and this signal is equal to either 1 (actual output) or 0 (no in the Yeni, for example containing 3 bits;
- (3 or 4 signal V, DI, VL, and can be, and WITH the supplied module 17 spatial processing and associated electronics);
- delayed video signal SR, consisting of the input video signal S delayed in the delay module 18 16 durations TR consecutive lines, and therefore on the duration distribution of the signal S in the matrix 21 size h to obtain a digital video signal simultaneously with the matrix representation of the matrix 21, and the content of this signal can be displayed on the TV screen or monitor;
all three signal module 19, namely the synchronization signal HP, the signals SL sequence of rows and SC sequence of columns.Nested rectangular matrix, is shown in Fig.4 and 7, you can replace nested hexagonal matrices (Fig.8) or inverted L-shaped matrix (Fig.9).In the case shown in Fig.8, sub-matrix (shown only coming to the center of the matrix MR1 and MR2) centered on the point MR0, which corresponds to the Central point (in which a binary signal "0") matrices M3, M9, shown in Fig.7. The advantage of the system with hexagonal matrix is that it pozwolenie with identical parties to perform isotropic calculation speed.The matrix shown in Fig.9, consists of a single row (Lu) and one column (Custarting from the Central position MRuin which two signals DP and respectively equal to "1" for DP and increases or decreases by one unit for, if we have the movement.Thus determined, is whether the direction (relative) movement:
- in the direction of the x: a signal WITH identical in all positions (rectangles) in the column With auand the binary signal DP is equal to 1 in all positions in the range of Lustarting from the initial MRuwith the value of COuto position in which WITH equal COu+1 or -1, inclusive.
- in the direction of the coordinates of the signal WITH identical in all positions (rectangles) in the range of Luand the binary signal DP is equal to 1 in all positions in the columnustarting from the initial MRuwith the value of COu- to position, which is COu+1 or -1, inclusive.
or, finally, inclined relative to the x and y binary signal DP is equal to 1 in positions (rectangles) Luand provisions (rectangles) si, and the inclination is determined by the perpendicular to the line passing through the two positions in which the signal COuuchange the value by one unit in two specific provisions of the Lu3and Cu5and specifies the corresponding slope of the Pp.In all cases, the movement speed is a function of position, which changes the value by one unit.If changes to one unit only in Luoruthis is consistent with the position value changes.If changes by one unit in a position inuand in a situation Withuspeed proportional to the distance between MRuand Ex(the intersection of the line perpendicular to Cu-Luand passing through MRu).The system described with reference to Fig.1-9, in a preferred embodiment, it is possible to add an additional system, which will be described with reference to Fig. 11-16, to obtain the global system 22, shown in Fig.10, which is, first, the node 11a, shown in Fig.2, which shows the signals V, DI, VL, C, and SR and full signal F (HP, SL, SC) transmitted by node 11 (which accepts the incoming digital signal S) in the node 22A (which delivers the full release signal ZH).The join line Z-Z1between nodes 11a and 22A shown in Fig.2, 10 and 11, and the outputs of m is non-cash, issued by module 22A (from the global device 22), consists of a full signal ZH, providing relevant information about the area with the relative movement in the scene 13A controlled by the camera 13.Additional module 22A connected to the outputs of module 11a, shown in Fig. 11 in the form of functional blocks; recall that this drawing is connected along line Z-Z1(at the top) with Fig.2 along the line Z-Z1(in the lower part of Fig.2).The module shown in Fig.11, consists essentially of the device for the formation and use of histograms and contains:
- Microline 23, which transmits the series of digital signals, described in detail below;
six positions indicated 24, 25, 26, 27, 28, 29 blocks the formation and processing of histograms designed to obtain histograms of signals: the delayed digital video signal SR, velocities V, oriented areas (codes Freeman) D1, constant (in time), the first x-axis(m) and the second axis y(m), respectively;
six blocks 30, 31, 32, 33, 34, 35 linear associations, each of which combines its input signals from bus 23 for forming a confirming signal1V2V3V4V5V6for the six blocks 24, 25, 26 is kOhm 26, unit 33, associated with the block 27, block 34, associated with the block 28, block 35, associated with the block 29;
block 36 moving area coordinate output signals of the blocks 28 and 29 for the x-axis(m) and y(m); and
unit 37 changes the data line, receiving signals orientation x(m)0and y(m)0for the x-axis(m) and y(m), as well as the added HP image elements, the signals SL sequence of lines and signals SC sequence of columns (and these three signals are grouped together in group F, as shown in Fig.2, 4, 10 and 11), coming from the module 19 shown in Fig.2 and 4, and generates the signals x(m)1and y(m)1sent in blocks 28 and 29, respectively.The blocks 24, 25, 26 and 27 send each of its output signals SR, SV, SDI, SDO bus 23, while each of the blocks 28 and 29 of the signals x(m)2and y(m)2the inputs of the block 36 moves zone, which combines these two signals of the blocks 28 and 29 and delivers the full signal xy(m) to the bus 23.The work of each of the blocks 24-29 formation and processing of histograms, which take, first, a confirmation signal V1V2V3V4V5V6from the associated block 30-35 linear Association, and the second signal SR, V, DI, CO, x(m)1or y(m)1obra 25 for forming and processing histograms of the velocity V, taken as an example; the only difference to other analog blocks 24, 26, 27, 28 and 29 is processed variable; note also that different incoming signals for the six blocks 24-29 all are digital signals, which provides a similar structure and operation of these six blocks.Fig. 12 schematically represents the envelope 38 and 39 histograms, respectively, by x and y axes of a Cartesian coordinate system the matrix 21 from h elements shown in Fig.4) velocity V zone, making the movement (Fig. 14 shows the elements such as C1C2, envelope 38 histogram); xMand yMrepresent the x and y coordinates of the maxima of the two envelopes 38 and 39, respectively, while laand lbfor the x-axis and lcand ldfor the y-axis represent the range of significant (interest) velocity, and laand lcare lower limits, and lband ldare upper limits significant plots of histograms.The vertical line Laand Lbx laand lband the horizontal line Lcand Ldy lcand ldform a rectangle that surrounds the shaded area 40 significant speeds (for existing near the main area 40.Therefore, all that is needed to characterize areas with the highest settings shown in the histogram, in this particular case, the velocity, V, is the identification of the coordinates of the four limits of la, lb, lwithand ldand two maxima xMand IM. Unit 25 continuously outputs the information concerning V, Microline 23.Similar blocks 24, 26 and 27 are continuously display the information regarding the zone of maximum values for SR, and DI, respectively, in this microsine 23 in the same way.Finally, similar to the blocks 28 and 29 are continuously display the information relating to the zone of maximum values of x(m)1and(m)1accordingly, in block 36, which combines information for the abscissa and the ordinate denoting its x(m)2and(m)2accordingly, in a full signal Hu(m), which is transmitted through the output unit 36 to the bus 23.Eventually, the bus 23 transmits the information for the maximum values SR, V, DI, and Hu(m), respectively, in other words, x(m)1and x(m)2to determine whether the moving area in the scene observed by the camera 13 to localize the area and to determine the speed of movement and (oriented) direction.In Fig. the and ZH, in particular for V and DI, in other words - for the speed and oriented direction of the moving zone that can be displayed in digital or analog form, can generate the signal light and/or buzzer, in particular if the speed exceeds the limit, or they can be transmitted via cable, optical fiber or radio systems with remote transmission of signals to the control module, for example, 10A in Fig.1, is placed near or at a distance from the device 11 corresponding to the present invention.Part of the modules shown in Fig.11 over bus 23 performing element-by-element processing the whole frame and the relationship of the elements in the frame, is used to specify external global values to determine whether the observed area area with relative movement, and, if so, is its localization, and then, if this area really moved, it is determined by its speed and oriented direction of this movement. This zone relative movement is identified in the observation plane along the two directions x, y, which are not necessarily orthogonal (for example, as shown in Fig.15 and 16), and formed part of the modules shown in block 25, and the associated linear block associations, such as the block 31, with reference to Fig.12, 13 and 14.Block 25 (Fig.13) includes part 25A forming the histogram, and part 25b forming the classifier for this histogram, and these two parts work when running through the software, persisted as part 25C of the integrated circuit, which determines the limits of la, lb, lwithand ldhistogram (Fig.12).The classifier 25b as classifiers of other blocks 24, 26, 27, 28, 29 of the formation and processing of the histogram (in the case of the two last - through module 36 Association) writes its output to the bus 23, and through the said bus - in module 31 of the linear combination, which, therefore, receives data in parallel from all classifiers modules 24, 25, 26, 27, 28, 29, and depending on this information, issuing or not issuing a confirmation signal V2in the node 25.Using software 25s, the classifier 25b defines different classes (each of which contains the same number of speed values in this case), which will determine such envelope, as 38 or 39 (Fig.12).In Fig. 14 shows a consistent classwise in the classifier 25b.Fig.15 and 16 illustrate the part used to build histograms for x(m) y(m) generated by the modules 28 and 29 and merged at the node 31 to obtain the slope.For example, consider the case of observation of the road using the camera with a digital output, placed on Board the vehicle, and the associated device corresponding to the invention.In Fig.15 shows the two sides of the road Bgleft and Bdto the right of the road R, and options tilts projection Pxrelative to x(m) defined by the module 28, and the projection Pyrelative to y(m) defined by the module 29, and the slopes are numbered, for example, from 0 to 7 (using a different order from that adopted in the code of Freeman).To set the maximum accuracy with respect to the right side of the Bdin other words , the maximum sensitivity of the readings (e.g., speed) in relation to that party, projectionxmust have a slope that is very close to the optimal slope of the Paboutthat is perpendicular to Bdand specifically a slope of 5. Therefore, the maximum value of the histogram is obtained for slope 5 defined by the module 28 (Fig.11).The same rationale applies to the left Saudi two optimal tilt, provides information about optimizing for the two sides of Bdand Bg.Fig. 16 illustrates the problem of determining the optimal slope of the Paboutthe projection of Pxcarried out, for example, for optimum driving of the vehicle Vh in continental Europe drive on the right side of the road (case a) or in the UK you drive on the left side of the road (case b), and finally, the aircraft Va for correct landing in the center of the runway of the airport (the case).Therefore, to ensure driving a ground vehicle (car, truck) on the road or flying the aircraft (aircraft, spacecraft, reusable) in the area of the runway of the airport device corresponding to the invention, additionally includes means represent the right and left sides of Bdand Bgaccordingly, a road or runway, as well as a means of orientation, at least one of the coordinate axes with adjustable slope so that it remains approximately orthogonal to the party concerned (Pabout).Still opisyvalis is to define the scope, not moving in the environment in which moves the background (e.g., crash or vehicle accident on the freeway), the opposite situation occurs when you need to localize the zones in which DP=0, relative to the environment in which DP=1. Obviously, in this case the speed will be zero in one area, so that the notion of direction makes no sense. Therefore, the calculation module 17A will be different.The device corresponding to the invention, if it is intended to be used only for defining the fixed zone, you can simplify it by removing modules or units, processing speed and direction, in particular blocks 25, 26, 31, 32, and can reduce the number of outputs from the module 11a and input module 17.In the operational module 10A of the device corresponding to the invention, it is possible to provide the display of histograms and/or values of the signals DP and on the monitor screen.The signal SR, namely the delayed digital video signal, usually served on a screen 10 of a TV or monitor for display of this signal is locally or remotely on the screen at the same time indicating the presence of relative motion to verify the nature of this relative of the presence of a zone of relative movement, for example, by a block visual alarm and/or buzzer.It is useful to facilitate the identification of areas with a relative movement on the screen by using different colors for the delayed digital video signal SR, with each color or color shade represents the speed and/or direction of movement.Different modules, which are described with reference to Fig.2, 3, 4, 11 and 13, consists of an electronic circuit of a known type, in particular with microprocessors that perform calculations and/or comparisons or using the signals of the scanner, mass storage devices, modules, delay, shift registers, modules, forming a linear histogram and linking histogram of this type in the plane, and microcinema.The combination of these electronic circuits in separate modules 11a and 22A or node 22, which consists of modules 11a and 22A, can be obtained by using two small integrated circuit or one small integrated circuit size of about 10 x 10 mm, for example, on the basis of 0.7 μm technology and to create a set of two interconnected integrated nodes or a single integrated circuit, which is connected to an input of digital video cameras or other surveillance devices and their outputs with the ones if the module 11a is used only simplified device, the device, preferably in the form of an integrated circuit, is placed between the specified digital output and one or more located here or remote user devices.Below are examples which do not restrict additional use of the device corresponding to the invention, in particular device corresponding to Fig. 1 and 10, in other words, the node shown in Fig.2 and 11, joined along line Z-Z1.The first additional application shown in Fig.17 is automatic cropping of a person moving in the room, for example, when communicating via video conference. Automatic cropping eliminates human movements, moving around, which improves the definition of the image of man, and observed using a video camera with a digital output, and if the digital signal is compressed, the compression process is simplified.In Fig.17 shows the camera 13 of the aforementioned type, which monitors a person P that can move around it. The digital video signal S from the camera is transmitted via cable, optical fiber, or happy, the output signal ZH this device affects the module 42, which in response to signals received from the device 11 and concerning the position and movement of the human P, controls the motors 43 of the specified camera 13 to direct the optical axis of the camera at the person, and specifically on his face, depending on the location, speed and direction of its movement, possibly changing the lens with a variable focal length, focal distance and/or focus of the camera, if person P moves forward or backward.You can also control the movements of one or more of the spotlight on the person (actor, singer) on stage, and the device corresponding to the invention, the centers of the man in the center of the image by changing the orientation of the camera and control the direction of the spotlight, for example, with one or two rotary mirrors for each spotlight.Another example of application of the device corresponding to the invention, shown in Fig. 18, which shows the camera 13 (or other surveillance devices), which produces a digital video signal S in the device 11 corresponding to the invention. In this application, the camera 13 controls the part of the motorway (highway) to detect nutmobile stopped after the collision.Therefore, the problem in this case is to determine whether a stationary object (a vehicle) in a moving environment (among other vehicles), in other words, to localize the area where DP=0 in the matrix 21 from h elements. Typically, the camera 13 observes the flow of vehicles, which gives the responses DP=1, to obtain the values of the speed and direction of movement. However, if any vehicle is stopped, it is identified by the response DP=0 in the zone of observation.Module 44, the receiving both signals ZH and SR, detects the appearance of the area in which DP=0 in ZH, and outputs a signal NR of errors, which, first, starts the buzzer and/or block visual alert device 45, and secondly, controls the switch 46, which feeds the signal S (or even delayed signal SR) on the screen 10 of the TV or monitor that allows the dispatcher warned by a buzzer or block visual alerts, to observe the highway, stop the vehicle or in the event of a collision, to take the necessary measures, for example, depending on the reaction of the driver of the stopped vehicle.Using invented the main road), each of which has a chamber 13 and the device 11 corresponding to the invention, moreover, the signals ZH and SR of each device 11 are transmitted via cable, optical fiber or radio dispatcher station, where there is a shared module 44; the observer need only look at the screen 10 in the event of an incident or accident, which alerted the alarm, because the occurrence of several incidents and/or accidents at the same time in different places is unlikely.In particular, the module 44 may include the input switch of the rotary type (not shown) that sends signals ZH (or SR) from different nodes 13-11, sequentially and cyclically placed along the motorway, part of this module 44 that form the signal NL.The same system can detect not only the case, when the vehicle is stopped or there is a collision, causing the stop of the vehicle, but it also reduces the flow speed of vehicles (very slow traffic) through the reduction of speed in different zones, in which DP=1, and Vice versa, the movement of the vehicle with a very high speed, so that the speed of the observed zones of the plants, namely, the interaction of the "man - machine" that is implemented by movement of the arm M or, more specifically, fingers DG within the zone SF, divided into rectangles coordinate system Withxand Cy.The video camera 13 with a digital output associated with the device 11 corresponding to the invention can be used for motion sensing hands, Feet and fingers DG, and can also be used to control your computer (for example, like "mouse") or to control certain functions in the car. For example, the node 13-11 could use deaf people who use a standard sign language of the deaf, based on the movements of the hands, to enter alphanumeric data, and hence the text in the computer without the use of a conventional keyboard; this operation, which, obviously, can perform speaking people learned the sign language provides input text into the computer without using the keyboard. A system of this type is insensitive to changes in time-dependent, and does not require precise signaling the beginning and end of the gesture.Fig. 20 and 21 schematically illustrate the application of the invention to monitor the drivers of the car, to signalisierung means, for example, above the rearview mirror, where she could watch the driver.The first operation is to crop the driver in the same way as in the case of the application shown in Fig.17. In Fig.20 schematically shows the image 1C of the driver on the video screen. First, precluded the use of the left and right sections (as shown by the horizontal shading image), reducing thus the treatment to the Central part of the image between the two locations.Then, in this Central part, it is enough to control the shaded area AA, shown in Fig.21 and encircling the head.The move in question and which are detected by the device corresponding to the invention, consist of flashes of the driver (indicated by the vertical movements in the area (AA) with a frequency, which is measured as the time before, when he falls asleep. If the frequency / speed of blinking falls below a certain threshold, the buzzer sounds, and the driver wakes up.In Fig.22 shows an extension tool for a limited number R of bits representing the time constant, when it is necessary to consider a wider range of speeds. This purpose will be used with clochette in sequential dividing the entire video on successive half, labeled 1, 2, 3, 4, 5, 6, 7. It provides compression, which are processed only part of the image. Thus, when p=4, in other words, if 2p=16, we can determine the speed within a wider range.If the original contour of the image scanning device of the invention, indicates that the speed of a moving object (in the broad sense of the term) exceeds the maximum rate defined by the value ofp= 16 for a constant time, then you want to use partial observed images 1, 2, 3, 4,... up until the speed of a moving object will not cease to exceed the specified maximum speed inside the partial frame image after compression.To use the decomposition Mallet you must enter the module 13A (shown in Fig.22) in the diagram shown in Fig.1 to implement this compression of the video signal. For example, this module may consist of an element defined as "Efficient multi-format video codec Di-VI 601", produced by the American Corporation "Analog Devices" ("ANALOG DEVICES") and described in the document "ADV 601 Preliminary Data Sheet", January 1996 Fig.2 shows an additional node 13A compression of this data type.The invention enables detection of the relative movement in the scene, observed an optoelectronic device such as a video camera, which transmits the observed scene in the form of a digital video signal consisting of a sequence of frames, which themselves consist of a sequence of strings consisting of the sequence of image elements, and this digital signal is analyzed to identify areas where the relative displacement if the area really moves relative to a practically stationary environment.Given that the device corresponding to the invention, determines the oriented direction and velocity of an object (in the broadest sense of the term), with this device you can combine some of the tools to use these two parameters to specify the future position of objects and direction of the camera 13 in advance in the direction of this future state.Note that the results obtained using the device corresponding to the invention does not require that the camera was stationary, so that the camera and the associated device can be set illustrated in Fig.16).After a very short period of initialization, is equal to N (of the order of 10 consecutive frames), the device corresponding to the invention, defines the parameters of the relative movement immediately after the end of each frame, which was held temporal and spatial processing, due to the recursiveness of the calculations in accordance with the invention.The above described the preferred implementation of the device corresponding to the invention and some of its applications. Obviously, this option and these applications are given as a non restrictive examples, and you can anticipate numerous variations and adaptations that will be obvious to those skilled in the art and are feasible within the scope of the invention defined in the following claims.For example, you could implement the methods of use of signals outputted to the output node 11 shown in Fig.2, differing from those illustrated in Fig.11, without going beyond the scope of the invention.With regard to applications of the device corresponding to the invention, they are by no means limited to those given as examples in the above description. So the Oba to stabilize it for movement, caused by random movements of the user.You could also use one or preferably several of the devices corresponding to the invention, associated with one or preferably several camescope or video cameras with a digital video output, housed in a room in the building to create a "smart room" through this system, able to detect and determine the location and movement of one or more persons in the room, to analyze this movement for security purposes, or to identify and/or assist in the solution of problems, such as finding kids in another room or clients in a Department store.In particular, it was assumed that the video signal is used with parts of the sequential frames, in particular, when discussing the capacity of the memory device 16, with a pairwise processing of two frames in the device corresponding to the invention. However, you could use only one frame of the two (for example, odd-numbered frame), reducing storage capacity and reducing the speed with which the obtained information, half. You could also use a video camera or other ustroystv applications, you can connect the sensors to the device, corresponding to the invention, for example, to be able to handle the extra movement options.It is obvious that the invention is not limited to the described specific options exercise or applications, but includes all variations and modifications that are part of a generalized definition of the invention.
1. Method for making real-time identification and localization of areas with relative movement in the scene observed by the monitoring system with an output signal consisting of a digital video signal containing a sequence corresponding frames, each of which consists of a sequence of rows, each of which consists of a sequence of image elements, and to determine the speed and oriented direction of movement, characterized in that it comprises the following sequence of operations on the digital video output: temporal processing for each position of the picture element, which consists in determining the difference between the amplitude of the signal component image in the current frame and the signal amplitude of the picture element, represent the global video distribution matrix with fewer rows and fewer columns, than, respectively, the number of rows and the number of picture elements per line in a digital video signal values, in the same moment of observation, of the elements of the image of the specified frame (the part is scanned across the matrix for the duration of the frame) signals characterizing changes in values of image elements, and of these signals at a given point of observation, i.e. for a given frame mentioned video or digital signal, generating signals representing at least the presence and localization of zones of relative movement and oriented direction of this displacement, if any, and the it includes smoothing the specified digital video signal using a digital time constant, the numerical value for each of the picture elements mentioned digital video signal and independently from each of them varies in accordance with changes in time or the absence of such changes each value of the image element memorizing, on the one hand, the frame of the output signal after smoothing and, on the other hand, the mentioned time constant for each position of the image element in said frame, said time is that, the amplitude of the possible significant changes in the amplitude of the signal component image between the current frame and the immediately preceding smoothed and stored by the frame, and generating two digital signals, the first signal is a binary, or bit, a signal with two possible values, one of which is the presence, and the other - no significant changes between the two mentioned frames, the current and immediately preceding smoothed, and the value of the specified binary signal changes the stored value mentioned time constant for the corresponding picture element, to decrease the time constant, if the specified binary signal has a first value, and to increase this time constant, if the specified binary signal has the second value, with the increase or decrease is carried out quantitatively, while the second digital signal, i.e. the signal amplitude is mnogopudovymi signal with a limited number of bits, quantitatively determining the amplitude of this change, and the spatial processing includes, for each frame of the output video signal distribution according to the mentioned matrix for length is, the definition in this instant double matrix representation of the specified binary signal and the digital signal amplitude for the same picture element, in which at the same time specified the binary signal has a desired value, representing the presence or absence of significant changes, and the specified digital signal amplitude changes or does not change by a significant amount between adjacent picture elements in the matrix along the oriented direction, starting with the original image element, in the same part of the frame, and therefore in the same moment, observations, and generate signals, representing the presence and localization of areas with potential relative displacement and the relative frame-to-frame speed and oriented direction of this possible relative movement in its environment, on the basis of the instantaneous matrix distribution of these two digital signals is a binary signal and signal amplitude.2. The method according to p. 1, characterized in that it also includes the formation, first, histograms of the values of the signals distributed in the matrix, and secondly, histograma significant change of the processed values in each of the formed histogram, and for each identified region, establishing the presence of a zone of relative movement and, if available, its localization and determination of the speed and oriented direction.3. The method according to p. 1 or 2, characterized in that the above matrix is a square matrix with the same odd number of rows and columns (2l+1), using nested matrix, which contains 33, 55, 77, . . . (2l+1)(2l+1) elements, centered on the center of this square matrix to determine the lowest nested matrix in which the specified digital signal changes along the oriented direction, starting from the specified center, and the value of the specified binary signal indicates that the threshold is exceeded along this direction.4. The method according to p. 1 or 2, characterized in that the above matrix is hexagonal matrix, using nested hexagonal matrices of increasing size, centered on the center of the hexagonal matrix, to determine the lowest nested matrix in which the specified digital signal changes along the oriented direction.5.him and one column using nested matrix, which contains 33 image elements, 55 image elements, 77 elements of the image . . . (2l+1)(2l+1) picture elements for one row and one column to determine the lowest nested matrix in which the specified digital signal changes along the oriented direction, namely, the line with the greatest slope and constant quantization.6. The method according to any of the preceding paragraphs, characterized in that the specified continuous period is represented in the form 2pwhere R is a number less than 16, resulting in its present using no more than 4 bits, and this time constant decrease or increase, subtracting or adding one unit to the R.7. The method according to p. 6, characterized in that use progressively smaller part of the full-frame based algorithm temporal scale Mallet and selects the largest of these parts, which gives evidence of displacement, velocity, and orientation that is compatible with the p value.8. A device for implementing real-time identification and localization of areas with relatively the using of the method according to p. 1, contains a monitoring system with an output signal consisting of a digital video signal containing a sequence corresponding frames, each of which consists of a sequence of rows, each of which comprises a sequence of image elements, the temporal processing module for determining for each position of the picture element difference between the signal amplitude of the picture element in the current frame and the signal amplitude of the picture element characterizing its value in the previous frame, and the spatial processing module, for each frame of the digital video signal distributes the matrix with fewer rows and fewer columns, than, respectively, the number of rows and the number of picture elements per line in a digital video signal values, in the same moment of observation, of the elements of the image of the specified frame (the part is scanned across the matrix for the duration of the frame) signals characterizing changes of image elements, and of these signals at a given point of observation, i.e. for a given frame mentioned video or digital signal, generates signals representing at least the presence and localidades fact, it contains a means (15) for smoothing a digital video output using a digital time constant (CO), the numerical value for each picture element of a digital video signal and independently for each of them varies in response to a change in time or the absence of such changes in the values of the picture element, means (16) for storing, in the first frame of the output signal after smoothing (LI), and secondly, the mentioned time constant for each position of the picture element in the above-mentioned frame, and also in the module (15) temporal processing means (15) comparisons to determine, firstly, the availability and, secondly, the amplitude of the possible significant changes in the amplitude of the signal component image between the current frame and the immediately preceding smoothed and stored by the frame, and to generate the first digital signal (DP), which is a binary or single-bit signal with two possible values, one of which is the presence of the abovementioned significant changes, and the other the absence of such a significant change between the two mentioned frames, the current and immediately preceding smoothed, and the value of the specified binary SNIA, to suitably reduce this time constant, if the specified binary signal has a first value, and to increase the time constant, if the specified binary signal has the second value, with the increase or decrease is carried out quantitatively, and the means of calculation (15C) to generate the second digital signal amplitude (CO), which is mnogopudovymi signal with a limited number of bits for quantization of the amplitude of this change, and in the module (11) spatial processing, the inputs of which block time processing filed referred to a serial binary signal and the digital signal amplitude for image elements of the same frame, the means for distribution by the above-mentioned matrix (21) during the frame duration, the first specified binary signal, and secondly, the specified digital signal amplitude, means for defining in this instant double matrix representation of the specified binary signal and the digital signal amplitude for the same picture element of the particular zone in which the specified binary signal has a desired value, representing the presence or absence nacimos is elicina between adjacent picture elements in the matrix along the oriented direction, since the source of the image element, in the same part of the frame, and therefore in the same moment of observation, and means for generating signals representing the presence and localization of areas with the possibility of relative movement, the relative frame-to-frame speed and oriented direction of this possible relative movement in its environment, on the basis of the instantaneous matrix distribution of these two signals is a binary signal and signal amplitude.9. The device under item 8, characterized in that the specified spatial processing module includes a first cascade-connected delay means (r), each of which introduces a delay equal to the time difference between two consecutive rows, and second cascade connected delay means (d) for each row, each of which introduces a delay equal to the time difference between two successive picture elements in the row, and the outputs of the second delay means (d) and the input of the cascade connection of the second delay means (d) for each row give the specified torque values specified binary signal and the digital signal amplitude at the same point in the criminal code is adsto (24-29) to form a histogram of output values specified module spatial processing and histogram of the slopes of the two having a variable inclination of the coordinate axes in the plane means for identifying a region for each histogram, in which there is a significant change in the processed value to confirm this scope on the output and get all histograms output signals that identify and localize the area of the relative movement in the observed scene, if it is, and the speed and oriented direction of this movement, if the specified area effectively moves relative to its environment.11. Device according to any one of paragraphs. 8-10, characterized in that the said means of smoothing (15C, 15d) includes an input for receiving a digital video signal S(PI) and generation for each subsequent picture element in the frame of the video signal smoothed signal (LO), in which temporal changes of the input digital video signal is reduced by using the trigger signal received at the other input, and the time constant (CO) associated with the position of each picture element in the frame, the value of which varies consistently, so that the smoothing was supported by the reduced tendency of the changes of the incoming digital video signal, moreover, the smoothing tool works in conjunction with the storage device (16), which is for each part of the image elements in the frame, and gives the position of each picture element of at least a sequence of values updated time constant and the values of the binary signal, which indicates that exceeded the threshold absolute value of the difference between the value of the item image and its smoothed value.12. Device according to any one of paragraphs. 8-11, characterized in that the module (17, 18) spatial processing, which performs matrix distribution in the reduced number of rows and columns of the output signals from the means of smoothing, i.e., successive values of the time constant (CO) and the specified binary signal contains identification device (17A) for identification in the specified matrix distribution area of the image elements, in which at the same time or the value of the binary signal corresponds to the threshold is exceeded, and the time constant is changed between adjacent picture elements to a meaningful value in one direction and for generating output signals, indicates the localization of the specified area and the speed and oriented direction of movement in this area, or the value of the binary signal corresponds to the threshold is exceeded, and the time constant does not change between sat module (11a) temporal and spatial processing and device (22A) for the formation and use of histograms, interconnected for generating signals related to the area with relative movement, if any, in the scene of digital video signal S(PI), obtained from observations of the scene, and for grouping signals that indicate the existence and location of this zone, the speed and orientation of this movement, and referred to the scene represents a road or runway airport on the left and right sides, and further comprises means for representing the right and left sides Bd and Bg road or runway, apparatus for forming and use of histograms has the means of the projection (37, 28, 29, 36) on coordinate axes with variable inclination and the means for orienting at least one of the coordinate axes with variable slope, so that it remained approximately orthogonal to the party concerned (P0), for providing driving a ground vehicle on the road or when landing an aircraft on the runway.14. Method of monitoring the state of wakefulness of the driver of the vehicle to detect possible trends in the death of the driver, which consists in the El, and then, continuously, in real time, sequential image only the eyes of the driver, (b) process the specified signal related only to the eye of the driver, in accordance with the method according to any of paragraphs. 1-7, for consistent, real-time detection in the specified image vertical eye movements ocular century, representing flashing, determine the frequency of these movements and the detection frequency, which is lower than the threshold, the flashing frequency, corresponding essentially to the transition of the driver from the state of wakefulness to drowsiness, and (C) trigger the warning signal that warns the driver as soon as the specified threshold frequency of flashing is completed.
SUBSTANCE: support frame is assigned with sign, showing information about direction of support frame, and during determining of predicted vector of movement of encoded block averaging operation is performed with use of vectors of movement of neighboring blocks, during which, if one of aforementioned blocks has movement vectors, information about direction of support frames is received, to which these movement vectors are related, and one of movement vectors is selected with reference to received information about direction, than averaging operation is performed with use of selected movement vector to receive subject movement vector of encoded block.
EFFECT: higher precision, higher reliability.
3 cl, 1 dwg, 3 ex
FIELD: movement detection systems, technical cybernetics, in particular, system and method for detecting static background in video series of images with moving objects of image foreground.
SUBSTANCE: method contains localization of moving objects in each frame and learning of background model with utilization of image remainder.
EFFECT: increased speed and reliability of background extraction from frames, with possible processing of random background changes and camera movements.
4 cl, 14 dwg
FIELD: device and method for recognizing gestures in dynamics from a series of stereo frames.
SUBSTANCE: method includes producing a series of stereo-images of object, on basis of which map of differences in depths is formed. System is automatically initialized on basis of probability model of upper portion of body of object. Upper portion of body of object is modeled as three planes, representing body and arms of object and three gauss components, representing head and wrists of object. Tracking of movements of upper part of body is performed with utilization of probability model of upper part of body and extraction of three-dimensional signs of performed gestures.
EFFECT: simplified operation of system, high precision of gesture interpretation.
3 cl, 12 dwg
FIELD: technology for processing images of moving objects, possible use, in particular, in theatric art, show business when registration/recording is necessary or repeated reproduction of scenic performance.
SUBSTANCE: method includes inserting enumeration system for each object and performing projection of enumerated objects onto plane, while projection is displayed in form of graph with trajectories of movement of enumerated objects in each staging.
EFFECT: spatial-temporal serial graphic display of scenic action for its further identification and repeated reproduction.
FIELD: system for encoding moving image, in particular, method for determining movement vector being predicted, of image block in B-frame in process of decoding of moving image.
SUBSTANCE: in accordance to method, at least one movement vector is produced for at least one block, different from current block, while aforementioned at least one block is related to one, at least, supporting frame in a row of supporting frame, movement vector is predicted for current block on basis of received one, at least, movement vector, while prediction operation includes also operation of comparison of value of order number of B-frame to value of order number of one, at least, supporting frame, while movement vector for current block and aforementioned one, at least, movement vector are vectors of forward movement.
EFFECT: increased efficiency.
2 cl, 1 dwg
FIELD: mobile robot, such as cleaner robot, and, in particular, device for tracking movement of mobile robot.
SUBSTANCE: suggested device for tracking movement of mobile robot includes: video camera for filming an individual object; unit for tracking movement and creation of image for setting support one in an image for current moment by means of filming of individual object by video camera and creation of image in current moment, for which support zone is set; unit for selecting image of difference of pixels of image support zone limit based on difference between pixels present only at limit of support zone of aforementioned images; and micro-computer for tracking movement of separate object on basis of selected image of difference.
EFFECT: decreased time of pixel comparison operation and increased efficiency of room perception.
5 cl, 4 dwg
FIELD: video encoding, in particular, methods and devices for ensuring improved encoding and/or prediction methods related to various types of video data.
SUBSTANCE: the method is claimed for usage during encoding of video data in video encoder, containing realization of solution for predicting space/time movement vector for at least one direct mode macro-block in B-image, and signaling of information of space/time movement vector prediction solution for at least one direct mode macro-block in the header, which includes header information for a set of macro-blocks in B-image, where signaling of aforementioned information of space/time movement vector prediction solution in the header transfers a space/time movement vector prediction solution into video decoder for at least one direct mode macro-block in B-image.
EFFECT: creation of improved encoding method, which is capable of supporting newest models and usage modes of bi-directional predictable (B) images in a series of video data with usage of spatial prediction or time distance.
2 cl, 17 dwg
FIELD: systems for automatic video surveillance of an object.
SUBSTANCE: system for automatic detection and tracking of individuals on basis of images and biometric identity recognition based on target list, realizes following operations: on basis of three-dimensional data about scene and two-dimensional data, characterizing optical flow, detection of objects-noises of scene is performed, static background objects are selected, and regular dynamic object-noises; on basis of comparison of two-dimensional and two-dimensional data about the scene in current frame with reference data on previous frames and a map of object-noises changes are determined on a scene, in three-dimensional zones of interest, preliminary check of presence of human-like objects is performed, zones of interest are determined more precisely and their changes are tracked: a contour of separate elements of human body is singled out, zones of interest are divided onto a set of sub-zones of interest for elements, detection of three-dimensional head of individual is performed and it is tracked in each zone of interest; face of individual is tracked in each zone of interest; images of detected face are normalized in terms of dimensions, angles and brightness; recognition is performed.
EFFECT: objectivity and stability of system operation.
FIELD: observation of moving objects.
SUBSTANCE: method includes using movement sensors, capable of recording two-dimensional distributions of intensity in form of images, where sensors are positioned with known spatial orientation, making it possible to perform simultaneous observation of one and the same scene, periodical query of sensors is performed during whole time period after their enabling, processing and analysis of data received from sensors is performed, which constitutes series of images, and output signal is generated in case of detection of three-dimensional moving object and determining of its spatial position, which signal is injected into output device.
EFFECT: increased trustworthiness when determining spatial position of a moving object.
3 cl, 1 dwg
FIELD: method for interpolating values of sub-pixels during encoding and decoding of data.
SUBSTANCE: method of interpolation during video data encoding is claimed, which features an image, containing pixels ordered in rows and columns and represented by values having given dynamic range, where pixels in rows are in integral horizontal positions, and pixels in rows are in integral vertical positions, which image is interpolated in such a way that values of sub-pixels are generated in fractional horizontal and vertical positions, aforementioned method containing following stages: a) when values are required for sub-pixels in half-integral horizontal positions and integral vertical positions and in integral horizontal positions and half-integral vertical positions, such values are interpolated directly using weighted sums of pixels located in integral horizontal and integral vertical positions; b) when values are required for sub-pixels in half-integral horizontal positions and half-integral vertical positions, such values are interpolated directly using a weighted sum of values for sub-pixels located in half-integral horizontal positions and integral vertical positions, computed in accordance with stage a); and c), when values are required for sub-pixel in quaternary horizontal position and quaternary vertical position, such values are interpolated by averaging of at least one pair from first pair of values of sub-pixel located in half-integral horizontal position and half-integral vertical position, and of sub-pixel, located in integral horizontal position and integral vertical position, and second pair of values of pixel, located in integral horizontal position and integral vertical position, and of sub-pixel, located in semi-integral horizontal position and semi-integral vertical position.
EFFECT: creation of improved method for interpolating values of sub-pixels during encoding and decoding of data.
13 cl, 26 dwg, 2 tbl