Method of identifying insert frames in multimedia data stream. RU patent 2506640.

FIELD: physics, computer engineering.

SUBSTANCE: invention relates to image data processing means. The method involves calculating overall differential brightness of each difference frame and the brightness threshold of three previous difference frames, comparing brightness characteristics of the difference frame with the threshold of the overall differential brightness calculated during replacement of video stream frames, identifying the first boundary of an insert frame and identifying an insert frame of said stream.

EFFECT: fewer erroneous detections of insert frames in multimedia data stream.

5 dwg

The invention relates to the field of computer engineering, namely to the systems of image analysis, and is intended for use in the field of processing of the image data.

Method of identification frames-inserts in the stream of multimedia data is needed to prevent the negative impact on the level of unconscious perception of the human subthreshold incentives. This energy is often used secretly in the form of additional personnel, inserted into a stream of multimedia data.

Under the frames-inserts refers to the number of frames from 1 to 3 added to any place of the video stream and different frames of the current storyline (scenes) visually and statistically.

The concept of identification and comparison of frames in the stream of multimedia data includes a number of methods. The majority of them as a statistical characteristic for the detection of hidden frames-inserts in the video stream use the value of the integral brightness personnel, comparing it with some reference value.

Known device for processing images and a way to define a linear shift image (patent RU 2138851 C1, publ. 05.08.1993, Authors: Christoph Eisenbarth (DE), Ira Finkelstein (US), Dennis Mac guy (US), Edward (US))that can detect the mismatch of the actual image relative to the reference and to ensure their maximum coincidence. The technical result is achieved due to the fact that they took the two sections of the actual image, remember two similar plot to the reference image, compare the relevant areas of actual and reference images and determine if a shift, and repeat comparison to those same areas after the shift of the actual image on the predefined number of pixels in X and Y axes, and calculate the absolute values of the differences between the relevant sections of actual and reference images for each shift and remain in memory as the absolute values, and their sum.

The disadvantage of this analogue is that the identification of video sequences in comparison, it is impossible to distinguish the reference image, the areas of the frame for comparison and the threshold value of linear shift, because it will change dynamically as a result of changing scenes in the video stream.

Closest to the technical nature and functions of the claimed invention (prototype) are a way to identify genuine series of images and device for its implementation (patent RU 2216044 C2, publ. 10.11.2003. Author Klaus Schulze (DE)), which allow to obtain a technical result, the increasing of efficiency of accounting procedures and timing of advertising messages, video clips, political speeches.

This result is due to the fact that the individual images of a series of images determine the characteristics of brightness, converts them into digital form and compared to a reference, the signs of brightness selection according to the multitude of images.

The disadvantage of the prototype is the low efficiency determine frames-inserts in the video stream, due to the fact that single image, which determines the reference signs of brightness, it is impossible to get a comprehensive statistical data characterizing the brightness of the properties of each of the observed dynamically changing scenes of the video stream because of their significant differences in color and brightness.

The objective of the invention is to create a method of identification of the frames-inserts in the stream of multimedia data, allowing to reduce the number of erroneous recognition as frames-inserts, staff, individual storylines (scenes).

This problem is solved by the method of identification frames-inserts in the stream of multimedia data, including operations determine the brightness of the frames, digital comparison according to brightness components of two frames, according to the invention supplemented operations per-pixel basis for comparison of two images by definition frame-difference and calculate the total differential brightness each frame is-the-difference and the threshold value of the brightness of the three previous frames-differences and comparisons brightness characteristics frame-difference of the calculated along the frame rate of the video stream value of the total differential brightness previous three frames-differences.

The result of the proposed method is achieved by means of formation and execution of mathematical criterion R:

when limitations:

- number of frames in the visual inserting not exceed three;

- the first four frame storyline are not insert;

- insert is not located on the border of the two story lines, where

- many frames in the video stream, l=1(1)N; N - number of frames; R=1(1)R - number of subjects;

- many shots-of differences, l=1(1)N-1;

- ?={Ohm 1 Tensor 2 } is the set of classes training video stream, Ohm 1 - frame-box; Ω 2 - normal frame of the storyline.

Listed new set of operations provides the possibility of reducing the number of erroneously identified frames-inserts and personnel of separate story lines at different speeds change of scenes in the video stream.

The analysis of the equipment has allowed to establish that the analogues, characterized by the totality of features identical to all features of the claimed technical solutions, no, that indicates compliance of the declared method of condition of patentability "novelty".

Search results known solutions in this and related areas of technology for signs that coincide with the distinctive features of the prototype of the declared object, showed that they are obvious from the prior art. Of the level of technology also not found fame distinctive essential features contributing to the same technical result in the claimed method. Therefore, the claimed invention meets the condition of patentability of "inventive step".

The claimed method is illustrated by drawings, where it is shown:

figure 1 - the General scheme of the operations, implementing, in the aggregate, a way of identifying frames-inserts;

figure 2 - the result of a logical operation on the formation of personnel-differences;

figure 3 - the algorithm of calculation of threshold brightness L p ;

figure 4 - the algorithm for the identification of training video stream by comparing the brightness characteristics of the personnel-the differences with their threshold L p ;

figure 5 - principle of operation of the extreme methods of the total differential brightness.

The General scheme of sequence of operations of the declared method presented in figure 1, is as follows.

The assignment block of rooms frames of video subjects and 2 served an ordered sequence of video frames from conversion unit of the video stream in the frames RGB 1. (Footage of Robert Rodriguez movie "From dusk till dawn" (From Durk Till Dawn).)

Digital frame in the video stream is a function of the distribution of brightness or colour on a two-dimensional plane, f(i,j), where i and j - Cartesian coordinates describing plane frame. From the mathematical point of view frame is a two-dimensional matrix Im[i,j] size (DimH X DimW), where i=1(1)H is an integer describing the number of the element in the string, j=1(1)W - matrix row number, in which the item is located. Item frame (pixel) has an integer value that is proportional to the value of the brightness distribution function f(i,j) in the given point of the plane. Denoted pixel of the frame, the p-th plot with coordinates (i,j) and color components of Red, Green, Blue in the form of

where : W - the width of the frame in pixels, and H is its height. Values of H and W are exported to block initialization 3.

In the block of a logical bitwise difference consecutive frames in a video stream 4 the operation in pixels of the current frame

leaves isolated only those bits that are missing in the corresponding pixels subsequent

in accordance with the values of N and W, coming from the output of the block initialization rule F:

, l=1(1)N-1, p=1(1)P,

F - build rule frames of differences:

i=1(1)H, j=1(1)W, p=1(1)P.

The result of the actions of this operation is shown in figure 2. (Vizilter J.V., Zheltov, S.Y., Prince V.A. and others Processing and analysis of digital images with examples in Lab VIEW IMAQ Vision. - M.: DMK Press, 2007. - 464 C.).

In section 4, are determined by the difference for frames

training video stream

entering the unit of calculation of the total differential brightness of each frame difference - 5. In block 5 is the calculation of the total brightness

personnel of differences in the expression of

Research training video stream showed that a static image problematic to get all the necessary information characterizing the properties of observed brightness dynamically changing scenes, because all of them are different. Hence the need for the definition of average brightness values inherent in each storyline.

In block 6 is the calculation of the threshold brightness L p p-th plot previous three frames of differences in the expression of

where Delta - absolute error of differential brightness, determined empirically, coming from the block initialization 3.

The scheme of calculation of threshold brightness L p th plot in figure 3 presents

After the definition of threshold values of brightness in block 7 compares the total differential of brightness subsequent frames-differences threshold value according to the following rule:

1. In block 4 is determined by the frame-the difference for

training video stream

and in block 5 of calculates its total differential brightness

With inequality

identifies the first boundary:

- frame-insert;

- to be allocated training video stream;

- a new p+1 plot.

investigate the next frame of the story l=l+1 and in block 6 calculate a new threshold values of brightness p L as the arithmetic average of the total value of the differential between the brightness of the three frames-the following differences:

2. In blocks 4 and 5 define the frame-the difference for

training video stream

and the total differential brightness

With inequality

identified

, frames-inserts, coming in the isolation block of the training video stream 8, and in the overall brightness

in blocks 4 and 5 define the frame-the difference for

training video stream

and the total differential brightness

stand out

frames in a video stream coming in block 8 and when

in blocks 4 and 5 define the frame-the difference for

training video stream

the total differential brightness

4. Subject

frames in a video stream coming in block 8, otherwise in block 2 of the transition on R+1 plot in a block of 6 recalculated threshold brightness relative to the

training video stream.

Selected frames of the video stream is stored in block 9. (Selected footage from Robert Rodriguez movie "From dusk till dawn" (From Durk Till Dawn)).

Algorithm, full identification criteria the video in a multimedia stream of the information reflected in figure 4.

Principle of operation of the extreme methods of the total differential brightness is presented in figure 5.

Experimental verification of the method of identification frames-inserts was performed on the computer in the package of applied mathematics Matlab 2008b under the following initial data:

1) 100 videos without frames-inserts, lasting from 1 to 5 minutes with a different number of storylines;

2) 100 videos posted every 100 frames frames-inserts (number of consecutive frames-inserts from 1 to 3);

3) frames videos and frames-inserts are not distorted and have high quality.

The results of experiments showed that, in the absence of a priori information about the presence in the analyzed video stream frames-inserts, the known method (prototype) correctly identified 10% of the frames-inserts an erroneous recognition storylines in 67%, and by the claimed process are able to properly identify 89% of the frames-inserts an erroneous recognition storylines in 12%. This confirms the significant positive effect of the introduction of a new way.

The results obtained give grounds for the conclusion that the declared method of identification of frames-inserts in the stream of multimedia data has better characteristics in recognition accuracy, frames-inserts, compared with the prototype.

Thus, when such a combination of essential features in the identification process, frames-inserts in the video stream is provided by reducing the number of erroneously identified frames-inserts and personnel individual storylines in the video stream.

Industrial applicability of the invention is due to the fact that the device implements the proposed method may be implemented using modern element base, with the achievement of specified in the invention of destination.

Method of identification frames-inserts in the stream of multimedia data, which consists in determining signs of brightness and compare their values, threshold, characterized by additionally calculate the total differential brightness of each frame-difference and the threshold value of the brightness of the three previous frames-differences, compare brightness characteristics of the frame-difference of the calculated during the frame rate of the video stream threshold of the total differential brightness, if the total differential brightness

identify first the edge of the frame insertion, and if the total differential brightness

identify frame-insert the specified stream, where

are the staff of the specified thread, and

is the threshold value of the brightness of the scene.