System and method for masking video signal errors

FIELD: video communications, in particular, technology for masking decoder errors.

SUBSTANCE: in accordance to one variant of invention, system and method decode, order and pack video information to video data packets for transfer via communication line with commutated channels, due to which system conceals errors, caused by loss of video data packets, when system receives, unpacks, orders and decodes data packets. In accordance to another variant, system and method decode and pack video information so that adjacent macro-blocks may not be positioned in same data packets. Also, system and method may provide information, accompanying packets of video data for simplification of decoding process. Advantage of described scheme is that errors caused due to data loss are distributed spatially across whole video frame. Therefore, areas of data, surrounding lost macro-blocks, are decoded successfully, and decoder may predict movement vectors and spatial content with high degree of precision.

EFFECT: improved quality of image.

4 cl, 10 dwg

 

REFERENCE TO RELATED APPLICATION

This application is related to provisional patent application No.60/314413, registered on August 23, 2001 and entitled "System and method for masking the error signal", which is incorporated herein by reference.

BACKGROUND of INVENTION

1. The technical field

The present invention relates, generally, to video and, more particularly, to a masking of the error signal.

2. Background of invention.

Video is becoming more and more important part of global communication. In particular, video conferencing and video links currently widely used in desktop and room video conferencing over the Internet and over telephone lines, for monitoring and control, telemedicine consultations and machine-learning and training professionals. In each of these applications video and accompanying audio information transmitted over telecommunication networks, including the telephone line, integrated digital communication network integrated services (ISDN), digital network (DSL) and radio network.

The standard video format used for video conferencing, is a common intermediate format (CIF), which is part of the standard N. 261 video conferencing International austincommunity (ITU). The primary format for the CIF also known as full CIF or FCLF. Also installed additional formats with a resolution above and below format FCIF. Fig. 1 is a table of requirements for resolution and data rate for different formats in accordance with the assumption that require 12 bits per pixel according to well-known prototypes. Data rate (in megabits per second, Mbps) are shown for uncompressed color frames.

Currently, the effective transmission and reception of signals may require encoding and compressing video signals and associated audio signals support.

The encoding by the video compression is one way of encoding digital video information as it requires less memory to store video data and reduces the required bandwidth of the transmission. Some schemes compression/decompression (encode-decode or process "CODEC") are often used to compress the video image to reduce the required data transfer rate. Thus, the hardware CODEC and the corresponding program allows you to compress the digital video data in a smaller binary format than that required for the original (i.e. uncompressed digital video format).

There are several usual approach is in and of standards for encoding and compression of the original video signals. Some standards are designed for a specific application type JPEG (joint group of experts in the field of photography) for still images, and standards H. 261, N. 263, MPEG (Expert group on the moving image), MPEG-2 and MPEG-4 for moving images. In these coding standards typically are compensated image elements based on motion compensation 16×16, commonly called macroblocks. The macroblock is a unit of information containing four data blocks brightness 8×8 and two data blocks color 8×8 data in accordance with a sampling structure of 4:2:0, where the data chroma subsampling 2:1 in the vertical and horizontal directions.

As a practical measure, the audio data must be compressed, transmitted, and synchronized with the video data. Synchronization, multiplexing and issuing Protocol is performed according to the standards of H-type. 320 (conference video-based ISDN), H. 324 (phone video based on the normal telephone network) and N. 323 (conference video-based local network or IP telephony) (or its predecessor, the Protocol X. 261), which constitute videocameras of these groups of standards.

Diagram of the motion estimation and compensation scheme is one the normal way, use the th, as a rule, to reduce the requirement on the bandwidth of the transmission signal. Since the macroblock is the basic module data, motion estimation and compensation scheme can provide a comparison of a given macroblock in the current video frame with the surrounding area of the given macroblock in a previously transmitted frame with the aim to match the data. Usually, agreed macroblock in a previously transmitted frame spatially offset from a given macroblock in the amount less than the width of the given macroblock. If a data match is found, the circuit subtracts the given macroblock in the current video frame from a consistent macroblock offset in previously transmitted frame, so that only the difference (i.e. the balance) and the spatial offset must be encoded and transmitted. The spatial offset is usually called the motion vector. If the motion estimation and compensation process is executed correctly, the residual macroblock should contain only the amount of information required to describe the data corresponding to the pixels that change from the previous frame to the current frame and the motion vector. Thus, the field of video frames that do not change (i.e. the background), not encoded and transmitted.

Traditionally, the standard N. 263 determines that the movement vectors is s, used for motion estimation and motion compensation are differentially coded. Although differential encoding converts the data in the quantity demanded for transmission, any error in which the data of the motion vector in one macroblock is lost or destroyed, a negative impact on neighboring macroblocks. As a result, errors due to poor data, which reduce video quality.

In preparing the information of the video frames for transmission over a communication network with a packet switched coding scheme converts the information about the video frames compressed by the motion estimation and compensation technique, the data packets for transmission through a communication network.

Although data packets allow to improve the transmission efficiency, lost, destroyed or delayed data packets may also contain errors that lead to the deterioration of the video quality. Alternatively, the video information may be transmitted through a heterogeneous communication system, in which the end points of contacts connected to the network circuit-switched and used a gateway or other device with packet transmission through the public switched network.

In the currently used practice, lost or destroyed data packets often reduce the video quality. Therefore, sushestvitelnoe system and method who organize and transmit data packets with masking (hiding) the errors caused by packet loss data.

BRIEF description of the INVENTION

The proposed system and method eliminate or substantially reduce the disadvantages of the known systems associated with packet loss of video information. In General, the present invention proposes a system and method that encode, reorder and stacked video information for transmission via the network switching channels with the possibility of masking the error signal caused by packet loss of video information.

In an exemplary embodiment of the invention, video signals are encoded in sets of macroblocks. Then the reorganization of the macroblock assigns an integer label, called identifiers of the group of macroblocks (Wi) for each macroblock.

In one exemplary embodiment, the neighboring macroblocks, mainly not indicated identical Wi. Then the device packaging macroblocks performs batching macroblocks so that the macroblocks having the same IGM, are packaged together. For those variants of the invention, in which the neighboring macroblocks do not have identical labels, Wi, spatially neighboring macroblocks are packaged together. In addition, relevant data, card type infra-macroblocks may be included is in the image header or transferred by some other mechanism, to facilitate the process of decoding.

In another embodiment of the invention in which the imaging device receives data packets containing coded macroblocks, the data packets are decompressed, and the encoded macroblocks are sorted and decoded. In an additional embodiment, the imaging device decompresses the received data packets and then decodes the macroblocks in the order in which they were received, to reduce the processing time. If you have lost one or more data packets, the data accompanying the macroblocks successfully transmitted data packets in order to reduce the adverse effect of lost data packets. Different methods are used, based on whether the lost macroblocks intra-coded or inter-kopirovanymi to compensate for the missing macroblocks. After compensation, the video signal may then be displayed on the monitor. As a result, the present system and method provide a masking error signal arising from loss of data package.

BRIEF DESCRIPTION of DRAWINGS

Fig. 1 - table the resolution and the requirements for data transfer speeds for different formats, in accordance with known technical field.

Fig. 2 is a block diagram of an exemplary system of videoconfer the C-link according to the present invention.

Fig. 3 is a block diagram of an exemplary station video conferencing system shown in Fig. 2.

Fig. 4 is a block diagram of an exemplary variant of the device of the image processing shown in Fig. 3.

Fig. 5 is an exemplary diagram of a recording device template macroblock to format video standard one variant of the CIF format with reduced four times the resolution, where each room Wi is assigned to the macroblock in the corresponding spatial location.

Fig. 6 is an exemplary diagram of a macroblock of a frame of CIF format with reduced four times the resolution, as shown in Fig. 5, where the data packet containing coded data of the macroblock for the macroblock with Wi = 5 lost.

Fig. 7 is a block diagram of a two-dimensional interpolation scheme using data corresponding to pixels detected in the neighboring macroblocks, according to one variant of the present invention.

Fig. 8 is an exemplary block diagram of the neighboring macroblock used for estimating the motion vector of the lost macroblock m in accordance with the present invention.

Fig. 9 is an exemplary block diagram of stages of a method for processing video data according to one variant of the present invention.

Fig. 10 is an exemplary block diagram of stages of a method for masking the error signal when receiving video data according to Nast is Adamu invention.

DETAILED DESCRIPTION of DRAWINGS

The present invention provides a masking errors in the video signal due to loss of the data packet. The present system and method is based on existing technologies packaging macroblocks in a flexible (i.e., not raster) order in the frame. In contrast to existing standards video encoding macroblocks are packaged in the manner prescribed in the template reorder macroblock. In addition, the motion vectors for each macroblock can be coded not differential. These improvements are aimed at reducing disturbances caused by the loss of the data packet through the communication channel. Scope of the present invention covers various video standards, including, but not limited to, N. 261, N. 263, N. 264, MPEG, MPEG-2 and MPEG-4.

In Fig. 2 shows an exemplary video conferencing system 200.

The video conferencing system 200 includes a local station for video conferencing 202 and a remote station video conference 204, which are interconnected via a network 206.

Although in Fig. 2 shows only two stations for video conferencing 202 and 204, the experts in this field understand that several stations for video conferencing 202 can be connected to the video conferencing system 200. It should be noted that the present system and method can be used in any system tie is, where is the transmission of video information over the network. The network may be any type of electronic transmission media environment, including, but not limited to, standard telephone network, a regular cable, fiber optic devices and radios.

Fig. 3 is a block diagram of an exemplary station video conferencing 300. For simplicity, station video conferencing 300, here will be described the local station for video conferencing 202 (Fig. 2), although this configuration may be remote station video conferencing 204 (Fig. 2). In one embodiment, the station for video conferencing 300 includes a display 302, a CPU 304, a memory 306, at least one device for collecting video information 308, the imaging device 310 and an interface 312. Alternatively, the station conferencing system 300 can be installed in other devices or not all of the above devices can be used. At least one data collection device video 308 can be used as a semiconductor photosensitive matrix of charge-coupled device, additional Luggage structure of metal-oxide-semiconductor (CMOS), or any other type of device records the image.

The system uses at least one data collection device video 308, data acquisition is x user hall meetings or other scenes that are sent to the imaging device 310.

Typically, the imaging device 310 converts the video image into data packets before the interface 312 forwards the data packets to the remote station for video conferencing 204. The imaging device 310 will be discussed in more detail below with reference to Fig. 4. On the other hand, the imaging device 310 also converts data packets received from the remote station video conferencing 204, the video signal displayed on the display 302.

In Fig. 4 presents a sample processing device, the image 310 shown in Fig. 3. The imaging device 310 includes an encoding device 402, the device reorder macroblocks 404, device packaging macroblock 406 and the communication buffer 408. Original video from the collection device video data 308 (Fig. 3) is introduced in the encoding device 402, which converts each video frame in the desired format and converts each frame of the video signal into a set of macroblocks. The macroblock is a data module that contains data blocks, including the components of luminance and chrominance associated with picture elements (also referred to as pixels). For example, in the format H. 263 macroblock consists of look no further than the x data blocks brightness 8× 8 and two corresponding blocks of chroma data 8×8 in the format selection signal chroma 4:2:0. The data block of 8×8 is a matrix of eight columns and eight rows of data, where each data correspond to the pixel of the video frame. Macroblock format chroma signal 4:2:0, contains the data, has a section of the video from 16×16 pixels.

However, the present invention is not limited to traditionally defined macroblocks, but can be applied to any module that contains the data of the brightness and/or color. In addition, the scope of the present invention includes other formats sample, for example, the format selection signal chroma 4:2:2, containing four data block brightness 8×8 and four corresponding block color 8×8.

The data or format selection signal chroma 4:4:4 contain four data block brightness 8×8 and eight corresponding 8×8 data blocks of color.

In addition, the encoding device 402 encodes (i.e. compress) each macroblock to reduce the number of bits representing the data content.

Each macroblock may be intra-coded or inter-coded, and the frame may be composed of any combination of intra-coded and inter-coded macroblocks.

Inter-coded macroblocks are encoded using temporal similarity (i.e. similarity to the E. there are between the macroblock from the same frame and just agreed macroblock of the previous frame). Specifically, this inter-coded macroblock contains the encoded difference between the macroblock and a precise and coordinated macroblock of the previous frame. Exactly agreed macroblock of the previous frame may contain data associated with pixels that are displaced with respect to the pixels associated with the current macroblock.

On the other hand, intra-coded macroblocks coded without using information from other frames in a manner similar to the standard JPEG encoding used to display a still image.

For example, to determine whether the current macroblock to be coded as an inter-coded macroblock, the encoding device 402 calculates the difference between the data of the given macroblock of the current video frame with the data of the macroblock of the previous frame (called a macroblock offset), when the difference can be realized, for example, as Sredneuralskaya error or mean square error between the data corresponding to the pixels located in aligned positions in the macroblocks. For a given macroblock, the encoding device 402 calculates the error for a set of macroblocks offset. If the encoding device 402 finds only errors above a predetermined threshold difference, the essential similarity between dannysmith given macroblock and the data from the previous frame are missing, and the macroblock is intra-coded. However, if the detected error is less than a predefined threshold for a given macroblock and the macroblock offset from the previous frame, the macroblock is inter-coded.

For inter-coding a given macroblock, the encoding device 402 subtracts the data of the given macroblock of the macroblock data offset (i.e. brightness data and the color associated with the pixel of the given macroblock is subtracted from the brightness data and the color associated with the corresponding pixel of the macroblock offset for each pixel to receive the data difference. Then encode the difference data using a coding standard type of discrete cosine transform and quantization technique, determine the displacement vector from the given macroblock to macroblock offset (called a motion vector) and encode the motion vector.

Then by the standards of coding, type N. 261 and N. 263, finding that the motion vectors of the inter-coded macroblock should be encoded differentially, in order to improve the coding efficiency. However, differential encoding leads to the fact that the error generated by the lost or destroyed data, motion vectors are propagated to the neighboring macroblocks, which otherwise would have been decoded without error because the coded data of the motion vector, associated with the macroblock, are not independent from the data of the motion vectors of the neighboring macroblocks. Thus, the impact of these motion vectors of the given macroblock is not limited spatial data of the macroblock. However, if the motion vectors for each inter-coded macroblock is not coded, then the impact of these motion vectors is limited by the given macroblock, leading to a significant increase in resistance to external perturbations. In most cases, a change in the method of coding motion vector from the differential to dedifferentiate methods leads to a small loss in coding efficiency (typically less than a few percent). According to one variant of the present invention the components of the motion vector associated with each inter-coded macroblock, as a rule, differentially encoded, as opposed to the conventional methods.

In another embodiment of the invention, the encoding device 402 can encode intra-coded macroblocks of the frame, using the mechanism of "total recovery". The mechanism of the "total recovery" - a deterministic mechanism that serves to remove the mismatch of the reference frame, called the drift data by intra-coding a specific pattern of macroblocks for each frame. In the encoding device 402 using the I macroblocks of the reference frame, as macroblocks offset by decoding the inter-coded macroblocks of the current frame. In one embodiment of the invention the mechanism of the "total recovery" is used for intra-coding pattern of the macroblock, using the interval integer w, selected from a set of predefined integer intervals "total recovery". For example, if w = 47, the encoding device 402 intra-encodes each macroblock. Interval total recovery can be selected on the basis of the transmission rate data and error rate. When "restored " intra-coded macroblocks are received in the encoding device remote station video conferencing 204 (Fig. 2), these "recovered" data replaces the corresponding macroblock from the previous frame, which could be destroyed because of errors in the transmission of video information.

Any macroblock that can be destroyed because of errors in the transmission of video (and not replaced) further extends and can increase the drift data when the encoding device remote station video conferencing 204 are damaged macroblock as the reference macroblock for decoding received other macroblocks.

Thus, intra-coded macroblocks "total recovery" provide a device of codiovan the remote station video conferencing 204 "fresh" set of intra-coded macroblocks, you can use as reference macroblocks, reducing, thus, the distribution of drift data.

In addition, the encoding device 402 can generate a map of the intra-macroblocks, which determines the macroblocks in the video frame are intra-coded. After building the map of intra-macroblocks, the imaging device 310 sends the card remote station video conferencing 204. The card can be sent as part of image header associated with, for example, an encoded video frame, although you may be used and other fields.

In accordance with the present invention, the encoding device 402 can generate a map of the intra-macroblocks in two ways. In one embodiment of the invention in the coding block 402 is used coding with variable length string to describe the location of intra-coded macroblocks within the frame. Encoding with variable-length string is a technique that allows to reduce the size of the repeating string of characters. In another embodiment of the invention, the encoding device 402 generates a bitmap, where each bit in the bitmap corresponds to one macroblock of the frame. The value of the bit determines the encoding type of the respective macroblock. For example, in one embodiment of the invention the bit "1" means that according to the current macroblock intra-coded. In another embodiment of the invention the bit " 1" means that the corresponding macroblock is inter-coded. In the present invention can be used and other ways of building the map of intra-macroblock.

In yet another variant of the invention, the encoding device 402 selects the encoding map intra-macroblock that uses fewer bits. For example, at a resolution of 352×288 pixels (i.e. a resolution of 352 pixels horizontally 288 vertical pixels) video frame contains 396 macroblocks, configured in the form of a matrix of macroblocks 22×18. Without including any required additional bit bitmap encoding method requires 396 bits (one bit for each macroblock). Thus used 396 bits to transmit the raster image of the map intra coded macroblock, regardless of the number of intra-coded macroblocks within the frame format FCIF. On the contrary, the number of bits used to transmit card microblocks, coded by the length of the string depends on the number of intra-coded macroblocks in the frame FCIF. The cost of the card transfer intra-macroblock coded by the string length is eight bits in intra-coded macroblock (i.e. eight bits on the value of the series), where the value of the series determines the location of the intra-coded macroblock within the Adra FCIF. Therefore, if the frame FCIF contains n intra-coded macroblocks, you will need 8n bits to transmit the encoded length map intra-macroblocks.

Thus, if the frame CIF contains less than 50 intra-coded macroblocks (n<50), the device coding source 402 of the choice of the method of encoding with variable-length rows, i.e. a device for encoding source 402 selects the encoding length of the string. The choice of encoding maps intra-macroblock depends on the video format, FCIF video frame is one example.

Then coded macroblocks are transferred to the unit reorder macroblocks 404. The device reorder macroblocks 404 reorder the encoded macroblocks. Specifically, each macroblock is assigned an identifier of a group of macroblocks (IGM) from a variety of Wi. In an exemplary embodiment, the macroblocks are numbered up to six according to the approximate distribution pattern of macroblocks shown in Fig. 5 for frame formatted with the format CIF reduced four times the resolution (QCIF), having nine rows by eleven macroblocks in the row. Maximum Wi is called the maximum group ID Wi. In Fig. 5 approximate variant Wi = 6. As shown in the drawing, Wi are assigned so as to minimize the assignment of the same neighbouring Wi the m macroblocks.

Alternatively, the same Wi can be assigned to other templates neighboring macroblocks or in any other order.

As will be discussed below with reference to Fig. 6, the distribution of macroblocks so that the neighboring macroblocks will not be assigned to the same Wi basically minimizes the concentration of errors in one area of the frame, because the macroblocks of the lost data packet is spatially distributed across the frame. Because errors due to lost packages will not be concentrated in one area of the frame, lost data associated with the lost macroblocks can be more accurately recovered by using data of neighboring macroblocks. In other words, the spatial interpolation of data from neighboring macroblocks or estimate the motion vectors of the macroblock will be more accurately determined if the loss of data within the spatial frame is not limited.

In the coding block 402 (Fig. 4) the processing device of the image 310 (Fig. 3) the remote station video conferencing 204 (Fig. 2) may be used various ways of masking errors together with the macroblock reordering to improve the video quality. For example, in one embodiment of the invention, the encoding device 402 decodes the neighboring macroblocks of the lost inter-coded Mac is oblaka, estimates the motion vector of the lost macroblock and then uses the estimated motion vector to reconstruct the data lost macroblock.

In another embodiment of the invention, the encoding device 402 may decode the neighboring macroblocks of the lost intra-coded macroblock and to spatially interpolate the decoded data of neighboring macroblocks to recover lost data. Scope of the present invention covers other methods of masking errors, used in combination with the macroblock reordering to improve the video quality due to lost or destroyed macroblocks.

In accordance with the present invention can be used, and other reorders templates and IGM. In one embodiment of the invention, the device reorder macroblocks 404 operates on the basis of selection of MGID, based on assessment data and/or video format.

As shown in Fig. 4, after the macroblocks have been assigned to Wi, device packaging macroblock 406 places the macroblocks in discrete data packets according to their Wi. Thus, the macroblocks with the same Wi (i.e. Wi = 1) will be placed in the common package of discrete data (i.e. the data packet 1). In Fig. 5 presents an exemplary variant of the invention in which the device packaging 406 is asmodae macroblocks in six data packets in the frame of QCIF format. However, the device packaging 406 may use more than one data packet from a data Wi to transport the macroblocks with this specific IGM. For example, device packaging 406 may generate the first data packet 1 containing a portion of macroblocks with Wi = 1, and the second data packet 1 with the remaining macroblocks with Wi = 1. Splitting packages in this way, as a rule, depends on the maximum size of a portable module (MRM)associated with the network 206 (Fig. 2).

Next, the data packets and the header image is transmitted in the communication buffer 408 for transmission via the network 206 (Fig. 2) using the interface 312 (Fig. 3). To increase robustness against packet losses, the header image can be transmitted more than once per frame. The image header may include a map of the intra-macroblocks.

In addition, the imaging device 310 also handles packets of video information received from the remote station and provides video signals for display. First packets of video information received on the interface 312 (Fig. 3) and then passed on to the communication buffer 408. Then the packets of video data transmitted to the device packaging macroblock 406, which decompresses the macroblocks. The device then reorder macroblocks 404 embeds the macroblocks in their original ordered the template (i.e. the template up to reordering of the macroblock on the remote station for video conferencing 204, which, as a rule, is a raster scan (Fig. 2)).

After that, the encoding device 402 operates as a decoder and determines whether the packet of video data is lost during transmission through the network 206. In Fig. 6 presents a scheme for reordering the pattern of the macroblock of the frame QCIF shown in Fig. 5, when the lost data packet containing coded data of the macroblock for the macroblock with Wi = 5. Lost macroblocks indicated by the symbol "x". It should be noted that in one embodiment of the invention the lost macroblocks predominantly spatially distributed across the frame QCIF, providing, thus, a precise and relatively simple method of masking errors, using operations such as spatial interpolation or estimation of motion vector compensation. Although for ease of discussion of Fig. 6 shows only one lost data packet, the scope of the present invention includes masking of errors in any number of data packets that are corrupted or lost during transmission. In addition, it should be noted that although described here, the same components that perform the functions of transmission and reception, these components can be implemented as a separate receivers and transmitters.

As shown in Fig. 4, for each lost macroblock, the encoding device 402 determines whether p is Teryan the macroblock is intra-coded or inter-coded.

For example, the encoding device 402 can check map of intra-macroblocks to determine whether the lost macroblock is intra-coded. As mentioned above, the map intra-macroblocks can be entered in the header field of the image or as additional information submitted outside of the video stream and can be compressed using an encoding algorithm with variable length of the string, with configlet bitmap that identifies intra-coded macroblocks, or using other effective methods of encoding.

If the lost macroblocks are intra-coded, then you can use several ways to mask errors. If, for example, the lost macroblock is intra-coded, as part of the mechanism of the "total recovery", the encoding device 402 replaces the lost macroblock content "appropriate" macroblock of the previous frame, where the two "relevant" macroblock cover the same spatial area of their frames. According to the present invention the speed of the cleaning mechanism of the "total recovery" is a function of data rate and error rate.

Alternatively, if the lost intra-coded macroblock is not coded as part of the mechanism of the "total recovery", the encoding device 402 spatial interpolare the contents of the lost macroblock of the neighboring macroblocks. In one embodiment of the invention each 8x8 block of the lost macroblock spatial interpolated between the two closest neighboring macroblocks. In Fig. 7 shows an exemplary diagram of interpolation using the data associated with pixels located in adjacent macroblocks. Fig. 7 includes a lost macroblock 705, the left adjacent macroblock 710, the upper adjacent macroblock 715 and right adjacent macroblock 720. For example, for recovery (i.e. interpolation) data for the upper-left block of 8×8 725 lost macroblock 16×16, an encoding device 402 (Fig. 4) interpolates the data in the last column of data 730 (indicated by the symbol ×) from the upper-right block of 8×8 735 left adjacent macroblock 710, and data in the last data row 740 (indicated by the symbol ×) from the lower left block of 8×8 745 upper adjacent macroblock 715.

Similarly, to restore the data to the upper-right block of 8×8 lost macroblock 705 encoding device 402 interpolates the data in the first data column 755 upper-left block of 8×8 upper right adjacent macroblock 760 right adjacent macroblock 720 and the data in the last data row 765 bottom right block 770 upper adjacent macroblock 715. Can be used with other types of interpolation and other neighboring blocks of the macroblocks within the framework of the present invention.

If the lost macroblock is inter-coded, the encoding device 402 calculates and estimates the motion vector of the lost macroblock, checking the motion vectors of the neighboring macroblocks. In Fig. 8 presents a block diagram of the neighboring macroblock used for estimating the motion vector of the lost macroblock m, according to one variant of the present invention. For the lost macroblock m is calculated median of motion vectors of three neighboring macroblocks a, b, and C.

For example,

x component of the estimated motion vector of the macroblock m - MVmX = median (MVaX, MVbX, MVCx) and the component of the estimated motion vector of the macroblock m - isMVmy = median (MVaY, MVby, y), where MVaX, MVbX, MV X - component x of the motion vectors of the macroblocks a, b and C, respectively, of a MVay, MVby MVy - components of the motion vectors of the macroblocks a, b and C, respectively. Although in the embodiments of Fig. 8 uses the motion vectors of neighbouring macroblocks a, b and C, to calculate the estimated motion vector for the macroblock m, you can use any number and any combination of the neighboring macroblocks to estimate the motion vector of the lost macroblock.

After the estimated motion vector of the lost macroblock, the encoding device 402 (Fig. 4) compensates for the lost macroblock, using the estimated motion vector to restore the contents of the lost macroblock. PEFC is how the restored data content of all the lost macroblocks of the frame, the encoding device 402 converts the macroblocks in the video signal for display on the display of the display device 302 (Fig. 3). Although in Fig. 4 shows only one lost data packet, the present invention can be used for masking errors with multiple lost packets of data.

In Fig. 9 presents an exemplary block diagram 900 of stages of a method for masking the error signal when transmitting packets of video information in circuit switched networks, according to one variant of the present invention. At stage 905 gathering device video data 308 (Fig. 3) captures the video image and generates a video signal.

Then, at stage 910, the encoding device 402 (Fig. 4) (also referred to as "encoder" when processing data for transmission) receives the signal and converts the signal into one or more intra-coded and inter-coded macroblocks. The video frame may contain inter-coded macroblocks, intra-coded macroblocks, or any combination of intra-coded and inter-coded macroblocks. In one embodiment of the invention the mechanism of the "total recovery" is applied to the pattern of intra-code the macroblock using the interval of the General recovery, selected from a set of predefined intervals total recovery. Interval total recovery can be selected on the Snov, the data transfer speed and frequency of errors. In addition, the encoding device 402 calculates not differentially coded motion vectors for each inter-coded macroblock.

Then, at stage 915, the encoding device 402 generates a map of intra-macroblocks, which determines the location of intra-coded macroblocks.

In one embodiment of the present invention map intra-macroblock is encoded using either method of encoding with variable-length string or bitmap encoding method, based on the total number of bits required to encode the card with intra-macroblocks.

The device then reorder macroblocks 404 (Fig. 4) at the stage 920 assigns each macroblock Wi. For example, macroblocks can be assigned to Wi in the template, as shown in Fig. 5. In one embodiment, the macroblocks are assigned to different Wi to minimize the number of neighboring macroblocks assigned to the same Wi. An alternative may be considered other options, when the neighboring macroblocks assigned to the same Wi.

Then the device packaging macroblock 406 (Fig. 4) creates the discrete data packets and places the macroblocks in discrete data packets according to their Wi at stage 925. For example, macroblocks with the same Wi can be placed in the common package of discrete data.

Alternatively, the moustache is the device packaging macroblock 406 may be a data device, which translates macroblocks in a specific format for transmission over the network is circuit-switched. Finally, at stage 930, the data packets and the header image (including a map of intra-macroblocks) are transmitted in the communication buffer 408 (Fig. 4) for transmission to a remote station for video conferencing 204 (Fig. 2).

In Fig. 10 presents an exemplary block diagram illustrating the progress of a method for masking the error signal when receiving video information, according to the present invention. At stage 1005 communication buffer 408 (Fig. 4) receives data packets transmitted from the remote station video conferencing 204 (Fig. 2) via the network 206 (Fig. 2). Then, at stage 1010, the device packaging macroblock 406 (Fig. 4) decompresses the received data packets into macroblocks.

Then, at stage 1015, the device reorder macroblocks 404 (Fig. 4) arranges the macroblocks and places the macroblocks in the proper spatial configu within the video frame.

Then, the encoding device 402 (Fig. 4) decodes the macroblocks at the stage 1020. The encoding device 402 (functioning as a decoder) or some other mechanism associated with the package of video data, converts (i.e. processes the data in real time), and in step 1025 determines are any macroblocks containing the video frame. Macroblocks loss is s, if one or more packets of video information is lost or destroyed during the packet transmission of video information over the network 206. If at the stage 1025 determined that no macroblock is not lost on stage 1030 macroblocks are displayed by the display device 302 (Fig. 3).

However, if at the stage 1025 determines that one or more macroblocks are missing, data associated with one or more lost macroblocks, on stage 1035 reversed depending on the type of encoding of the macroblocks. The encoding device 402 may use map intra-macroblocks to determine the type of encoding each lost macroblock.

For example, if the lost macroblock is intra-coded as part of the mechanism of the "total recovery", the encoding device 402 replaces the contents of the lost macroblock content data corresponding macroblock of the previous frame. Alternatively, if the lost intra-coded macroblock is not coded as part of the mechanism of the "total recovery", the contents of the lost macroblock spatial interpolated from the neighboring macroblocks. In one embodiment of the present invention, in the encoding device 402 is used two-dimensional interpolation to interpolate the data of neighboring macroblocks (Fig. 7).

Alternative is, if the lost macroblock is inter-coded, the encoding device 402 estimates the motion vector of the lost macroblock, checking the motion vectors of the neighboring macroblocks. In one embodiment of the invention, the motion vector is calculated as the median of the three motion vectors of neighboring macroblocks (Fig. 8). The encoding device 402 then uses the estimated motion vector to compensate for the data content of the lost macroblock by revising content evaluation data lost macroblock.

After restoring the data lost macroblocks these macroblocks are displayed on the display device 302 at the stage 1025.

The invention is described above in the exemplary embodiments. For specialists in this area it is obvious that can be done various modifications of the invention without departing from the spirit and scope of the invention. In addition, although the invention has been described in the context of its implementation in a particular environment and for particular applications, specialists in this field understand that the usefulness of the present invention is not limited to these applications, and that the invention can be used in any equipment in the different fields of application. Accordingly, the above description and drawings should be considered more illustrative than a restrictive sense is.

1. System for processing video information containing the encoding device for processing each frame of the video signal to generate macroblocks for encoding these macroblocks; a device for reordering the macroblocks for the assignment ID of the group of macroblocks (IGM) from a variety of Wi for each encoded macroblock and the device packaging macroblocks for the placement of each of the encoded macroblocks in a specific data packet according to the IHM macroblock.

2. The system according to claim 1, in which the device reorder macroblocks assigns a different Wi neighboring coded macroblocks.

3. The system according to claim 1, in which the encoding device generates adifferential coded motion vectors for each of the encoded macroblocks, which is internatioanal the macroblock.

4. The system according to claim 1, in which the device packaging macroblocks places coded macroblocks with differentiating assigned IGM in different data packets.

5. The system according to claim 1, in which the IHM has a value from 1 to the maximum group ID (MIGM).

6. The system according to claim 5, in which the device reorder macroblocks defines MIGM based on the transmission speed of video data.

7. The system according to claim 5, in which the device reorder macroblocks definition is no MIGM-based video format.

8. The system according to claim 1, in which the encoding device encodes the macroblock of the current frame as an intra-coded macroblock, if there is a significant difference between the data of the given macroblock of the current frame and the data are most consistent macroblock offset of the previous frame.

9. The system according to claim 1, in which the encoding device encodes the macroblock of the current frame as an inter-coded macroblock, if the essential similarity between the data of the given macroblock of the current frame and the data are most consistent macroblock offset of the previous frame.

10. The system according to claim 1, in which the encoding device additionally generates a map of intra-macroblocks determining intra-coded macroblocks in a given frame.

11. The system of claim 10, in which the encoding device encodes the map intra-macroblocks by selecting the encoding map intra-macroblocks, which generates the smallest number of bits.

12. The system of claim 10, in which the device uses the encoding with variable-length rows to encode map intra-macroblocks.

13. The system of claim 10, in which the encoding device uses the bitmap to encode the map intra-macroblocks.

14. The system according to claim 1, in which the device packaging macroblocks receiving device, raspal which indicates the encoded macroblocks.

15. The system according to claim 1, in which the device reorder macroblocks receiving device arranges the encoded macroblocks.

16. The system according to claim 1, in which the encoding device to the receiving device decodes the encoded macroblocks and detects lost decoded macroblocks.

17. System according to clause 16, in which the device for encoding a receptor spatially interpolates the data of this lost decoded macroblock data from neighboring decoded macroblocks to mask the influence of the error signal, if this lost decoded macroblock was encoded as intra-coded macroblock.

18. System according to clause 16, in which the encoding device to the receiving device estimates the motion vector of this lost decoded macroblock based on the motion vectors of the neighboring decoded macroblocks to restore the data content of this lost decoded macroblock through motion compensation for the masking effect of the error signal, if this lost decoded macroblock was encoded as an inter-coded macroblock.

19. The system according to claim 1, in which the device packaging macroblock is a transmission device for the placement of each of the encoded macroblocks in accordance with the GM in a specific format for transmission through the network circuit switched channels.

20. Method for processing video data, comprising the following stages:

the processing of each frame of the video signal to generate macroblocks; encoding macroblocks; assigning each encoded macroblock group ID of microblocks (IGM) from a variety of Wi and placing each of the encoded macroblocks in a specific data packet in accordance with the IHM.

21. The method according to claim 20, in which stage of the assignment stage further comprises assigning different Wi neighboring coded macroblocks.

22. The method according to claim 20, further containing phase formation adifferential coded motion vectors for each of the encoded macroblocks, which is inter-coded macroblock.

23. The method according to claim 20, in which the stage of placing further comprises the stage of placement of coded macroblocks with different assigned IGM in different data packets.

24. The method according to claim 20, in which the IHM has a value from 1 to the maximum group ID (MIGM).

25. The method according to item 23, further containing phase definition MIGM based on the transmission speed of video data.

26. The method according to item 23, further containing phase definition MIGM based on the video format.

27. The method according to claim 20, in which the phase encoding further comprises stadiumlovee a given macroblock of the current frame as an intra-coded macroblock, if there is a significant difference between the data of the given macroblock of the current frame and the data are most consistent offset of the macroblock of the previous frame.

28. The method according to claim 20, in which the phase encoding further comprises phase encoding of the given macroblock of the current frame as an inter-coded macroblock, if there is substantial similarity between the data of the given macroblock of the current frame and the data are most consistent offset of the macroblock of the previous frame.

29. The method according to claim 20, additionally containing the stage of building the map of intra-macroblocks, identifying intra-coded macroblocks in a given frame.

30. The method according to clause 29, additionally containing phase selection method of encoding maps intra-macroblocks, which generates the smallest number of bits.

31. The method according to clause 29, additionally containing the stage of use of encoding with variable-length rows to encode map intra-macroblocks.

32. The method according to clause 29, additionally containing the stage of use raster to encode map intra-macroblocks.

33. The method according to claim 20, further containing stage decompression of encoded macroblocks.

34. The method according to claim 20, further containing phase ordering of encoded macroblocks.

35. The method according to claim 20, further containing stage zakodirovana is coded macroblocks and detection of lost decoded macroblocks.

36. The method according to p, optionally containing phase spatial interpolation of data of this lost decoded macroblock data from neighboring decoded macroblocks to mask the influence of the error signal, if this lost decoded macroblock was encoded as intra-coded macroblock.

37. The method according to p, optionally containing stage estimates the motion vector of this lost decoded macroblock based on the motion vectors of the neighboring decoded macroblocks to restore the data content of this lost decoded macroblock by the motion compensation for the masking effect of the error signal, if this lost decoded macroblock was encoded as an inter-coded macroblock.

38. The method according to claim 20, in which the stage of placing further comprises input stage of each of the encoded macroblocks in accordance with the IHM in a specific format for transmission over the network circuit switched channels.

39. System for processing video information containing:

the device is decompressed macroblocks designed to receive multiple data packets, with each packet contains a number of macroblocks having the group ID of the macroblock (Wi), the specified device is also intended for WPI is ecene macroblocks of the packages;

the device reordering of macroblocks designed for reordering the extracted macroblock in a predetermined order; and

a decoding device that is designed to handle reordered macroblocks and convert them to video.

40. System 39, in which the predetermined order is the order of raster scan.

41. System 39, in which the set of data packets includes adifferential coded motion vectors for each of the encoded macroblocks, which is inter-coded macroblock.

42. System 39, in which each package contains macroblocks having the General IGM.

43. The system according to claim 1, in which the range of the Wi is in the range from 1 to the maximum group ID (MIGM).

44. The system according to item 43, in which MIGM corresponds to the transmission speed of video data.

45. The system according to item 43, in which MIGM corresponds to the video format.

46. System 39, in which the set of data packets includes a map of intra-macroblocks, determining intra-macroblocks in a given frame.

47. The system according to item 46, in which the card intra-macroblocks encoded using encoding with variable-length rows.

48. The system according to item 46, in which the card intra-macroblocks encoded using a raster display.

49 System according to item 46, in which the decoding device determines coded whether the map intramacrophagic using encoding with variable-length string or bitmap display, and processes the macroblocks accordingly.

50. System 39, in which the device decoding macroblocks detects macroblocks, dropped out of the reordered macroblocks, and recovers lost data.

51. The system according to item 50, in which the device decoding spatially interpolates the data from the lost macroblock data from neighboring decoded macroblocks, if the lost macroblock was encoded as intra-coded macroblock.

52. The system according to item 50, in which the decoder estimates the motion vector of this lost macroblock based on the motion vectors of the neighboring decoded macroblocks to restore the data content of this lost macroblock by the motion compensation, if this lost decoded macroblock was encoded as an inter-coded macroblock.

53. Method for processing video information containing the following stages:

receiving multiple data packets in which each packet of data contains a number of macroblocks, with each macroblock has a group ID of macroblocks (Wi);

raspakovyvaniya macroblocks;

the reordering of the received macroblock in a predetermined order different from the order in which they were received; and

decoding reordered macroblocks to generate a video signal.

54. The method according to item 53, in which the predetermined order is the order of raster scan.

55. The method according to item 53, wherein a set of data packets contains dedifferentiate coded motion vectors for each of the encoded macroblocks, which is inter-coded macroblock.

56. The method according to item 53, in which the macroblocks in a particular packet of data have a common Wi.

57. The method according to item 53, in which the range of the Wi range Wi is in the range from 1 to the maximum group ID (MIGM).

58. The method according to 57, in which MIGM corresponds to the transmission speed of video data.

59. The method according to 57, in which MIGM corresponds to the video format.

60. The method according to item 53, wherein a set of data packets contains a map of intra-macroblocks, determining intra-macroblocks in a given frame.

61. The method according to p in which map intra-macroblocks encoded using encoding with variable-length rows.

62. The method according to p in which the card is intra-coded macroblocks using a raster display.

63. The method according to item 53, further comprising phase detection the Oia lost macroblocks.

64. The method according to p, further comprising a stage of spatial interpolation of data given the lost macroblock data from neighboring decoded macroblocks, if the lost macroblock was encoded as intra-coded macroblock.

65. The method according to p additionally includes the stage of estimating the motion vector of this lost macroblock based on the motion vectors of the neighboring macroblocks to restore the data content of this lost macroblock by the motion compensation if the lost macroblock was encoded as an inter-coded macroblock.



 

Same patents:

FIELD: electrical communications; data digital computation and processing including reduction of transferred information redundancy.

SUBSTANCE: proposed message compression and recovery method includes pre-generation of random quadrature matrix measuring m x m constituents and k random key matrices measuring N x m and m x N constituents on transmitting and receiving ends, and generation of quantum reading matrix of fixed half-tone video pattern measuring M x M constituents. Matrices obtained are transformed to digital form basing on addition and averaging of A images, each image being presented in the form of product of three matrices, that is, two random rectangular matrices measuring N x m and m x N constituents and one random quadrature matrix measuring m x m constituents. Transferred to communication channel are constituents of rectangular matrix measuring N x m constituents. Matrix of recovered quantum readings of fixed half-tone video pattern measuring M x M constituents is generated basing on rectangular matrix measuring N x m constituents received from communication channel as well as on random quadrature matrix measuring m x m constituents and random rectangular matrix of m x N constituents, and is used to shape fixed half-tone video pattern.

EFFECT: enhanced error resistance in digital communication channel during message compression and recovery.

2 cl, 26 dwg, 1 app

FIELD: technology for encoding multimedia objects.

SUBSTANCE: method for encoding a multimedia object includes following stages: multimedia object is encoded for producing a bit stream and information about quality is added to bit stream, while information about quality denotes quality of multimedia object relatively to given position or relatively to given part of bit stream, while information about quality is provided in quality tags, aforementioned quality tag provides a values of quality tag, and value of quality tag characterizes distortion in encoded multimedia object being reproduced, when bit stream is truncated in point, related to quality tag.

EFFECT: development of improved and efficient method/system for encoding multimedia objects.

13 cl, 2 dwg

FIELD: data filtration technologies, in particular, signaling adaptive filtration for lower blocking effect and contour noise.

SUBSTANCE: during forming of frame of blocks of given size, following operations are performed: production of blocking information for decreasing blocking noise and production of contouring information for decreasing contour noise of coefficients of previously given pixels of upper and left threshold areas of data block, when frame, received by decomposition of image data in the stream of binary digits for inverse quantizing is an internal frame, and adaptive filtration of image data passing through inverse quantizing and inverse discontinuous cosine transformation, in accordance to produced blocking information and contouring information. Thus, blocking effect and contouring noise can be removed from image, restored from image on basis of blocks, improving the image restored from compression.

EFFECT: decreased blocking effect and contouring noise of encoding with high compression level.

2 cl, 7 dwg

FIELD: data filtration technologies, in particular, signaling adaptive filtration for lower blocking effect and contour noise.

SUBSTANCE: during forming of frame, following operations are performed: production of blocking information for decreasing blocking noise and production of contouring information for decreasing contouring noise of coefficients of previously given pixels of upper and left threshold areas of data block, when frame, received by decomposition of image data in the stream of binary digits for inverse quantizing is an internal frame, and adaptive filtration of image data passing through inverse quantizing and inverse discontinuous cosine transformation, in accordance to produced blocking information and contouring information. Thus, blocking effect and contouring noise can be removed from image, restored from image on basis of blocks, improving the image restored from compression.

EFFECT: decreased blocking effect and contouring noise of encoding with high compression level.

2 cl, 7 dwg

FIELD: technologies for data filtering.

SUBSTANCE: when a frame is formed of blocks of preset size, following operations are performed: generation of blocking information for decrease of blocking effect and contouring information for decrease of contouring noise from coefficients of preset pixels of upper and left limiting areas of data block, when a frame, received by decomposition of image data in a stream of binary bits for inverse quantizing, is an inner frame, and adaptive filtering of image data, passing through inverse quantizing and inverse discontinuous cosine transformation, in accordance to generated information of blocking and information of contouring. That is why blocking effect and contouring noise can be removed from an image, restored from image on basis of blocks, to improve the image, restored from compression.

EFFECT: decreased blocking effect and contouring noise.

2 cl, 7 dwg

FIELD: electrical communications; data processing including reduction of data redundancy.

SUBSTANCE: proposed process includes similar way of generation of random quadrature matrix measuring m x m items and k random key matrices measuring N x m and m x N items on sending and receiving ends. Then k matrices of quantum readings of motionless gray-level video picture measuring M x M items are formed from k motionless gray-level video pictures which are then converted into product of three following matrices: random rectangular matrix measuring N x m items, random square matrix measuring m x m items, and random rectangular matrix measuring m x N items; in the process items of rectangular matrix measuring N x m items are transferred to communication channel. On receiving end k matrices of recovered quantum readings of motionless gray-level video pictures measuring M x M items are formed around random matrix measuring N x m items received from communication channel, as well as around random quadrature matrix measuring m x m items, and random rectangular matrix measuring m x N items, and motionless gray-level video pictures are produced from mentioned k matrices of recovered quantum readings.

EFFECT: enhanced data transfer speed at desired quality of recovered messages.

4 cl, 24 dwg

The invention relates to a digital camcorder, and more particularly to a device is a digital camcorder that uses methods of video compression that is compatible with the decoding device MPEG-2

The invention relates to filter the data, in particular to the way the signal adaptive filtering for reducing blocking effect and noise trim, signal adaptive filter and a machine-readable medium for storing program

FIELD: engineering of systems for encoding moving image, namely - methods for encoding moving image, directed at increase of encoding efficiency with use of time-wise remote supporting frames.

SUBSTANCE: in the method in process of encoding/decoding of each block of B-frame in direct prediction mode movement vectors are determined, using movement vector of shifted block in given frame, utilized for encoding/decoding B-frame, and, if type of given frame is time-wise remote supporting frame, one of movement vectors, subject to determining, is taken equal to movement vector of shifted block, while another one of movement vectors, subject to determining, is taken equal to 0.

EFFECT: increased encoding efficiency in direct prediction mode, decreased amount of information bits for frame, wherein a change of scene occurs.

2 cl, 6 dwg

FIELD: engineering of systems for encoding moving images, namely, methods for encoding moving images, directed at increasing efficiency of encoding with use of time-wise remote supporting frames.

SUBSTANCE: method includes receiving index of supporting frame, standing for supporting frame, pointed at by other block, providing movement vector for determining movement vector of current block, and determining movement vector of current block with utilization of supporting frame index, denoting a supporting frame.

EFFECT: increased efficiency of encoding in direct prediction mode, decreased number of information bits for frame, in which scene change occurs.

3 cl, 6 dwg

FIELD: television.

SUBSTANCE: device has blocks: first interface block, providing receipt of data about switching of programs by subscriber, electronic watch block, first memory block for archiving data about time of viewing of each selected program, second memory block, containing electronic addresses of broadcast companies, block for rearranging data about viewing time, processor, forming packet of data about which TV program and time of its viewing, third interface block, providing output along phone network of data about viewing time of each TV program to server of company, which broadcast current TV program.

EFFECT: higher efficiency.

1 dwg

FIELD: television.

SUBSTANCE: device has scaling block, two delay registers, block for forming pixel blocks, buffer register, block for calculating movement vectors, two subtracters, demultiplexer, enlargement block, pulsation filtering block, mathematical detectors block, multiplexer, reverse scaling block, as a result of interaction of which it is possible to detect and remove some series of TV frames from programs, which cause harmful effect to viewer, specifically pulsations of brightness signals and color signals with frequency 6-13 Hz.

EFFECT: higher efficiency.

1 dwg

The invention relates to radio engineering and is intended for the discharge of compressed clock signal to a device for separating signal and the clock signal, essentially agreed with a synchronizing signal encoding device

The invention relates to the fields of electronics, communications, computer science, television, interactive television, video telephony and video conferencing

The invention relates to a method of dimensional television broadcast in the frequency band flat TV

The invention relates to image compression to reduce bandwidth requirements of digital video decoder

The invention relates to a noise reduction device in the system of video coding

FIELD: television.

SUBSTANCE: device has scaling block, two delay registers, block for forming pixel blocks, buffer register, block for calculating movement vectors, two subtracters, demultiplexer, enlargement block, pulsation filtering block, mathematical detectors block, multiplexer, reverse scaling block, as a result of interaction of which it is possible to detect and remove some series of TV frames from programs, which cause harmful effect to viewer, specifically pulsations of brightness signals and color signals with frequency 6-13 Hz.

EFFECT: higher efficiency.

1 dwg

Up!