Image processing method and apparatus

FIELD: information technology.

SUBSTANCE: image composed of macroblocks measuring 16x16 is selected from a reference frame, wherein each macroblock is assigned a pixel band with width "a", which serves as field region, as a motion-compensated image and is considered as the input image for the filtration process. The value "a" is defined according to the number of filter branches with a finite impulse response. The filtration process is performed using a motion-compensated image as the input image and the predicted image measuring 16x16 pixels is transmitted to the output as the output image of the filtration process. The predicted image is added in an adder to the output image of the inverse orthogonal transformation circuit and the summation result is used as a macroblock making up the decoded frame.

EFFECT: generating a predicted image with high accuracy without increasing processor load.

6 cl, 1 dwg

 

The technical field to which the invention relates.

The present invention relates to a method and apparatus for image processing, and more specifically to a method and apparatus for image processing, which allow to generate a projected image with high accuracy without increasing the processing load.

The level of technology

Usually as encoding schemes for processing moving images used algorithms coding using motion compensation such as MPEG (Group of experts on moving images) or NH, and orthogonal transformation such as discrete cosine transformation, the transformation of karunen-Loev or wavelet transformation (the transformation of elementary waves). In such methods of encoding moving images reducing the amount of code is achieved by using the correlation in the spatial direction and the time axis between the characteristics of the input image signal, which is subject to encoding.

For example, according to the standard H.264, apply unidirectional or bidirectional prediction in the process of generating the inter-frame, i.e. the frame that should be subjected to processing inter-frame prediction (inter prediction) using the correlation in the direction of the time axis. the ri interframe prediction to generate predicted image frame based taking place at different points in time.

Figure 1 presents a diagram illustrating an example of a unidirectional prediction.

As shown in figure 1, when you encode the frame R0i.e. the frame that corresponds to the current point in time, generate using unidirectional prediction, motion compensation is carried out by using, as reference frames, already encoded frame corresponding to past or future time relative to the current time. The residual error between the predicted image and the actual image code using the correlation in the direction of the time axis, which reduces the amount of code. Information of the reference frame and the motion vector are used, respectively, as information that defines the reference frame, and information that defines the reference position in the reference frame, and transmits these pieces of information from the coding side decoding side.

Here the number of reference frames is not necessarily equal to one. For example, according to the standard H.264, you can use several frames as the reference frames. As shown in figure 1, if two frames arranged in time closer to the non-coding frame R0marked as keyframes R0and R1, pixel values is arbitrary macroblock in the composition of the to-be-encoded frame R 0you can predict on the basis of values of arbitrary pixels in the reference frame R0or R1.

Figure 1 rectangles allocated within each frame represent macroblocks. If you want to encode the frame R0the macroblock that you want to predict is the macroblock MBP0then in the reference frame R0this macroblock MBP0corresponds to the macroblock MBR0determined by the motion vector MV0. In addition, in the reference frame R1the considered macroblock corresponds to a macroblock MBR1determined by the motion vector MV1.

If the pixel values in the macroblock MBR0and MBR1(values of pixels in the image motion compensation) to denote MC0(i, j) and MC1(i, j), since the values of the pixels of any image motion compensation are used as values of pixels of the projected image for the case of unidirectional prediction, the predicted image Pred(i, j) is represented by equation (1) below. Here (i, j) denotes the relative position of the pixel in the macroblock and satises 0≤i≤16 and 0≤j≤16. In equation (1), the sign “||” indicates that only take one of the values MC0(i, j) or MC1(i, j).

Pred(i,j )=MC0(i,j)||MC1(i,j)...(1)

Note that you can also split one macroblock of size 16×16 pixels into smaller block sizes, for example, 16×8 pixels and to perform motion compensation for each individual unit, formed as the result of the split, with reference to different reference frames. In the transmission of the motion vector with decimal error, not a motion vector with integer accuracy, and perform interpolation filter with finite impulse response (FIR) or the FIR filter defined in the standard, you can use the values of the pixels surrounding the corresponding view point, for motion compensation.

Figure 2 presents a diagram illustrating an example of bidirectional prediction.

As shown in figure 2, when you encode the frame In0i.e. the frame that corresponds to the current point in time, generate using bidirectional prediction, motion compensation is carried out by using, as reference frames, already coded frames within the relevant past and future points in time relative to the current time. Here as the reference frames used several already-encoded frame and the residual error between the predicted image and the actual image code using the correlation between these keyframes, which reduces the amount of code. According to the standard H.264 can also be used as reference frames to use a few shots from the past and a few frames from the future.

As shown in figure 2, if a frame from the past and the frame of the future relative to the to-be-encoded frame In0serving base selected as keyframes L0and L1, pixel values in an arbitrary macroblock in the composition of the to-be-encoded frame In0it is possible to predict on the basis of values of arbitrary pixels in the reference frames L0and L1.

In the example shown in figure 2, the macroblock of the reference frame L0corresponding to the macroblock MBB0in the underlying coding frame0marked as macroblock MBL0determined by the motion vector MV0. In addition, the macroblock of the reference frame L1corresponding to the macroblock MBB0in the underlying coding frame0marked as macroblock MBL1determined by the motion vector MV1.

If the pixel values in the macroblock MBL0and MBL1denote MC 0(i, j) and MC1(i, j), respectively, then the value of the pixel Pred(i, j) in the predicted image Pred(i, j) can be obtained as the average value for these values of pixels according to the following equation (2).

Pred(i,j)=(MC0(i,j)+MC1(i,j))/2...(2)

In this motion compensation, as described above for the unidirectional prediction, the accuracy of the projected image is improved by increasing the precision of the motion vector and reduce the size of the macroblock to reduce the residual error relative to the real image, which improves the efficiency of encoding.

Moreover, when motion compensation using bidirectional prediction as values of pixels of the projected image using the average values of the pixels of the reference frames close in time, which makes it possible stable from a probabilistic point of view smart is the solution of the residual error of prediction.

List of literature

Non-patent literature

NPL 1: "Improving resolution by combining images, Michal Irani and Shmuel Peleg, Department of computer science, the Hebrew University of Jerusalem, 91904 Jerusalem, Israel handed Frame Chellapa received 16 June 1989; accepted 25 may 1990 ("Improving Resolution by Image Registration", MICHAL IRANI AND SHMUEL PELEG, Department of Computer Science, The Hebrew University of Jerusalem, 91904 Jerusalem, Israel Communicated by Rama Chellapa, Received June 16, 1989; accepted May 25, 1990)

Disclosure of inventions

Technical problem

In the case of conventional unidirectional prediction, even if you can choose from several reference frames, it is necessary to selectively use pixel values from any one of the reference frames as the values of the pixels of the frame to be coded. Thus, since the unselected reference frame is not used for motion compensation, the correlation in time between this keyframe and the frame to be coded, cannot be used sufficiently, so that there remains considerable room for improvement from the point of view of increasing the efficiency of the encoding.

In addition, in the case of a conventional bidirectional prediction as values of pixels in a frame to be coded, use the average value of values of pixels of the two reference frames, so that made tsetse the low pass filtering by time, resulting in the projected image are lost high-frequency components. As a result, since the residual error signal comprising high-frequency components, coding is impossible, the image obtained by decoding, does not include high-frequency components, which leads to degradation of image resolution.

The present invention was created in light of this situation, and is intended to provide the possibility of generation of precision of the projected image without increasing the processing load.

Solution

The imaging device according to one aspect of the present invention includes means for determining to determine in accordance with the number of taps of the filter used to filter, the number of pixels across the width of the strip placed outside of the macroblock including the reference block representing a block of a decoded reference frame and in contact with the supporting unit, means for receiving, from the reference frame specified reference block and the band corresponding to the number of pixels found means of determining if the reference block representing a block of the reference frame corresponding to a block included in the image, powergems the filtering is in contact with the border of the specified macroblock including the reference block, and filtering means for performing filtering of the image of the reference block and the band obtained by means of a receipt.

Means of obtaining can obtain the reference block from the reference frame, if the reference block is in contact with the boundary of the macroblock including the reference block.

Means for determining can determine the number of pixels equal to the greatest integer that is less than or equal to the value obtained by dividing the number of taps of the filter used for filtering, for two, and a certain number of pixels is the number of pixels in the direction of the width region of the strip.

Filtering may include the first filtering means to filter the lower frequencies in relation to the differential image between multiple images, the second filtering means for performing filtering high frequencies in the image obtained by the low pass filter implemented by the first filtering means, and an adder for adding the image obtained by filtering the low frequencies, performed the first filtering means, and the image obtained by filtering the high-pass performed by the second means Phil is ation, to any of the multiple images for generation of the predicted image in units of macroblocks.

Consider the imaging device may further include memory means for storing, as a reference frame, the decoded frame obtained by decoding performed in units of macroblocks that make up this frame. Tools of the sample can obtain the reference block and the band of the reference frame, is recorded in the memory means.

The imaging device may further include setting means for setting the reference block based on the motion vector.

The specified filter can be a filter with finite impulse response (FIR-filter).

The method of image processing according to one aspect of the present invention includes the step of determining to determine in accordance with the number of taps of the filter used to filter, the number of pixels across the width of the strip placed outside of the macroblock including the reference block representing a block of a decoded reference frame, and are in contact with the reference block, the step of obtaining to obtain, from the reference frame specified reference block and the band corresponding to the number of pixels found at the stage of determining if op the NSS unit, representing the block of the reference frame corresponding to the block included in the image filtered is in contact with the border of the specified macroblock including the reference block, and the phase filter to filter the image of the reference block and the band received at the time of receipt.

According to one aspect of the present invention, the number of pixels across the width of the strip placed outside of the macroblock including the reference block representing a block of a decoded reference frame and in contact with the reference block is determined in accordance with the number of taps of the filter used to filter the specified reference block and the band corresponding to the detected number of pixels from the reference frame, if the reference block representing a block of the reference frame corresponding to a block constituting an image filtered is in contact with the border of the specified macroblock including the reference block, and the filtering is made in relation to the image of the reference block and the band.

The result of inventions

According to the present invention can form a projected image with high accuracy without increasing the processing load.

Brief description of drawings

Fig presents a diagram illustrating the example of the unidirectional prediction.

Figure 2 presents a diagram illustrating an example of bidirectional prediction.

Figure 3 is a block diagram illustrating an example configuration of a decoding device according to one of the variants of the present invention.

Figure 4 presents a diagram illustrating the principle of the third prediction mode.

Figure 5 is a block diagram illustrating an example circuit configuration of the prediction/motion compensation, is shown in figure 3.

6 is a diagram illustrating an example of reference frames.

7 is a diagram illustrating another example of the reference frames.

Fig is a block diagram illustrating an example circuit configuration of the filter, shown in figure 5.

Figure 9 is a logic diagram to illustrate the sequence of operations of the decoding process in the decoding device.

Figure 10 is a logic diagram for explanation of the process flow prediction/motion compensation performed at step S9 figure 9.

11 is a block diagram illustrating an example configuration of encoder.

Fig is a block diagram illustrating an example configuration definition schema mode, shown at 11.

Fig represents the t of a block diagram, illustrating an example circuit configuration of the prediction/motion compensation, is shown at 11.

Fig is a logical diagram to illustrate the sequence of operations of the encoding process in the encoding device.

Fig is a logical diagram to illustrate the sequence of process definition mode is performed at step S58 to Fig.

Fig is a logic diagram for explanation of the process flow prediction/motion compensation performed at the step S61 to Fig.

Fig is a block diagram illustrating another example configuration of the filtering.

Fig is a block diagram illustrating another example of a circuit configuration of the filter.

Fig is a diagram illustrating an example of a case when using three keyframes.

Fig is a block diagram illustrating a configuration example of filtering for the case when using three keyframes.

Fig is a diagram illustrating an example of the interpolation pixels.

Fig is a diagram illustrating the principles of the process using motion compensation with the addition of fields section.

Fig is a diagram illustrating an example of image motion compensation.

Fig is a block diagram, the sludge is ustrious example of a circuit configuration of forecasting, shown in figure 5.

Fig is a logical diagram to illustrate the sequence of operations of the process of motion compensation in the prediction scheme.

Fig is a diagram illustrating an example of the partitioning of the macroblock to be decoded.

Fig is a diagram illustrating an example of the reference frame.

Fig is a diagram illustrating example blocks constituting the macroblock shown in Fig.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of fields section.

Fig is a diagram illustrating an example of image motion compensation.

Fig is a diagram illustrating an example of the FIR filter.

Fig is a diagram illustrating an example of the filtering process.

Fig is a diagram illustrating a result obtained by encoding with primenenia.prochie image, generated by filtering shown in Fig.

Fig is a diagram illustrating another result obtained by encoding using the predicted image generated through filtering, shown in Fig.

Fig is a block diagram illustrating an example configuration of a personal computer.

Fig is a block diagram illustrating a main example configuration of a television receiver in which is applied the present invention.

Fig is a block diagram illustrating a main example configuration of a mobile phone that utilizes the present invention.

Fig is a block diagram illustrating a main example configuration of a recording device on a hard magnetic disk, in which is applied the present invention.

Fig is a block diagram illustrating an example of the basic configuration of a video camera that incorporates the present invention.

Description options inventions

Figure 3 is a block diagram illustrating an example configuration of the decoding device 1 according to one variant of the present invention.

Information of the image compressed and encoded in the encoding device, discussed below, is supplied to the decoding device 1 according to the cable, che is ez communications network or on removable recording media. This compressed image information is the image information, the compressed and encoded in accordance, for example, with the standard H.264.

Intermediate buffer 11 sequentially writes the input bit streams as a compressed image. The information stored in the intermediate buffer 11, as necessary, reads the circuit 12 decodes lossless. Reading occurs fragments (units) of a certain size, such as the macroblocks that make up the frame. According to the standard H.264 process can be carried out not only in units of macroblocks of 16×16 pixels, as well as in units of blocks of size 8×8 pixels or 4×4 pixels, obtained by further partitioning of macroblocks.

Scheme 12 decode lossless performs a decoding procedure corresponding to a coding method, such as decoding of variable length codes or arithmetic decoding, the image read from the intermediate buffer 11. Scheme 12 decode lossless transmits the quantized conversion coefficient obtained by the decoding, the circuit 13 dekvantovanie.

In addition, the circuit 12 decodes lossless identifies on the basis of the identification flag included in the header image to be decoded, encoded whether this image is their using intraframe coding or interframe coding. If circuit 12 decodes lossless determines that the image to be decoded, encoded intraframe method, scheme 12 decode lossless transfers in scheme 22 intra-frame prediction information on the mode of intra-frame prediction recorded in the image header. This mode information intraframe prediction includes information relating to the intra-frame prediction, such as the block size that is used as a unit process.

If circuit 12 decodes lossless determines that the image to be decoded, encoded interframe way, the circuit 12 decodes lossless transmits the motion vector and an identification flag recorded in the image header, the circuit 21 prediction/motion compensation. Identification flag identifies the prediction mode to generate a predicted image using interframe prediction. Identification flags set in units of, for example, macroblock or frame.

In addition to the mode unidirectional prediction, shown in figure 1, and the mode of bidirectional prediction, shown in figure 2, a third prediction mode for generating the predicted image by performing f is litraly image motion compensation, selected from multiple reference frames located in one direction along the time axis, or in both directions.

Figure 4 presents a diagram illustrating the principles of the third prediction mode.

In the example shown in figure 4, if we take as a basis the time corresponding to the current frame (predicted frame), the frame preceding in time for one time step to the current frame, denoted as the reference frame R0and the frame preceding by one time step this keyframe R0designated as the reference frame R1. In this case, in accordance with the third mode prediction image MS0and MC1the motion-compensated, isolated from the reference frame R0and R1injected into the filter circuitry, and the values of the pixels of the output image filtering is considered as values of pixels of the projected image representing the target macroblock.

In the future, the prediction mode, where the values of the pixels of any image motion compensation allocated from multiple reference frames located in the same direction, as shown in figure 1, taking as values of pixels of the projected image, simply referred to as mode unidirectional prediction. In addition, to the to shown in figure 2, the prediction mode, where the mean values of pixels of image motion compensation allocated from multiple reference frames located in both directions, accept as values of pixels of the projected image, simply referred to as a mode of bidirectional prediction.

The third prediction mode, which, as shown in figure 4, values of pixels of the projected image will get through the filter of individual images with motion compensation allocated from multiple reference frames located in one or in two directions, referred to as mode prediction filtering. This prediction mode filtering will be described in detail below.

If we return to figure 3, scheme 13 dekvantovanie provides dekvantovanie quantized conversion coefficient from the circuit 12 decodes lossless, using the method corresponding to the quantization method used on the side of coding. Scheme 13 dekvantovanie outputs the conversion coefficient obtained by performing dekvantovanie in scheme 14 inverse orthogonal transformation.

Scheme 14 the inverse orthogonal transform performs inverse orthogonal transform on the fourth then the dka above conversion factor, coming from scheme 13 dekvantovanie, using the method corresponding to the method of orthogonal transformation is used on the coding side, such as discrete cosine transformation or conversion of karunen-Loev, and transmits the acquired image to the adder 15.

The adder 15 performs the summation of the image coming from the circuit 14 inverse orthogonal transformation, and the predicted image supplied from the circuit 21 prediction/motion compensation or from the circuit 22 intraframe prediction through the switch 23, and outputs a complex image in deblocare filter 16.

Deblocare filter 16 removes block noise included in the image received from the adder 15, and displays the image from which you have removed the block noise. The output image from deblokiruyuschee filter 16 is supplied to the buffer 17 permutations and memory 19 frames.

The buffer 17 permutations temporarily stores the image received from deblokiruyuschee filter 16. The buffer 17 generates permutations of individual frames on the basis of the recorded images in it, for example, in units of macroblocks, rearranges the generated frames in a specific order, such as order of presentation on the display, and displays these images in digital-to-analog Converter (DAC) 18.

The DAC 18 to perform yet digital to analog conversion of individual frames coming from the buffer 17 permutations, and outputs signals of these individual frames.

The memory 19 temporarily stores frames of the image coming from deblokiruyuschee filter 16. The information recorded in the memory 19 of frames passed in scheme 21 prediction/motion compensation or circuit 22 intraframe prediction through the switch 20.

The switch 20 is connected to terminal a1in the case of generating the predicted image by interframe prediction, and is connected to the terminal b1in the case of generating the predicted image by intra-frame prediction. The operation of the switch 20 is controlled, for example by circuit 31 of the control.

Scheme 21 prediction/motion compensation determines the prediction mode in accordance with the identification flag from the circuit 12 decodes lossless, and selects the frame to use as a reference frame among the already decoded frames stored in the memory 19 frames, depending on the prediction mode. Scheme 21 prediction/motion compensation determines the macroblock corresponding to the interest of the projected image, from multiple macroblocks forming the reference frame based on the motion vector supplied from the circuit 12 decodes lossless, and videla is t defined in this way, the macroblock in the image quality with motion compensation. Scheme 21 prediction/motion compensation determines the pixel values of the predicted image based on values of pixels in the image with motion compensation in the prediction mode, and outputs the predicted image with defined this way values of pixels in the adder 15 through the switch 23.

Scheme 22 intraframe prediction performs such intra-frame prediction in accordance with the information about the intra-frame mode prediction coming from the circuit 12 decodes lossless, and generates the predicted image. Scheme 22 intraframe prediction transmits the generated predicted image to the adder 15 through the switch 23.

The switch 23 is connected with the terminal a2when the projected image has been formed by the circuit 21 prediction/motion compensation, and is connected to the terminal b2if the projected image was created by the circuit 22 intra-frame prediction. The operation of the switch 23 also controls, for example, the circuit 31 of the control.

Circuit 31 of the control switches the connection of the switches 20 and 23, and controls the overall operation of the decoding device 1. This scheme 31 management can be identified, coded whether the image to be processed, with p the physical alteration of intraframe or interframe encoding.

Figure 5 is a block diagram illustrating an example configuration circuit 21 prediction/motion compensation, is shown in figure 3.

As shown in figure 5, the circuit 21 prediction/motion compensation is composed of circuit 41 determination of the prediction mode, the circuit 42 of the unidirectional prediction circuit 43 bidirectional prediction circuit 44 forecasting and circuit 45 of the filter. The motion vector and an identification flag from the circuit 12 decodes lossless, is directed to the input circuit 41 determination of the prediction mode.

Scheme 41 determine the prediction mode determines the prediction mode in accordance with the identification flag coming from the decoder 12 without a loss. This scheme 41 determine the prediction mode transmits the motion vector in scheme 42 unidirectional prediction when it is determined that the projected image should be generated through unidirectional prediction, or transmits the motion vector in scheme 43 bidirectional prediction when it is determined that the projected image should be generated through bidirectional prediction. In addition, if it is determined that the projected image should be generated by predictive filtering circuit 41 determination re the ima forecasting transmits the motion vector in scheme 44 forecasting.

Thus, to enable identification of the prediction filter can be set as a value of the identification flag value other than the values representing the unidirectional prediction, and a value representing the bi-directional prediction, as defined in the usual standard H.264. In an alternative embodiment, the prediction mode can be determined by any formulated in advance of the way instead of defining the mode identification flag, to reduce the amount of information.

Scheme 42 unidirectional prediction considers several frames arranged in one direction on the time axis as a reference frame and defines in these reference frames, macroblocks corresponding to the projected image, based on the motion vectors, as shown in figure 1. In addition, the circuit 42 unidirectional prediction reads certain it macroblocks in the respective reference frames from the memory 19 as image motion compensation, and generates a predicted image using values of pixels of any of these images with motion compensation in units of pixels of the projected image. Scheme 42 unidirectional prediction lane who gives it to the predicted image to the adder 15. To perform the unidirectional prediction circuit 42 unidirectional prediction using, for example, the algorithm unidirectional prediction, defined in the standard H.264.

Scheme 43 bidirectional prediction considers several frames arranged in two directions along the time axis, as the reference frame and defines in these reference frames, macroblocks corresponding to the projected image, based on the motion vectors, as shown in figure 2. In addition, the circuit 43 bidirectional prediction reads certain it macroblocks in the respective reference frames as images with motion compensation from the memory 19 frames, and generates the predicted image by using the average values of the pixels of these images with motion compensation in units of pixels of the projected image. Scheme 43 bidirectional prediction transmits the predicted image to the adder 15. To perform bidirectional prediction circuit 43 bidirectional prediction uses, for example, the algorithm of bi-directional prediction, defined in the standard H.264.

Scheme 44 forecasting defines several frames located in one or in two directions along the time axis, as a reference to the wood. What frames should be used as a reference frame, it is possible to determine in advance or can be selected on the basis of information transmitted coding side together with the identification flag.

6 is a diagram illustrating an example of reference frames.

In the example shown in Fig.6, when the time corresponding to the projected image, serves as a base, two frames located on the time axis one step ahead and two steps before, respectively, the predicted frame, considered as a reference frame, similarly to the description given with reference to figure 4. From these two reference frames, one which is located on the time axis is closer to the predicted frame, i.e. precedes it one step at a time designated as the reference frame R0and the frame preceding the one step time of the reference frame R0designated as the reference frame R1.

7 is a diagram illustrating another example of the reference frames.

In the example shown in Fig.7, when the time corresponding to the projected image, serves as a base, two frames, one of which is one time step before, and the other one step at a time after the predicted frame, considered as the reference frame. From these two reference frames the frame that precedes the PR is nosireebob the frame at one time step, designated as a reference frame L0and the frame, which is one time step after the predicted frame, designated as a reference frame L1.

Thus, when the prediction filter as the reference frames used several frames arranged in one direction on the time axis, or multiple frames, which are arranged in two directions along the time axis.

In addition, the circuit 44 forecasting determines on the basis of the motion vectors supplied from the circuit 41 determination of the prediction mode, the macroblock corresponding to the projected image, among the already decoded macroblocks in the reference frames defined, as shown at 6 or 7.

Scheme 44 forecasting reads the selected macroblocks of the respective reference frames as images with motion compensation from the memory 19 frames and transmits the read image to the motion compensation circuit 45 of the filter. The motion vectors can be defined in units of macroblocks of 16×16 pixels, and in units of blocks obtained by further partitioning of macroblocks. Image, for example, in units of macroblocks is passed to the input circuit 45 of the filter. Figure 5 two arrows directed from the circuit 44 forecasting scheme 45 filtering, represent the transfer of the two images to what pensala movement.

Scheme 45 filtering takes images with motion compensation from the scheme 44 prediction filters these images, and outputs the predicted image obtained by filtering in the adder 15.

Fig is a block diagram illustrating an example configuration circuit 45 of the filter. In the circuit 45 of the filter shown in Fig, the signal is filtered in the time domain.

As shown in Fig, the circuit 45 consists of circuit 51 calculating the difference, the filter 52 lowpass circuit 53 adjusting the gain of the filter 54 of the upper frequencies, the circuit 55 adjusting the gain of the adder 56 and the adder 57. Image MS0the motion-compensated coming from the circuit 44 forecasting, is fed to the input circuit 51 calculating the difference and the adder 57, and the image MC1with motion compensation to the input circuit 51 calculating the difference.

If, as shown in Fig.6, the predicted image generated, for example, through unidirectional prediction, the image extracted from the reference frame R0considered as having a higher correlation with the projected image is considered as an image MS0with motion compensation, and the image extracted from the reference frame R1consider how the image MC1 with motion compensation. The image extracted from the reference frame R0can be considered as the image MC1with motion compensation, and the image extracted from the reference frame R1can be considered as the image MS0with motion compensation.

On the other hand, if the predicted image generated through bidirectional prediction, as shown in Fig.7, the image is separated from the preceding one time step keyframe L0consider as an image MS0with motion compensation, and the image extracted from the reference frame L1one time step later, the projected image is designated as the image MC1with motion compensation, for example. The image extracted from the reference frame L0can be considered as the image MC1with motion compensation, and the image extracted from the reference frame L1can be considered as the image MS0with motion compensation.

Scheme 51 calculating the difference calculates the difference between the image MS0with motion compensation and image MC1with motion compensation and transmits the differential image in the filter 52 of the lower frequencies. The differential image D is represented by the following equation (3).

D(i,j)=MC0(i,j)-MC1(i,j)...(3)

In equation (3), the indices (i, j) represent the relative position of a pixel in the image with motion compensation. If processing is performed in units of macroblocks of 16×16 pixels, run of the ratio 0≤i≤16 and 0≤j≤16. The same is true later.

The filter 52 of the lower frequencies contains the FIR filter. The filter 52 of the lower frequencies to filter out the lower frequencies in the difference image D obtained from the circuit 51 calculating the difference, and transmits the acquired image in scheme 53 adjustment of the gain and the filter 54 of the upper frequencies. The differential image D'obtained by performing the low pass filter represented by the following equation (4). In equation (4), LPF(X) represents the filtering of low frequencies in the input image X using a two-dimensional FIR filter.

D'=LPF(D)...(4)

Scheme 53 adjusting the gain adjusts the gain of the differential image D', coming from the filter 52 of the lower frequencies, and transmits the image adjusted by the gain in the adder 56. The output image X(i, j) scheme 53 adjusting the gear ratio represented by the following equation (5).

X(i,j)=αD'(i,j)...(5)

The filter 54 of the upper frequencies contains the FIR filter. This filter 54 high-pass filters high-pass in the differential image D', coming from the filter 52 of the lower frequencies, and displays the resulting image in scheme 55 adjustment of the gear ratio. The differential image D obtained by performing the high-pass filter represented by equation (6) below. In this equation (6), HPF(X) represents the filtering high frequencies in the input image X using a two-dimensional FIR filter.

D''=HPF(D')...(6)/mrow>

Scheme 55 adjusting the gain adjusts the gain of the differential image D", coming from filter 54 of the upper frequencies, and displays the image with the adjusted transmission efficiency in the adder 56. The output image Y(i, j) scheme 55 adjusting the gear ratio represented by the following equation (7).

Y(i,j)=βD''(i,j)...(7)

As the coefficient α in equation (5) and β in equation (7) can be selected, for example, α=0.8 and β=0.2 is, but you can choose other values to improve the accuracy of the projected image. Moreover, these values can be adaptively changed depending on the properties of the input sequence.

The adder 56 adds the image X(i, j) and the image Y(i, j) with adjusted coefficients of transmission and transmits the output image obtained in the result of summation. The output image Z(i, j) of the adder 56 is represented by the following equation (8).

Z(i,j)=X( i,j)+Y(i,j)(8)

The output image Z(i, j) represents the high frequency components of the image, which can be determined on the basis of a difference between the image MS0with motion compensation and image MC1the motion-compensated, i.e. the correlation between these images.

The adder 57 performs the summation of the output image Z(i, j)supplied from the adder 56, depicting MS0with compensation movement and transmits the acquired image to the adder 15 as the projected image. The projected image S(i, j), i.e. the final output image adder 57, represented by the following equation (9).

S(i,j)=MC0(i,j)+Z(i,j)(9)

Thus, in the prediction mode filter generate an image, obtained by summing the image representing high the frequency components, image MS0with motion compensation, as the projected image. This projected image has more high frequency components than the predicted image generated in the case of simple perform bi-directional prediction. As described above, because the values of the pixels of the projected image taking average values of pixels of multiple images with motion compensation, in the predicted image is generated using bidirectional prediction, high-frequency components are lost.

Furthermore, since the adder 15 of the projected image includes a large number of high-frequency components, summarize with the decoded image, the final output image decoding device 1 is also an image with high resolution, including a large proportion of high-frequency components.

Moreover, it is possible to generate the predicted image by more efficient use of image correlation in time, compared to the case with simple execution of the unidirectional prediction. As described above, by using values of pixels of one image with motion compensation of several such the images compensated predicted image, generated through unidirectional prediction, not to mention generated using a sufficient degree of correlation of images in time.

Next will be described the principle of the decoding device 1 having the above configuration.

First, the process of decoding performed by the decoding device 1 may be considered with reference to the logic diagram of the sequence of operations shown in Fig.9.

The process, shown in Fig.9, begins when, for example, the circuit 12 decodes lossless reads an image of a certain size, such as a macroblock size of 16×16 pixels, from the data stored in the intermediate buffer 11. At each stage, depicted in figure 9, the data processing can be performed, if necessary, in parallel with the processing of another stage, or you can change the sequence of execution stages. The same applies to the processing of data at the appropriate stages in the individual logic circuits, discussed below.

At step S1, the circuit 12 decodes losslessly decodes the image read from the intermediate buffer 11, and transmits the quantized coefficient conversion circuit 13 dekvantovanie. In addition, the circuit 12 decodes lossless transmits information about the intra-frame mode of prognozirovaniya scheme 22 intraframe prediction, if the image to be decoded, encoded intraframe method, and transmits the motion vector and an identification flag circuit 21 prediction/motion compensation, if this picture interframe coded by method.

At step S2 scheme 13 dekvantovanie provides dekvantovanie using the algorithm corresponding to the quantization method used on the coding side, and outputs the conversion coefficient in scheme 14 inverse orthogonal transformation.

At step S3 scheme 14 inverse orthogonal transform performs such inverse orthogonal transform for conversion received from scheme 13 dekvantovanie, and outputs the resulting image to the adder 15.

At step S4, the adder 15 performs a summation of the decoded image supplied from the circuit 14 inverse orthogonal transformation, and the predicted image supplied from the circuit 21 prediction/motion compensation or from the circuit 22 intra-frame prediction, and outputs the total image in deblocare filter 16.

At step S5 deblocare filter 16 filters for removing block noise included in the total image, and displays the output image, which eliminated these block the noise.

At step S6, the memory 19 is Adrov, managing temporarily records the image, received from deblokiruyuschee filter 16.

At step S7 scheme 31 management determines coded if the image is intra-frame method.

If in step S7 it is determined that the image is encoded in intraframe mode, then at step S8 scheme 22 intra-frame prediction to generate a predicted image by intra-frame prediction, and outputs generated in this way, the projected image in the adder 15.

On the other hand, if in step S7 it is determined that the image is not encoded intraframe method, i.e. the picture interframe coded way, then at step S9 scheme 21 prediction/motion compensation performs a prediction/motion compensation. The predicted image generated by this procedure prediction/motion compensation, is passed to the adder 15. Following the procedure of prediction/motion compensation will be described with reference to the logic diagram shown in figure 10.

At step S10 circuit 31 of the control determines whether the above procedure is performed on the macroblocks in one complete frame. If circuit 31 management has determined that the procedure has not been performed for all of the macroblocks of one full frame, again, repeat the process from step S1 to another macroblock./p>

On the other hand, if in step S10 it is determined that the procedure was performed on all the macroblocks of one full frame, then at step S11, the buffer 17 permutations outputs the generated frame to the digital-to-analog Converter 18 to the command circuit 31 of the control.

At step S12 DAC 18 performs digital-to-analog conversion of the frame received from the buffer 17 permutations, and outputs the analog signal. The above procedure is performed for individual frames.

Next, the procedure of prediction/motion compensation performed at the step S9 shown in figure 9, will be described with reference to a logical scheme shown in figure 10.

At step S21 scheme 41 determine the prediction mode of composition scheme 21 prediction/motion compensation determines if the identification flag received from the decoder 12 without loss that processing should be done in prediction mode filtering.

If at step S21 it is determined that the identification flag indicates that processing should be done in prediction mode filter, then at step S22 perform unidirectional or bidirectional prediction and generate the predicted image.

Thus, if the identification flag indicates that data processing should be performed in mode adsonar is undertaken forecasting, the motion vectors are passed from the circuit 41 to determine the prediction mode in scheme 42 unidirectional prediction, and the circuit 42 unidirectional prediction performs unidirectional prediction. In addition, if the identification flag indicates that data processing should be done in the mode of bidirectional prediction, motion vectors are passed from the circuit 41 to determine the prediction mode in scheme 43 bidirectional prediction, and circuit 43 bidirectional prediction performs bidirectional prediction. After displaying the projected image in the adder 15, the process returns to step S9 shown in Fig.9, and is used for further processing of the data.

On the other hand, if at step S21 it is determined that the identification flag indicates that data processing should be made in the prediction mode with filtering, then at step S23 scheme 44 forecasting selects the image with the motion compensation of each of multiple reference frames and transmits the image from the motion compensation circuit 45 of the filter. The motion vectors received from the circuit 41 to determine the prediction mode in scheme 44 forecasting, where the selection of images with motion compensation is performed using these vectors.

At step S24 scheme 51 Vici the population difference in the scheme of 45 filter computes the difference between the image MS 0with motion compensation and image MC1with motion compensation, and outputs the differential image in the filter 52 of the lower frequencies.

At step S25, the filter 52 of the lower frequencies to filter out the lower frequencies in the difference image received from the circuit 51 calculating the difference, and outputs the resulting image in scheme 53 adjustment of the gain and the filter 54 of the upper frequencies.

At step S26 scheme 53 adjusting the gain adjusts the gain of the image coming from the filter 52 of the lower frequencies, and displays the image with the adjusted transmission efficiency in the adder 56.

At step S27, the filter 54 high-pass filters high-pass in the difference image supplied from the filter 52 of the lower frequencies, and displays the resulting image in scheme 55 adjustment factor.

At step S28 scheme 55 adjusting the gain adjusts the gain of the differential image coming from the filter 54 of the upper frequencies, and displays the image with the adjusted transmission efficiency in the adder 56.

At step S29, the adder 56 performs the summation of the image coming from the circuit 53 adjustment of the gear ratio, and image coming from the circuit 55 adjustment of the gear ratio, to obtain a high frequency with the estimated image. The received high-frequency components supplied from the adder 56 to the adder 57.

At step S30, the adder 57 performs the summation of the image coming from the adder 56, (high frequency component) with the image of MS0with motion compensation, and outputs the resulting image as a predicted image in the adder 15. After this, the process returns to step S9 shown in Fig.9, and executes the subsequent processing.

Thus, decoding is performed using the predicted image generated by the prediction filter, which makes it possible to obtain a decoded image with high resolution.

Next will be described the configuration and operation of the device on the coding side.

11 is a block diagram illustrating an example configuration of encoder 101. Information about the compressed image obtained by encoding in the encoding device 101, is fed to the input of the decoding device 1, shown in figure 3.

Analog-to-digital Converter 111 (ADC) performs analog-to-digital conversion of the input signal and transmits the image to the buffer 112 permutations.

The buffer 112 performs permutations a permutation of frames in accordance with a GOP (group of frames) information about the JUA is the first image and transmits the image in the form of some units, such as macroblocks. The output image from the buffer 112 permutations supplied to the adder 113, the circuit 123 definition mode, the circuit 125 prediction/motion compensation and circuit 126 intra-frame prediction.

The adder 113 determines the difference between the image from the buffer 112 of permutations, and the predicted image generated by the scheme 125 prediction/motion compensation or circuit 126 intraframe prediction and supplied through the switch 127, and outputs the residual error in the schema 114 orthogonal transformations. The closer the predicted image to the original image and the smaller the specified residual error, the smaller the amount of code that should be assigned to the residual error, and therefore the higher the coding efficiency.

Scheme 114 orthogonal transformation performs orthogonal transform such as discrete cosine transformation or conversion of karunen-Loev, residual errors coming from the adder 113, and outputs the conversion coefficient obtained as a result of such orthogonal transform circuit 115 quantization.

Circuit 115 performs quantization quantization conversion factor coming from the circuit 114 orthogonal transformation in accordance with the commands of Hemi 118 control sampling frequency and outputs the quantized conversion coefficient on the output. The conversion coefficient quantized in the scheme 115 quantization, enters the circuit 116 lossless encoding and circuit 119 dekvantovanie.

Scheme 116 lossless encoding compresses the conversion factor received from circuit 115 quantization, by performing lossless encoding, such as coding in the variable length code or arithmetic coding, and outputs data in the intermediate buffer 117.

In addition, the circuit 116 lossless encoding sets the value of the identification flag in accordance with the information received from the circuit 123 definition mode, and inserts this identification flag in the image header. As described above, the decoding device 1 determines the prediction mode on the basis of the identification flag, the scheme described above 116 lossless encoding.

Scheme 116 lossless encoding describes also the information coming from the circuit 125 prediction/motion compensation or circuit 126 intra-frame prediction, in the image header. The motion vectors and other such information as determined by performing interframe prediction come from circuit 125 prediction/motion compensation, and information relating to the application of intra-frame prediction comes from the circuit 126 intra-frame prediction.

Intermediate buffer 117 temporarily stores the information received from the circuit 116 lossless encoding, and outputs this information to the information of the compressed image at certain points in time. Intermediate buffer 117 transmits information about the amount of generated code in circuit 118 controls the sampling frequency.

The circuit 118 controls the sampling rate calculates the quantization scale based on the amount of code from the output of the intermediate buffer 117 and controls the circuit 115 quantization so that quantization was carried out in accordance with the calculated scale.

Scheme 119 dekvantovanie provides dekvantovanie conversion, quantum circuit 115, and outputs the coefficient conversion circuit 120 inverse orthogonal transformation.

Scheme 120 inverse orthogonal transformation carries a inverse orthogonal transform for conversion coming from schema 119 dekvantovanie, and displays the resulting image in deblocare filter 121.

Deblocare filter 121 removes block noise appearing in the locally decoded image, and outputs an image from which were eliminated these block noise in the memory 122 frames.

The memory 122 writes the image frames received from deblokiruyuschee filter 121. This image is agenie, recorded in the memory 122 frames, reads as appropriate scheme 123 definition mode.

Scheme 123 definition mode determines whether to perform intra-frame coding or inter-frame coding on the basis of the image recorded in the memory 122 of the frame, and the original image supplied from the buffer 112 permutations. In addition, if the scheme 123 mode determination determines that it should perform inter-frame coding, in this case, the circuit 123 definition mode selects one of the mode - mode unidirectional prediction mode of bidirectional prediction, or prediction mode filtering. Scheme 123 definition mode transmits information indicating the determination results, the circuit 116 encoding without loss of quality information about the mode.

If the scheme 123 mode determination determines that it should perform inter-frame coding, in this case, the circuit 123 definition mode transmits a frame stored in the memory 122 of the frame, and the frame received in the local decoding, the circuit 125 prediction/motion compensation through the switch 124.

In addition, if the scheme 123 mode determination determines that it should perform intra-frame coding, in this case, the circuit 123 definition mode transmits a frame stored in the memory 122 of the frame, and the frame received is the first in the local decoding, in the circuit 126 intra-frame prediction.

The switch 124 is connected to terminal a11when to perform interframe encoding, and is connected to the terminal b11when to perform intra-frame coding. The operation of the switch 124 controls, for example, the circuit 131 controls.

Scheme 125 prediction/motion compensation determines the motion vectors on the basis of the original image from the buffer 112 permutations, and reference frames read from memory 122 frames, and displays the found motion vectors in the diagram 116 lossless encoding. In addition, the circuit 125 prediction/motion compensation performs motion compensation using the detected motion vectors and reference frames to generate a predicted image, and outputs the generated predicted image to the adder 113 through the switch 127.

Scheme 126 intra-frame prediction performs such intra-frame prediction based on the original image from the buffer 112 permutations, and support personnel, locally decoded and recorded in the memory 122 of frames, and generates the predicted image. Scheme 126 intra-frame prediction transmits the generated predicted image to the adder 113 through the switch 127 and passes information is the situation regarding the regime intraframe prediction circuit 116 lossless encoding.

The switch 127 is connected to terminal a12or to terminal b12and outputs the predicted image generated by the scheme 125 prediction/compensation movements or circuit 126 intra-frame prediction, the adder 113.

Scheme 131 control switches the connection of the switches 124 and 127 in accordance with the regime, found the scheme 123 definition mode, and controls the overall operation of the coding device 101.

Fig is a block diagram illustrating an example configuration schema 123 definition mode, shown at 11.

As shown in Fig, circuit 123 definition mode is composed of circuits 141 intraframe prediction circuit 142 interframe prediction circuit 143 calculate the forecasting errors and circuit 144 definitions. In the scheme 123 determine the mode of intra-frame prediction and inter-frame prediction is performed by the blocks, the sizes of which differ from one another, and in what mode, it is necessary to make predictions, determine on the basis of the obtained result. In the case of the interframe prediction processing is performed in each of the prediction modes, i.e. in the regime of unidirectional prediction mode of bidirectional prediction in the prediction mode filtering. The image of the original from the buffer 11 permutations, enter into the scheme 141 intraframe prediction circuit 142 interframe prediction and circuit 143 calculate the forecasting errors.

Scheme 141 intra-frame prediction performs intra-frame prediction blocks, the sizes of which differ from one another, based on the original image and the image read from the memory 122 frames, and transmits the generated predicted image to the schema 143 calculation error of prediction. The subcircuit 151-1 forecasting in the format of 4×4 performs intra-frame prediction blocks of size 4×4 pixels, and the subcircuit 151-2 forecasting in the format of 8×8 performs intra-frame prediction blocks of size 8×8 pixels. Podhuma 151-3 forecasting a 16×16 performs intra-frame prediction blocks of size 16×16 pixels.

Scheme 161 forecasting in the scheme 142 interframe prediction determines the motion vectors in units of blocks, the sizes of which differ from one another, based on the original image and the reference frame read from the memory 122 frames. In addition, the scheme 161 forecasting performs motion compensation on the basis of the detected motion vectors, and outputs image motion compensation used for generating the predicted image.

In the subcircuit 161-1 forecasting a 16×16 is image processing in units of blocks of size 16×16 pixels. In the subcircuit 161-2 forecasting a 16×8 is image processing in units of blocks of size 16×8 pixels. In addition, in the subcircuit 161-(n-1) forecasting in the format of 4×4 is image processing in units of blocks of size 4×4 pixels. In the subcircuit 161-n prediction skipping/direct prediction of the motion vectors determined prediction mode with or omissions in the direct mode prediction and perform motion compensation using the detected motion vectors.

Image motion compensation allocated from multiple reference frames located in one direction relative to the current frame received from the corresponding sub circuit 161 forecasting scheme 162 unidirectional prediction. Image motion compensation allocated from multiple reference frames, which are arranged in two directions relative to the current frame received from the corresponding sub circuit 161 forecasting scheme 163 bidirectional prediction.

If you want to make a prediction filter using, as described above, image motion compensation, selected from multiple reference frames located in one direction, these images with motion compensation allocated from multiple reference ka the ditch, situated in one direction, direct from the respective subcircuit schematic forecasting 161 schema 164 filtering. If you want to make a prediction filtering using image motion compensation allocated from multiple reference frames, which are arranged in two directions, these images with motion compensation allocated from multiple reference frames, which are arranged in two directions, sent from the respective subcircuit schematic forecasting 161 schema 164 filtering.

The circuit 162 unidirectional prediction performs unidirectional prediction using image motion compensation, which differ in size from one another and from the respective subcircuits of the circuit 161 forecasting, thereby generating the predicted image, and transmits the generated predicted image to the schema 143 calculation error of prediction. For example, the circuit 162 unidirectional prediction generates a predicted image using values of pixels of one image from the set of multiple images with motion compensation size of 16×16 pixels coming from the subcircuit 161-1 forecasting, as values of pixels of the projected image.

Scheme 163 bidirectional formation is of performs bidirectional prediction using image motion compensation, different sizes from one another and from the respective subcircuits of the circuit 161 forecasting, thereby generating the predicted image, and transmits the generated predicted image to the schema 143 calculation error of prediction. For example, the scheme 163 bidirectional prediction generates a predicted image using the mean values of pixels of multiple images with motion compensation size of 16×16 pixels coming from the subcircuit 161-1 forecasting, as values of pixels of the projected image.

The circuit 164 filtering performs prediction filter using each of the image motion compensation, vary in size from one another and from the respective subcircuits of the circuit 161 forecasting, thereby generating the predicted image, and transmits the generated predicted image to the schema 143 calculation error of prediction. The circuit 164 of the filter corresponds to the diagram 45 filter in the decoding device 1 and has the same configuration as shown in Fig.

For example, when generating a predicted image for images MS0and MC1the motion-compensated by 16×16 pixels each, coming from subcircuit 161-1 forecasting, the circuit 164 is istratii determines the difference between these images MS 0and MC1with motion compensation and performs low pass filtering in the resulting difference image. In addition, the circuit 164 filter to filter out high frequencies in the output signal after filtering out the lower frequencies and sums the output image after filtering, high-pass was set up by the gear ratio and the output image after filtering out the lower frequencies with the tweaked transfer coefficient. The circuit 164 filtering summarizes the image obtained in the above summation and representing high-frequency components, with the image of MS0with motion compensation, thereby generating the predicted image, and outputs the generated predicted image to the schema 143 calculate the forecasting errors.

Scheme 143 calculation error of prediction determines the difference between the original image and the corresponding projected images supplied from the respective circuits in the scheme 141 intra-frame prediction, and outputs the residual error signal representing the obtained difference, the circuit 144 definitions. In addition, the scheme 143 calculation error of prediction determines the difference between the original image and the corresponding projected images is, coming from the circuit 162 unidirectional prediction circuit 163 bidirectional prediction and circuit 164 filtering circuit 142 interframe prediction, and outputs the residual error signal representing the obtained difference, the circuit 144 definitions.

The circuit 144 definition measures the intensity of the residual error signals coming from the circuit 143 calculate the forecasting errors, and selects the method of prediction used for generating the predicted image, characterized by a small difference relative to the original image, as a predictor for generating predicted image to use when encoding. The circuit 144 definition displays information representing the results of selecting which information on the mode, the circuit 116 lossless encoding. This mode information includes information representing the block size that is used as a unit of processing, etc.

In addition, when the circuit 144 determining determines that the predicted image to generate way interframe prediction (will determine what you should perform interframe coding), this scheme 144 definition displays keyframes read from memory 122 frames, the circuit 125 prediction/compensation movement is tion, together with information about the mode. If the schema 144 determining determines that the predicted image to generate way intraframe prediction (will determine what you should perform intra-frame coding), this scheme 144 definition displays used for intra-frame-prediction reference image read from the memory 122 of the frame, the circuit 126 intra-frame prediction, together with information about the mode.

Fig is a block diagram illustrating an example configuration schema 125 prediction/motion compensation, is shown at 11.

As shown in Fig, the circuit 125 prediction/motion compensation is composed of circuit 181 determination of motion vectors, the circuit 182 unidirectional prediction circuit 183 bidirectional prediction circuit 184 forecasting and circuit 185 filtering. Scheme 125 prediction/motion compensation has the configuration similar to the circuit 21 prediction/motion compensation, is shown in figure 5, except that instead of scheme 41 determination of the prediction mode is present scheme 181 determine motion vectors.

Scheme 181 determining motion vectors does this definition of a motion vector by comparing blocks or similar objects on the basis of the original image from the buffer 112 permutations, and op the situations frames, coming from the circuit 123 definition mode. Scheme 181 determining motion vectors are compared with the information coming from the circuit 123 definition mode, and outputs the motion vectors together with keyframes in one schema - the schema 182 unidirectional prediction circuit 183 bidirectional prediction or circuit 184 prediction filtering.

Scheme 181 determining motion vectors outputs the motion vectors together with keyframes in the circuit 182 unidirectional prediction, if you have chosen the way of the unidirectional prediction, and outputs these pieces of information in the schema 183 bidirectional prediction, if you selected a method of bi-directional prediction. Scheme 181 determining motion vectors displays these motion vectors together with keyframes in the scheme 184 prediction filtering, if you have selected a method for predicting filtering.

Similar to the scheme 42 unidirectional prediction, shown in figure 5, the circuit 182 unidirectional prediction generates the predicted image by performing unidirectional prediction. This scheme 182 unidirectional prediction outputs formed by it projected image to the adder 113.

Similar to the scheme 43 bidirectional prediction, shown in figure 5, the circuit 183 dunproofin the th prediction generates the predicted image, performing bidirectional prediction. This scheme 183 bidirectional prediction outputs formed by it projected image to the adder 113.

Similarly, the circuit 44 prediction, shown in figure 5, the circuit 184 forecasting allocates image with motion compensation from multiple (e.g. two) reference frames, and outputs these few selected images from the motion compensation circuit 185 filtering.

Similarly, the circuit 45 of the filter shown in figure 5, the circuit 185 filter generates the predicted image by predicting a filter. This scheme 185 filter transmits the generated predicted image to the adder 113. Note that the scheme 185 filter has the configuration similar to the configuration circuit 45 of the filter shown in Fig. Further, the circuit configuration 185 filter will be described by using appropriate references to the schema configuration 45 of the filter shown in Fig.

The predicted image generated by the prediction filter may include a large amount of high-frequency components compared with the predicted image generated through unidirectional prediction or bidirectional prediction, and to have a small difference relative to the original image is agenia. Therefore, the amount of code assigned to the residual error is small, which improves the coding efficiency.

Moreover, such a prediction filtering may be performed when the number of reference frames is at least two, and therefore, such increase encoding efficiency can be realized without increasing the complexity of data processing. For example, the residual error relative to the original image can be reduced and the encoding efficiency can be improved by generating a predicted image with high accuracy when a large number of reference frames used for inter-frame prediction, and the use of a formed image. However, in this case, the processing complexity increases as the number of used reference frames becomes large.

Note that when you need to choose a method for predicting, to the signal intensity of the residual errors can be added weighting factor corresponding to the amount of code and is determined based on the amount of code information such as motion vectors required for prediction, and the encoding mode, and by summing the weighting factor corresponding to the amount of code, so what is the best way of forecasting. Rela is estwenno, you can further improve the coding efficiency. In addition, to simplify the encoding procedure can adaptively choose the method of forecasting using characteristic values of the input original image in the direction of the time axis in the spatial directions.

The section describes the principle of operation of the coding device 101 having the above configuration.

Procedure coding in the encoding device 101 will be described with reference to the logic diagram in Fig. This procedure starts when the image of some units, such as macroblock supplied from the output buffer 112 permutations.

At step S51, the adder 113 determines the difference between the image from the buffer 112 of permutations, and the predicted image generated in the circuit 125 prediction/motion compensation scheme 126 intra-frame prediction, and outputs the residual error in the schema 114 orthogonal transformations.

At step S52, the circuit 114 performs orthogonal transformation orthogonal transformation of the residual errors coming from the adder 113, and outputs the coefficient conversion circuit 115 quantization.

At step S53, the quantization scheme 115 performs quantization conversion factor coming from the circuit 114 orthogonalen the th conversion, and outputs the quantized conversion coefficient on output.

At step S54 scheme 119 dekvantovanie provides dekvantovanie of the conversion coefficient, the quantized scheme 115 quantization, and outputs the coefficient conversion circuit 120 inverse orthogonal transformation.

At step S55 circuit 120 inverse orthogonal transform performs inverse orthogonal transformation for the conversion from the scheme dekvantovanie 119, and outputs the resulting image in deblocare filter 121.

At step S56 deblocare filter 121 filters for removing block noise, and outputs an image from which were fixed block noise, in the memory 122 frames.

At step S57, the memory 122 writes the image frames received from deblokiruyuschee filter 121.

At step S58 scheme 123 definition mode performs a procedure to determine the mode. During this procedure the determination of the mode determines the mode in which you want to generate the predicted image. This procedure definition mode will be described below with reference to the logic diagram shown in Fig.

At step S59 circuit 131 control determines whether to perform intra-frame prediction, on the basis of the operation of the circuit 123 definition mode

If at step S59 is defined to perform intra-frame prediction, then the circuit 126 intra-frame prediction performs such intra-frame prediction at step S60, and outputs the predicted image to the adder 113.

On the other hand, if at step S59 is determined that the intra-frame prediction to make is not required, i.e. you should perform interframe prediction, then the scheme 125 prediction/motion compensation performs this prediction/motion compensation at the step S61, and outputs the predicted image to the adder 113. This procedure prediction/motion compensation will be described below with reference to the logic diagram shown in Fig.

At step S62 circuit 116 lossless encoding compresses the conversion factor received from circuit 115 quantization, and outputs it to an intermediate buffer 117. In addition, the circuit 116 lossless encoding describes the identification flag in the image header in accordance with the information received from the circuit 123 definition mode, or describes the motion vector received from the circuit 125 prediction/motion compensation, the title of this image.

At step S63 intermediate buffer 117 temporarily stores the information received from the circuit 116 lossless encoding.

At step S64 schemes the management 131 determines was performed the above procedure for all macroblocks of one full frame. If it is determined that this procedure has not been performed for all of the macroblocks of one full frame, again follow the procedure for another macroblock from step S51.

On the other hand, if in step S64 it is determined that the procedure has already been performed for all of the macroblocks of one full frame, then the intermediate buffer 117 transmits the compressed image information in accordance with commands from the circuit 131 controls the step S65. The above procedure is performed for individual frame.

Next, the procedure of determination of the mode performed in step S58 shown in Fig, will be described with reference to the logical framework presented in Fig.

At step S81 scheme 141 intra-frame prediction and circuit 142 interframe prediction perform intra-frame prediction and inter prediction, respectively, for blocks of different sizes from one another, thereby generating the predicted image. The generated predicted image come into the scheme 143 calculate the forecasting errors.

At step S82 scheme 143 calculation error of prediction determines the difference between the original image and the corresponding projected image p is stepping from the respective circuits in the circuit 141 intra-frame prediction and from the circuit 162 unidirectional prediction, schema 163 bidirectional prediction and circuit 164 filtering in the scheme 142 interframe prediction. Scheme 143 calculation error of prediction outputs the signal of the residual error in the circuit 144 definitions.

At step S83, the circuit 144 definition fulfills the definition of a predictor for generating predicted image to transfer to the adder 113 on the basis of the intensity of the residual error signal coming from the circuit 143 calculate the forecasting errors.

At step S84, the circuit 144 definition displays information about the mode that represents information about the selected method of forecasting, the circuit 116 lossless encoding. After that, the process returns to step S58 shown in Fig, and performs the subsequent processing operation.

Next, the procedure of prediction/motion compensation performed at step S61 shown in Fig, will be described with reference to the logical framework presented in Fig.

At step S91 scheme 181 determine the motion vector estimates the motion vectors on the basis of the original image and reference frames.

At step S92 scheme 181 determining motion vectors checks determined whether the scheme 123 determine what processing should be performed in the prediction mode filtering.

If at step S92 revealed that is not installed, what processing should be done in prediction mode filter, then at step S93 perform unidirectional prediction or bidirectional prediction and generate the predicted image.

Namely, if it is determined that the processing should be performed in mode unidirectional prediction, motion vectors are passed from the schema 181 determining the motion vector in the diagram 182 unidirectional prediction and implement unidirectional prediction scheme 182 unidirectional prediction. In addition, if it is determined that the processing should be carried out in a mode of bidirectional prediction, motion vectors are passed from the schema 181 determining the motion vector in the diagram 183 bidirectional prediction and perform bi-directional prediction in the schema 183 bi-directional prediction. After the projected image is sent to the adder 113, the procedure returns to step S61, presented at Fig, and performs subsequent processing.

On the other hand, if at step S92, it was determined that the processing should be made in the prediction mode with filtering, then the scheme 184 forecasting allocates image with motion compensation from multiple reference frames and transmits these images with compensation movement is possible in scheme 185 filtering step S94. The motion vectors are passed from the schema 181 determining motion vectors in the diagram 184 forecasting and produce images with motion compensation using these vectors.

At step S95 scheme 51 calculating the difference (Fig) in the scheme 185 filter computes the difference between the image MS0with motion compensation and image MC1with motion compensation, and outputs the differential image in the filter 52 of the lower frequencies.

At step S96, the filter 52 of the lower frequencies from the schema 185 filter to filter out the lower frequencies in the difference image supplied from the circuit 51 calculating the difference, and outputs the resulting image in scheme 53 adjustment of the gain and the filter 54 of the upper frequencies.

At step S97 scheme 53 adjustment of the gear ratio in the scheme 185 filter adjusts the gain of the differential image coming from the filter 52 of the lower frequencies, and displays the image with the adjusted transmission efficiency in the adder 56.

At step S98, the filter 54 of the upper frequencies of the composition schema 185 filter to filter out high frequencies in the difference image supplied from the filter 52 of the lower frequencies, and displays the resulting image in scheme 55 adjustment factor.

At step S99 scheme 55 adjustment of gain in sozdavshimi 185 filter adjusts the gain of the image, coming from the filter 54 of the upper frequencies, and displays the image with the adjusted transmission efficiency in the adder 56.

At step S100, the adder 56 from the circuit 185 filtering summarizes the image coming from the circuit 53 adjustment of the gear ratio, and the image coming from the circuit 55 adjustment factor for obtaining high-frequency components. Found high-frequency components received from the adder 56 to the adder 57.

At step S101, the adder 57 of the composition schema 185 filtering summarizes the image coming from the adder 56, (high frequency components) with the image of MS0with motion compensation, and outputs the resulting image to the adder 113 in the quality of the projected image. After that, the procedure returns to step S61 shown in Fig, and is carried out subsequent processing.

Thus, coding is performed by using the predicted image generated by the prediction filter, which makes it possible to improve the coding efficiency.

In the above description, the circuit 45 and 185 filtering have the configuration shown in Fig, however, this configuration can be modified.

Fig is a block diagram illustrating another configuration example of the circuit 45 of the filter. Element of the configuration, the respective elements of the configuration shown in Fig assigned to the same digital positional notation. Redundant description will, where possible, omitted.

Scheme 51 calculating the difference shown in Fig, calculates the difference between the image MS0with motion compensation and image MC1with motion compensation, and outputs the differential image in the filter 52 of the lower frequencies.

The filter 52 of the lower frequencies to filter out the lower frequencies in the difference image supplied from the circuit 51 calculating the difference, and outputs the resulting image to the adder 57.

The adder 57 performs the summation of the image coming from the filter 52 of the lower frequencies, with the image of MS0with motion compensation, and outputs the obtained image as the predicted image.

When using the configuration shown in Fig, the amount of processing can be reduced compared to option using the configuration shown in Fig, so that it is possible to implement the high speed.

Fig is a block diagram illustrating another configuration example of the circuit 45 of the filter. Configuration elements, the respective elements of the configuration shown in Fig assigned to the same digital positional notation. Redundant description will be where this is possible, omitted.

In the circuit 45 of the filter shown in Fig, filtering is not used to the time domain signal, and the signal in the frequency domain. Both schemes 45 filter on Fig and 17 perform filtering of the signal in the time domain.

Scheme 51 calculating the difference shown in Fig, calculates the difference between the image MS0with motion compensation and image MC1with motion compensation, and outputs the differential image schema 201 orthogonal transformations.

Circuit 201 performs orthogonal transformation on the differential image orthogonal transformation represented by discrete cosine transform (DCT), Hadamard transform, and the transform of karunen-Loev (KLT), and outputs a signal obtained after the orthogonal transformation, a band-pass filter 202. Performing orthogonal transformation and implementation of the filtering of the signal in the frequency domain allows for more flexibility to implement filtering with high accuracy compared with the signal in the time domain.

In the case of applying a discrete cosine transform (DCT) as the orthogonal transform output signal DF obtained after orthogonal transformation represented by the following equation (10). In equation (10), DCT(X) to depict the place performing two-dimensional discrete cosine transform (DCT) on the signal X.

DF=DCT(D)...(10)

Band-pass filter 202 filters the signal output circuit 201 orthogonal transformations, and outputs the signal in a certain range.

Circuit 203 adjustment factor adjusts the gain of the output signal bodoano-pass filter 202 by multiplying it by a coefficient α, and adjusts the frequency components. The output signal XF circuit 203 adjustment of the gear ratio represented by the following equation (11). In equation (11) BPF(X) represents the execution papaano-pass filtering of the signal X.

XF=αBPF(DF)(11)

The circuit 204 inverse orthogonal transform performs inverse orthogonal transformation using the algorithm corresponding to the orthogonal transformation is performed in the circuit 201 orthogonal transformation to convert a signal in the frequency domain, coming from circuit 203 p is golinowski transfer coefficient, in the signal in the time domain. For example, if the schema 201 orthogonal transformation uses the discrete cosine transform (DCT) as the orthogonal transform circuit 204 inverse orthogonal transform performs the inverse transform DCT (IDCT). The output signal X of the circuit 204 inverse orthogonal transformation represented by the following equation (12). In equation (12) IDCT(X) is performing two-dimensional inverse transform IDCT on the signal X.

X=IDCT(XF)(12)

The adder 57 performs the summation signal X supplied from the circuit 204 inverse orthogonal transform to image MS0the motion-compensated in the time domain, and outputs the resulting image as the predicted image. The projected image S(i, j), representing the final output signal of the adder 57 is provided by the following equation (13).

S(i,j)=MC0(i,j)+X(i,j )...(13)

Thus, the projected image with high precision can be generated by performing filtering of the signal in the frequency domain.

In addition, in the above consideration of the prediction filtering is performed using two reference frames, however, as the reference frames can be used two or more frames.

Fig is a diagram illustrating an example of a case when using three keyframes.

In the example shown in Fig, if we take as a basis the time corresponding to the predicted frame, the reference frames are the frame preceding predicted frame at one time step, the frame preceding predicted frame at two time step, and the frame at three time steps before the predicted frame. Closest to the predicted frame, the preceding one time step the frame is designated as the reference frame R0the frame on one time step before the reference frame R0designated as a reference frame R1and the frame in one time step before the reference frame R1designated as a reference frame R2.

Fig is a block diagram illustrating a configuration example of filtering for the case when using three keyframes.

As shown in Fig, the circuit 211 of the filter formed by the circuit 221 filtering and circuit 222 of the filter. Each of these circuits 221 and 222 of the filter has the configuration shown in Fig, 17 or 18. The circuit 211 filter configured to operate with three inputs and one output by cascade connection circuits 45 of the filter, having a structure with two inputs and one output.

Further description here will be given assuming that the image motion compensation allocated from the reference frame R0is the image MS0with motion compensation, image motion compensation, selected from the reference frame R1is the image MC1with motion compensation and image motion compensation allocated from the reference frame R2is the image MS2with motion compensation. Image MC1and MS2the motion-compensated received in the diagram 221 filtering, and image MS0the motion-compensated is supplied to the circuit 222 filtering.

Circuit 221 of the filter performs filtering using image MC1and MC2with motion compensation in image quality MS0and MC1the motion-compensated shown in Fig and other, respectively, and outputs the intermediate output signal X, which is the result the volume of the filter, in the circuit 222 filtering.

The circuit 222 filter performs filtering using said intermediate output signal X and the image MS0with motion compensation in image quality MS0and MC1the motion-compensated shown in Fig and other, respectively, and outputs the filtering result to the output as a predicted image.

You can also apply the scheme 211 filtering, processing three such reference frame in the decoding device 1, shown in figure 3, or in the encoding device 101, shown at 11, instead of the circuit 45 of the filter.

In addition, the circuit 221 filtering and circuit 222 of the filter does not need to have the same configuration, and their individual configuration may differ from one another, so that, for example, one may have the configuration shown in Fig, and the other configuration is presented on Fig. In addition, you can also vary the parameters of the filters, taking into account the characteristics of the input/output obtained before and after filtering.

Circuit 211 filtering can filter for images with motion compensation allocated from the three reference frames, which are arranged in two directions along the time axis, but not for images with motion compensation allocated from the reference frames located in one n the Board along the time axis.

Note that when using frames preceding predicted frame and following him along the time axis, as reference frames, including the case described with reference to Fig.7, a parameter such as the ratio-of-way, used for filtering, you can dynamically change depending on the direction between reference frames along the time axis or the distance between the frames.

The transmission of compressed data of the image encoding device 101 decoding device 1 is realized by means of a variety of media including recording media such as optical disks, magnetic disks, flash memory devices, systems, satellite broadcasting, cable television, Internet and mobile phone communication network.

Now will be considered the motion compensation in the prediction filtering. Next will be described the configuration and operation principle of the circuit 44 prediction in the decoding device 1, the transmitting image motion compensation circuit 45 of the filter. This description applies to is shown in Fig scheme 161 forecasting and shown in Fig scheme 184 forecasting, which transmit image motion compensation in the circuit filter.

Motion compensation for decoding the image encoded in accordance with the standard H.264 is the e in units of macroblocks of 16×16 pixels, and in units of blocks obtained by further partitioning of macroblocks. This means that the motion compensation performed, focusing on the individual blocks obtained by dividing a macroblock to be decoded, and considering the values of the corresponding pixels of the corresponding block in the reference frame as the values of corresponding pixels in the target block based on the motion vector. In addition, during the above-described prediction filter this filter using the FIR filter is carried out for image motion compensation received as a result of executing the specified motion compensation, as an input image.

In this case, since the block size that is used as a unit to perform filtering, is small, its influence cannot be ignored. For example, to generate pixels near the edge of the projected image which should be formed by filtering, it is necessary to interpolate the pixels of the image with motion compensation, which is the input image, in accordance with the number of taps of the FIR filter (FIR).

Fig. presents a diagram illustrating an example of the interpolation pixels.

The input image of size 16×16 pixels shown in the upper part Fig, represents and what the imagination with motion compensation, formed by Assembly of individual units obtained through motion compensation. The output image size of 16×16 pixels shown in the lower part Fig, is a projected image formed by pixels, by performing filtering using FIR filter for image motion compensation, are presented in the upper part. Here it is assumed that the FIR filter has five branches. To obtain a pixel of the projected image requires five pixels of the image with motion compensation.

For example, to generate the pixel P1 on the lower left edge of the projected image of the required five pixels in the horizontal direction, so that the pixel P1 on the lower left edge representing a corresponding pixel of the image with motion compensation should be in the center. However, to the left of the pixel P1, as shown in broken lines, pixels no. In addition, to generate pixel P16 on the lower right edge of the projected image also needs five pixels in the horizontal direction, so that the pixel P16 on the lower-right corner represents the corresponding pixel in the image with motion compensation should be in the center. However, the right of the pixel P16, as shown in broken lines, PI is Seli not.

Therefore, the filtering procedure can be performed after the interpolation of missing pixels by copying the pixels P1 and P16 or perform the interpolation using pixels in symmetrical positions so that the border is in the center. However, the values of these pixels differ from the true values of the pixels indicated by broken lines, so that the accuracy of the values of pixels of the projected image is reduced. In order to avoid such degradation, it is necessary to use the true values of the pixels, however, from the reference frame in the existing motion compensation received only region whose size is equal to the block size, and therefore, such an interpolation is necessary.

Then in scheme 44 prediction motion compensation carried out in such a way as to generate macroblocks having a field, the number of pixels which corresponds to the number of taps of the FIR filter.

Fig is a diagram illustrating the principle of a process that uses motion compensation with the addition of fields section.

As shown by the arrow A1 Fig, the image in which the image of the macroblock size of 16×16 pixels added to the fields area, separated from the reference frame as an image with motion compensation and is considered as the input image for the filtering process.

Figure 3 presents a diagram illustrating an example of image motion compensation.

As shown in Fig, picture size (16+2A)×(16+2A) of the pixels obtained by adding a kind of strip width "a" of pixels around an image size of 16×16 pixels to extend this image extracted from the reference frame as an image with motion compensation. The "a" value is determined in accordance with the number of taps of the FIR filter.

Then do the filtering using such image motion compensation in the input image and transmit the output projected image size of 16×16 pixels as the image output of the filtering process, as shown by the arrow A2 in Fig. In the adder 15 shown in figure 3, this projected image is summed with the output image of the circuit 14 inverse orthogonal transform, and the image resulting from the summation, used as a macroblock constituting the decoded frame.

Thus, in the process of motion compensation in the circuit 44 forecasting using the fact that a keyframe is a frame from the past or from the future, close to the target frame, and that the entire frame can be used for motion compensation.

Through the use of image motion compensation, oblad the irradiation region fields corresponding to the number of taps of the FIR filter for performing filtering process, this process can be implemented using the real values of the pixels, so that the accuracy of the values of pixels of the projected image can be improved even in the case of generating pixels at the edges of the projected image. In other words, there is no need to interpolate the pixels for the implementation of the filtering process.

Because the accuracy of the values of pixels of the projected image can be increased, the accuracy of the decoded frame in the decoding device 1 can be increased and the residual error relative to the original image can be reduced and the efficiency of encoding in the encoding device 101 can be improved.

Fig is a block diagram illustrating an example configuration schema 44 prediction, shown in figure 5.

As shown in Fig, scheme 44 forecasting contains the schema 231 block break, the limiter buffer 232 and 233. The motion vector coming from the circuit 41 determination of the prediction mode, shown in figure 5, enters the limiter 232.

Circuit 231 split unit performs the partitioning of the macroblock to be decoded, and transmits information about the areas of the respective blocks in the limiter 232.

The limiter 232 focuses the individual blocks, the components of the macroblock to be decoded, and determines the block in the reference frame corresponding to the target block based on the motion vector supplied from the circuit 41 to determine the prediction mode. The specified block in the reference frame is a block of the same size as the target block. In the following description was found on the basis of the motion vector of the block in the reference frame corresponding to the target block, will if necessary be cited as the reference block.

In addition, the limiter 232 determines whether the considered reference block in contact with the boundary of the macroblock including the reference block. If the limiter 232 determines that the reference block is not in contact with the boundary of the macroblock, the limiter 232 reads the information about the reference block from the memory 19 shots and passes this information as values of pixels of the target block in the buffer 233.

If the limiter 232 determines that the reference block is in contact with the boundary of the considered macroblock, the limiter 232 reads from the memory 19 personnel information about this support block, and information about the fields that represent the band in contact with the reference block and outside of the macroblock including the reference block, and outputs this information to the buffer 233.

After attention was alternately focus is on all blocks, thereby obtaining information about the image motion compensation, is shown in Fig, where the macroblock to be decoded, added a field, the buffer 233 passes this information to the circuit 45 of the filter. In the process of prediction filtering using multiple images with motion compensation, and individual images with motion compensation to generate in scheme 44 prediction in the same way.

Next, the process of motion compensation performed in figure 4 prediction will be described with reference to the logic diagram of the sequence of operations shown in Fig. The procedure presented on Fig, is to generate individual images with motion compensation.

At step S1, the circuit 231 split blocks performs a partitioning of the macroblock to be decoded, and transmits information about the areas of the respective blocks in the limiter 232.

Fig is a diagram illustrating an example of the partitioning of the macroblock to be decoded.

In the example shown in Fig, the macroblock size of 16×16 pixels is divided into blocks B11 at B44.

At step S2 limiter 232 focuses on a single block.

At step S3, the limiter 232 determines the reference frame.

At step S4 limiter 232 evaluates whether the accuracy of the motion vector, the village is upavshego from the circuit 41 determination of the prediction mode, decimal precision.

If in step S4 it is determined that the accuracy of the vector is not decimal precision limiter 232 indicates the position of the reference block corresponding to the target block based on the motion vector in integer precision at step S5.

On the other hand, if in step S4 it is determined that the accuracy of the motion vector is the decimal precision limiter 232 interpolates pixels using interpolation filter corresponding to the standard, and specifies the reference block at the location indicated by the motion vector with decimal precision, at step S6.

After you specify a reference block limiter 232 evaluates contacts if one of the sides of the bearing block with the boundary of the macroblock that includes the reference block, at step S7.

Here such assessment is made based on the criteria described by the following equations (14) through (17). If at least one of these conditions is satisfied, a decision is made that one of the sides of the bearing block is in contact with the boundary of the macroblock. If not satisfied none of the conditions, a decision that neither side of the reference block is not in contact with the boundary of the macroblock. Write (bk_pos_x, bk_pos_y) represents the position of the upper left vertex of the reference block, and the write (bk_width, bk_height) represents the width and you the GTC of the reference block. Write (MB_pos_x, MB_pos_y) represents the position of the upper left vertex of the macroblock, including consideration of the reference block.

MB_pos_x=bk_pos_x(14)

MB_pos_y=bk_pos_y(15)

MB_pos_x+16=bk_pos_x+bk_width(16)

MB_pos_y+16=bk_pos_y+bk_height(17)

Fig is a diagram illustrating an example of the reference frame.

In the example shown in Fig, to depict what aulani nine macroblocks with MB11 on MW. Here will be described the case when the reference blocks corresponding to the individual blocks obtained by dividing a macroblock to be decoded represent the individual blocks constituting the macroblock MB, highlighted by surrounding it with a bold line L.

Fig is a diagram illustrating example blocks constituting the macroblock MV shown in Fig. The blocks b11 at b44 presented on Fig, serve as reference blocks for blocks, respectively, with B11 at B44 depicted on Fig and components of the macroblock to be decoded.

For example, when attention is focused on the block B11, depicted on Fig, block b11, shown in Fig and employee reference block for the block B11, satisfies the conditions represented by equations (14) and (15), and therefore is recognised in contact with the boundary of the macroblock MB.

When attention is focused on the blocks B12 and B13 shown in Fig, blocks b12 and b13, depicted on Fig and employees support blocks for these blocks satisfy the condition represented by equation (15), and therefore are recognised in contact with the boundary of the macroblock MB.

When the focus is on the block B14, depicted on Fig, block b14, shown in Fig and employee reference block for the block B14, satisfies the conditions represented by equations of the (15) and (16), and therefore is recognised in contact with the boundary of the macroblock MB.

When attention is focused on the blocks B21 and V shown in Fig, blocks b21 and b31 shown on Fig and employees support blocks for these blocks satisfy the condition represented by equation (14), and therefore are recognised in contact with the boundary of the macroblock MB.

When attention is focused on the units In 24 and V shown in Fig, blocks b24 and b34 depicted on Fig and employees support blocks for these blocks satisfy the condition represented by equation (16), and therefore are recognised in contact with the boundary of the macroblock MB.

When the focus is on the block B41 content shown on Fig, block b41, shown in Fig and employee reference block for the block B41 content that satisfies the conditions represented by equations (14) and (17), and therefore is recognised in contact with the boundary of the macroblock MB.

When attention is focused on the blocks V and W shown in Fig, blocks b42 and b43 depicted on Fig and employees support blocks for these blocks satisfy the condition represented by equation (17), and therefore are recognised in contact with the boundary of the macroblock MB.

When the focus is on the block B44 depicted on Fig, block b44 shown in Fig and employee jacket for nl is ka B44, satisfies the conditions represented by equations (16) and (17), and therefore is recognised in contact with the boundary of the macroblock MB.

When attention is focused on the blocks B22, b, V32 and V shown in Fig, blocks b22, b23, b32, and b33, depicted on Fig and employees support blocks for these blocks do not satisfy any of these conditions and therefore are recognised as having no contact with the boundary of the macroblock MB.

Returning to Fig, if in step S7 it is recognized that one side of the support block is in contact with the boundary of the macroblock, the limiter 232 determines at step S8 whether the top of the support block with the top of the macroblock including the reference block. To determine whether the top of the support block with the top of the enclosing macroblock on the basis of conditions that satisfies the reference block.

These conditions for the case when the top of the support block coincides with the top macroblock is expressed by the following equations (18) through (21). If you have any of these conditions, this means that the top of the support block coincides with a vertex of the macroblock.

MB_pos_x=bk_pos_x

and

MB_pos_y=bk_pos_y...(18)

MB_pos_x=bk_pos_x

and

MB_pos_y+16=bk_pos_y+bk_height...(19)

MB_pos_x+16=bk_pos_x+bk_width

and

MB_pos_y=bk_pos_y...(20)

MB_pos_x+16=bk_pos_x+bk_width

and

MB_pos_y+16=bk_pos_y+bk_height...(21)

For example, the block b11 shown in Fig and which is the reference block when the focus is on the block B11 shown in Fig, satises the above equations (18) equations (14) and (15)), which determined that the top of the support block coincides with a vertex of the macroblock.

Unit b14 shown in Fig and which supports the first unit in the case when the focus is on the block 14 shown in Fig, satises the above equations (20) (equations (15) and (16)), which determined that the top of the support block coincides with a vertex of the macroblock.

Block b41, shown in Fig and which is the reference block when the focus is on the block B41 content shown on Fig, satises the above equations (19) equations (14) and (17)), which determined that the top of the support block coincides with a vertex of the macroblock.

Block b44 shown in Fig and which is the reference block when the focus is on the block B44 shown in Fig, satises the above equations (21) (equations (16) and (17)), which determined that the top of the support block coincides with a vertex of the macroblock.

When attention is focused on some other block shown in Fig, the reference block does not satisfy any of the conditions of equations (18) through (21), whereupon it is determined that the peak of the reference block does not coincide with the peak of the macroblock.

If in step S8 it is determined that the top of the support block coincides with the top macroblock, the limiter 232 sets the area size and×a pixels in contact with the respective top macroblock at one point and beyond macrobi the ka, as the fields on the step S9.

On the other hand, if in step S8 it is determined that the peak of the reference block does not coincide with the top macroblock, or if at the step S9 is set to the fields area, the process goes to step S10.

At step S10 limiter 232 sets the area size and×w pixels in contact with the reference block and outside of the macroblock including the reference block, as the fields. Here, "w" denotes the length of a side of the reference block in contact with the boundary of the macroblock. When the reference block is selected as shown in Fig, w=4.

At step S11 limiter 232 reads the information about the specified reference block to which is added a field, from the memory 19 of the frame and passes the buffer 233.

On the other hand, if in step S7 it is determined that none of the sides of the bearing block has no contact with the boundary of the macroblock, the limiter 232 reads the information of the reference block from the memory 19 of the frame and passes the buffer 233 to step S12.

At step S13 limiter 232 checks whether already the focus turns on all blocks. If it is determined that the focus was not yet on all the blocks, the procedure returns to step S2, where the focus is on another block, then a similar process is repeated.

If in step S13 it was determined that attention has already been the focus alternately on all blocks, the buffer 233 transmits the macroblock together with the field that serves as the image motion compensation, the circuit 45 filtering step S14 and the process terminates.

Next will be described specific examples of fields section.

For example, when attention is focused on the block B11 shown in Fig, the top of the block b11, employee reference block coincides with the top macroblock MV, and thus shown in Fig region A1 size and×a pixels in contact with the specified matching top at one point, is set as a scope field. In addition, shown in Fig region A2 and A3 in size and×w pixels each in contact with the block b11 and are positioned outside of the macroblock MB, also installed as areas of fields. Unit b11, employee reference block, and set specified area A1, A2 and A3 fields cut out from the reference frame and store in the buffer 233 in the quality of the information about the block B11, serves as the target block.

The area A1 is an area composed of pixels of the macroblock MB 11 (Fig)located outside of the macroblock MB. The region A2 is a region composed of pixels of the macroblock MB, and the area A3 is an area composed of pixels of the macroblock MB12.

When the focus is on the block B12, shown in Fig depicted in Phi is .30 region a11, an area size and×w pixels in contact with the block b12, employee reference block, and outside of the macroblock MB set as a scope field. Block b12 that serves as a reference block, and the area A11 of the fields set in this way, cut out from the reference frame and store in the buffer 233 in the quality of the information about the block B12 that serves as the target block. Region a11 is an area composed of pixels of the macroblock MB12 located outside of the macroblock MB.

Similarly, when attention is focused on the block B13 shown in Fig depicted on Fig region A21, an area the size and×w pixels in contact with the block b13, employee reference block, and outside of the macroblock MB set as a scope field. Unit b13, employee reference block, and a field A21 installed this way, cut out from the reference frame and store in the buffer 233 in the quality of the information about the block B13, serves as the target block. The area A21 is an area composed of pixels of the macroblock MB12 located outside of the macroblock MB.

When the focus is on the block B14 shown in Fig, the top of the block b14, employee reference block coincides with the top macroblock MV, and thus depicted in Fig region A31, representing about the region size and×a pixels, in contact with a matching top at one point, set as a scope field. In addition, in this case shown in Fig region A32 and A33, each of which represents a region of size and×w pixels in contact with the block b14 and outside the macroblock MB set as areas of fields. Unit b14 that serves as a reference block, and the fields A32 and A33, established by the above method, cut out from the reference frame and store in the buffer 233 in the quality of the information about the block B14, serves as the target block.

The area A31 is an area composed of pixels of the macroblock MB located outside of the macroblock MB. The area A32 is an area composed of pixels of the macroblock MB, and A33 is an area composed of pixels of the macroblock MB12.

When the focus is on the block B21 shown in Fig depicted on Fig area A41, an area the size and×w pixels in contact with the block b21, employee reference block, and outside of the macroblock MB set as a scope field. Unit b21, employee reference block, and a field A41 installed this way, cut out from the reference frame and store in the buffer 233 in the quality of the information about the block B21, serves as the target block. Area A41 represents the way the second region, composed of pixels of the macroblock MB located outside of the macroblock MB.

When the focus is on the block B22 shown in Fig, the block b22, the clerk of the reference block is a block that is not in contact with the boundary of the macroblock MB, and thus, no fields fields are not set, as shown in Fig. This block b22 cut out from the reference frame and store in the buffer 233 in the quality of the information about the block B22, serves as the target block.

Similarly, when attention is focused on the block b shown in Fig, block b23, employee reference block is a block that is not in contact with the boundary of the macroblock MB, and thus, no fields fields are not set, as shown in Fig. This block b23 cut out from the reference frame and store in the buffer 233 in the quality of the information about the block b that serves as the target block.

When the focus is on the block In 24 shown in Fig depicted on Fig area A51 motorway, an area the size and×w pixels in contact with the block b24, employee reference block, and outside of the macroblock MB set as a scope field. Block b24 that serves as a reference block, and the fields area of the A51 motorway, set this way, cut out from the reference frame and store in the buffer 233 as information about the unit is 24, serves as the target block. Area A51 motorway is an area composed of pixels of the macroblock MB located outside of the macroblock MB.

Similarly, when attention is focused on units with V 44, shown in Fig set properly the fields and stores them in the buffer, together with information about the considered reference block.

Fig is a diagram illustrating an example of image motion compensation coming from the output of the circuit 44 forecasting.

Shown in Fig image motion compensation to generate using the above described method on the basis of information about the reference blocks and information about the areas of the fields read from the memory 19 frames. Shown in Fig image motion compensation represents the image size (16+2A)×(16+2A) pixels.

In scheme 45 filtering for multiple images with motion compensation, generated as described above, perform the differencing, the filter using the FIR filter, the adjustment of the gear ratio, summation, etc. and forming the projected image.

Fig is a diagram illustrating an example of the FIR filter included in the filter 52 of the lower frequencies and the filter 54 of the upper frequencies in the circuit 45 of the filter.

For example, in the use of the implement shown in Fig FIR filter, in which the number of taps is five, set the value of "a"equal to two, and the input circuit 45 filter picture with motion compensation, representing the image of 20×20 pixels, obtained by extending each of the four sides of the macroblock size of 16×16 pixels by two pixels.

Width (pixels) is determined using the following equation (22), where it is assumed that the number of taps of the FIR filter is So the Entry floor(x) denotes a function for calculating the maximum integer not greater than X.

afloor(T/2)...(22)

Thus, the width "a" is small, when there is little number of taps T, and the width "a" is high when a large number of taps So Note that for performing two-dimensional filtering in the filter 52 of the lower frequencies and similar schemes are used for the FIR filter with five taps to the input of five pixels arranged in the horizontal direction, and output a single pixel and the FIR filter with five taps to enter five pixels in the vertical direction, and output a single pixel.

Fig is a diagram illustrating an example of the filtering process, performed the CSO for image motion compensation, with a field.

The input image size (16+2A)×(16+2A) pixels shown in the upper part Fig, is a picture with motion compensation, generated using the above described method of motion compensation performed in the circuit 44 forecasting. The output image size of 16×16 pixels shown in the lower part, formed by the pixels, obtained by filtering to be performed with respect to image motion compensation, are presented in the upper part, using the FIR filter.

For example, in the case of generating the pixel P1 on the lower left end of the projected image can be obtained the value of this pixel P1 by entering the five pixels in the horizontal direction, so that the pixel P1, which represents the corresponding pixel in the image with motion compensation, is located in the center of this five pixels P21, P22, P1, P2 and P3, in the FIR filter (FIR). The pixels P21 and P22 are in the fields section.

In addition, in the case of generating pixel P16 on the lower right end of the projected image can be obtained the value of this pixel P16 by entering the five pixels in the horizontal direction, so that the pixel P16, representing the corresponding image pixel with motion compensation, is located in the center is the five pixels R, R, P16, P31 and R, in the FIR filter. The pixels P31 and R are in the fields section.

Accordingly, the filtering procedure can be performed using the real values of the pixels, which improves the accuracy of the values of pixels of the projected image. Since the two-dimensional image has a strong correlation in the spatial directions, we can assume that the values of pixels within the macroblock are highly positively correlated with the values of pixels outside of the macroblock. Therefore, values of pixels of the projected image of high reliability in comparison with a case of performing filtering using interpolation from the pixels in the macroblock.

In General, from the point of view of the characteristics it is preferable that the number of taps of the FIR filter. However, when a large number of taps increases the probability of using the values of the pixels outside the block, during processing, so that the range within which are affected by the values of the pixels of the output image are expanded.

In particular, in the case of performing filtering in units of macroblocks using a FIR filter with three taps as a lowpass filter described by equation (4)are affected about 25% of the pixels full frame due to the lack of pixels in the input image is the situation. In addition, when using a FIR filter with five taps, the process affects about 50% of the pixels full frame. When using a FIR filter with seven taps are affected by approximately 73% of the pixels full frame.

When using the above image motion compensation as the input image for the process of filtering this filtering process can be implemented using a FIR filter with a large number of taps.

According to the above description, for the generation of a pixel of the projected image to the inputs of the FIR filter serves several pixels in a horizontal direction, so that the corresponding image pixel with motion compensation is in the center of this group of pixels, but the group of pixels are supplied to the inputs of the filter is not limited to pixels arranged in the horizontal direction, or pixels, arranged in a vertical direction. For example, can also be coupled to the inputs of the filter corresponding to the pixel of the image with motion compensation and the pixels above, below, left and right of that pixel, or to submit to the inputs of the filter corresponding to the pixel of the image with the motion-compensated pixels from the top right, top left, bottom right and bottom left of this pixel.

Fig and fremstillet scheme, to illustrate the results obtained by coding using the predicted image generated by the circuit 45 of the filter shown in Fig.

The graph on Fig illustrates the amount of encoded data generated in the encoding process.

The horizontal axis shows the number of frames. For example, a "1" on the horizontal axis represents the first frame of the moving image to be processed, and "2" represents the second frame. On the vertical axis the amount of code of the individual frames in units of bits. The smaller this amount of code, the higher the compression ratio of the frame.

Line L1represents the amount of code in the case where encoding is performed using conventional AVC standard, and the line L2. represents the amount of code in the case where encoding is performed using the prediction filter. In this example, the result of compression (infra-picture or I-frame) of one frame is inserted every 15 frames, and 14 frames other than I-frame, considered as the predicted P-frames (P-pictures).

As shown in Fig, an I-frame is the same as in the case of the use of the AVC standard, and in the case of the encoding algorithm using prediction filtering, so that the amount of generated code are the same. In addition to the CSO, for coding using prediction filter requires two reference frame, resulting in the amount of code of the second P-frame, for which only the first I-frame can be used as a reference frame, has the same value as the amount of code in the case of the use of the AVC standard. As for the code amounts for other P-frames, using a coding method using prediction filtering these values, as shown by the line L2are less than the values represented by the line L1.

The reason that the amount of generated code can be reduced when applying the encoding method using the prediction filter is a high precision of the projected image and that the amount of encoded data of the residual error can be reduced compared with a case of using the AVC standard.

The graph on Fig illustrates the image quality of the transmitted coded data.

The horizontal axis on Fig similarly, the horizontal axis on Fig represents the frame number. The vertical axis represents the value of the PSNR (signal-to-noise ratio for the image). The value of PSNR is an objective measure, so that the similarity of the considered image with the original image the better (higher quality pictured is I), the greater the value. The unit of measurement of this ratio is [dB].

Line L11represents the value of the ratio PSNR for the case when the encoding is done using the usual algorithm AVC, and the line L12represents the value of the ratio PSNR for the case where encoding is performed using the prediction filter. For the same reasons as in the case of Fig, the values of PSNR for the I-frame and the second P-frame are also the same as in the case of application of the AVC standard, and in situations coding using prediction filtering.

On the other hand, as for the other P-frames, the values of PSNR for application coding using prediction filter is represented by the line L12are more values relationships PSNR for application of the AVC standard, represented by the line L11.

The reason that the ratio PSNR, i.e. the image quality can be increased by applying an encoding algorithm using prediction filtering, is that the accuracy of the projected image can be increased.

The above sequence of processes may be implemented in hardware or software. In the case of software how to implement is Itachi this sequence of processes of the program, components corresponding software is installed from the recording media programs in a computer incorporated in dedicated hardware, or in a personal General-purpose computer capable of performing various functions by installing various programs of the type, or similar device.

Fig is a block diagram illustrating an example hardware configuration of a personal computer that performs the above-described sequence of processes in accordance with the program.

The Central processor CPU 251, a persistent storage device (ROM) 252 and a random access memory (RAM) 253 is connected via bus 254.

Bus 254 is connected to the interface 255 input/output. With this interface 255 I/o connected device 256 input formed by a keyboard, mouse, microphone and the like, the device 257 conclusion, including a display, a loudspeaker, and the like, a storage device 258 includes a drive on your hard disk, nonvolatile memory and the like, the module 259 connection, including network interface and the like, and the actuator 260 to removable media 261 records, such as an optical disk or semiconductor storage device.

In the computer configured as described above, the processor CPU 251 loads the program, kept the Yuexiu storage device 258, in the RAM 253 interface 255 I/o and bus 254 and executes the program, thereby realizing the above-described sequence of processes.

The program executed by the processor CPU 251, deliver in a record, for example, on a removable media 261 writing or by cable or wireless communication channel, such as LAN network connection, Internet, or digital broadcasting, and is installed in the storage device 258.

In addition, the program executed by the computer may be a program for performing the processes sequentially in time in accordance with the described order, or a program that executes the processes in parallel or at the right times, for example when you call.

Range variants of the present invention is not limited to the above option. Here there are a variety of changes that are not beyond the scope of the present invention.

For example, the decoding device 1 or the encoder 101, described above, can be applied in any electronic device. Examples of this will be discussed below.

Fig is a block diagram illustrating a main example configuration of a television receiver using the decoding device to which is applied the present invention.

The television receiver is 300, shown in Fig, includes tuner 313 for receiving terrestrial television, video decoder 315, the processor 318 of the video signal generator 319 graphics, circuit 320 control panel and panel 321 of the display.

Tuner 313 for receiving terrestrial television receives the broadcast signal from the system analog terrestrial broadcasting via an antenna, demodulates the signal, receives the signal and transmits it to the video decoder 315. This video decoder 315 performs the decoding of the video signal coming from the tuner 313 for receiving terrestrial television transmission, and transmits the received digital signal component in the processor 318 video.

The processor 318 video performs some processing, such as noise from the structure of the video data received from video decoder 315, and transmits the received video data generator 319 graphics.

Generator 319 graphics generates video data of the program that need to be on the panel 321 of the display image data by performing the process in accordance with the application received through a communication network or other similar method, and transmits the generated video data and the image data circuit 320 control panel. In addition, the generator 319 graphics performs as necessary processes such as the generation of a video (graphics) to represent the and the screen, used by the user to select viewing television programs or other object, the imposition of these graphics video on a video viewing program to obtain the total video data and transmitting the resulting total video device 320 control panel.

The device 320 control panel controls panel 321 of the display in accordance with the video data supplied from the generator 319 graphics, and causes the image representation of the program and the various screens described above, the panel 321 of the display.

Panel 321 of the display may be based on a liquid crystal display (LCD) or similar device and represents an image of a program or other image under the management device 320 control panel.

In addition, the television receiver 300 has an analog-to-digital Converter 314 audio signal, the processor 322 of the audio signal, the circuit 323 echo cancellation/audio synthesis signal, the amplifier 324 audio signal and the speaker 325.

Tuner 313 for receiving terrestrial television demodulates the received broadcast signal to obtain not only video but also audio. Tuner 313 for receiving terrestrial television transmits the received audio signal in analog-to-digital Converter 314 audio signal.

Analog-to-digital conversion max the Converter 314 audio signal, performs analog-to-digital conversion of the audio signal, coming from the tuner 313 for receiving terrestrial television transmission, and transmits the digital audio signal processor 322 of the audio signal.

The processor 322 of the audio signal, performs some processing, such as noise, the audio data coming from the analog-to-digital Converter 314 audio, and transmits the received audio data in the schema 323 echo cancellation/synthesis audio.

Circuit 323 echo cancellation/synthesis audio signal transmits audio data received from the processor 322 of the audio signal, the amplifier 324 audio.

The amplifier 324 audio performs d / a conversion and amplification of the audio data received from circuit 323 echo cancellation/audio synthesis signal, to adjust the audio signal to a certain volume and then outputs the audio signal through a speaker 325.

In addition, the television receiver 300 includes a digital tuner 316 and the MPEG decoder 317.

The digital tuner 316 receives the signal of digital broadcasting (terrestrial digital broadcasting, digital broadcasting through the BS (Broadcasting satellite)/C8 (communications Satellite)via an antenna, demodulates the signal, extracts from it the transport stream MPEG-TS and transmits the MPEG decoder 317.

The MPEG decoder 317 descrambled transport stream MPEG-TS supplied from the digital tuner 316, and a thread that includes these programs are subject to the play the structure (for viewing/listening). The MPEG decoder 317 performs the decoding of the audio packets forming the selected stream, and transmits the audio data to the processor 322 of the audio signal, and decodes the video packets that make up the specified dedicated thread, and transmits the received video data processor 318 video. In addition, the MPEG decoder 317 transmits the data of the electronic program guide (EPG), selected from the transport stream MPEG-TS, the processor CPU 332 on the path, not shown in this drawing.

The television receiver 300 uses the decoding device 1 described above, as the MPEG decoder 317, which decodes the video packets in this way. Thus, as in the case of the decoding device 1, the MPEG decoder 317 performs decoding using the predicted image generated by the prediction filter, and as a result, it is possible to obtain a decoded image with high resolution more effectively through the use of image correlation in time.

Video data supplied from the MPEG decoder 317, is subjected, as in the case of video data coming from the video decoder 315, a certain processing in the processor 318 video and placed properly on top of the processed video other video data or similar information generated by the generator 319 graphics. Polucen the e video go to the panel 321 of the display through the circuit 320 control panel, so what appears on the display corresponding to this image data.

The audio data supplied from the MPEG decoder 317, is subjected, as in the case of audio data coming from the analog-to-digital Converter 314 audio signal, some processing in the processor 322 of the audio signal is passed to the amplifier 324 audio signal through the circuit 323 echo cancellation/audio synthesis signal and then perform digital-to-analog conversion and amplification of the data. As a result, the loudspeaker 325 outputs the sound, the volume was adjusted up to a certain level.

In addition, the television receiver 300 has a microphone 326 and analog-to-digital Converter 327.

This analog-to-digital Converter 327 receives the audio signal corresponding to the sound from the user picked up by the microphone 326, built in the television receiver 300, for use in an audio conversation, performs analog-to-digital conversion of the received audio signal and transmits the digital audio data in the schema 323 echo cancellation/audio synthesis signal.

When receiving the audio data of the user (user a) of the television receiver 300 from the analog-to-digital Converter 327 circuit 323 echo cancellation/audio synthesis signal performs acoustic echo cancellation in audio data from user and triggers the output audio data received PU is eat synthesis with other audio data, through the amplifier 324 audio signal and then through a speaker 325.

In addition, the television receiver 300 has the audio codec 328, the internal bus 329, SDRAM (Synchronous dynamic RAM) 330, flash memory 331, the CPU processor 332, a USB interface (universal serial bus) 333 and the network interface 334.

Analog-to-digital Converter 327 receives the audio signal from the user captured by the microphone 326, installed in the television receiver 300 for voice conversation, performs analog-to-digital conversion of the received audio signal and transmits the digital audio data audio codec 328.

The audio codec 328 converts the audio data supplied from the analog-to-digital Converter 327, data in a format for transmission over the communication network and transmits the data to the network interface 334 via the internal bus 329.

The network interface 334 is connected to the communication network via a cable attached to a network terminal 335. This network interface 334 transmits the audio data supplied from the audio codec 328, for example, another device connected to the communication network. In addition, the network interface 334 receives through the network terminal 335, for example, audio data transmitted from another device connected to the communication network, and transmits the audio data to the audio codec 328 via the internal bus 329.

The audio codec 328, Preobrazhenie di put, coming from the network interface 334, the data in a certain format and transmits the data to the schema 323 echo cancellation/synthesis audio.

Circuit 323 echo cancellation/audio synthesis signal performs echo cancellation for audio data supplied from the audio codec 328, and causes the audio data obtained by synthesis with other Audi passed through the amplifier 324 audio signal and then through a speaker 325.

Storage device SDRAM 330 stores different types of data needed by the processor CPU 332 for processing.

Flash memory 331 records a program executed by the processor CPU 332. processor CPU 332 reads the program recorded in the flash memory 331 in the appropriate time, for example when switching on the television receiver 300. In the flash memory 331 write data of the electronic program guide obtained from the signal of the digital broadcasting, and data received from a particular server via the network connection.

For example, the flash memory 331 stores the transport stream MPEG-TS including the content data received from a specific server through the communications network under the control of the processor the CPU 332. Flash memory 331 transmits the transport stream MPEG-TS to the MPEG decoder 317 via the internal bus 329 running, for example, processor CPU 332.

The MPEG decoder 317 processes the transport stream MPEG-TS method, an is a logical occasion of the transport stream MPEG-TS, coming from the digital tuner 316. Thus, the television receiver 300 may receive the content data composed of video data, audio data, and such data through the communication network, decode the data content using the MPEG decoder 317 and present video data on the display or output the audio signal.

In addition, the television receiver 300 also includes a photo receiver 337, receiving the radiation of the infrared signal transmitted by the remote 351 remote control.

The photodetector 337 receives infrared rays from the remote 351 remote control and transmits a control code indicating the content of user commands defined in the demodulation, the Central processor CPU 332.

Processor CPU 332 executes the program recorded in the flash memory 331, and controls the overall operation of the television receiver 300 in accordance with a control code or a similar code received from the sensor 337. Processor CPU 332 is connected to the individual nodes of the television receiver 300 according to the paths that are not shown on the drawing.

USB interface 333 transmits/receives data to/from an external device, television receiver 300 connected via the USB cable attached to the USB terminal 336. The network interface 334 is connected to the communication network via a cable attached to the network is erminal 335, and also transmits/receives data other than audio data to/from different types of devices connected to the network connection.

The television receiver 300 uses the decoding device 1 as the MPEG decoder 317, which allows him to generate a projected image with high accuracy without increasing the processing load. As a result, the television receiver 300 may receive the decoded image with a higher resolution based on the broadcast signal received through the antenna, or data content received over a communication network, and to present the image on the display.

Fig is a block diagram illustrating a main example configuration of a mobile phone that uses the decoding device and the encoder, in which is applied the present invention.

The mobile phone 400, shown in Fig, includes main unit 450 controls configured for joint management of individual blocks, block 451 power block 452 the input control unit 453, the image encoding, interface 454 camcorder, block 455 control LCD display, decoder 456 image, the demultiplexer 457, block 462 recording/playback, the modulator 458 and the audio codec 459. They are all connected via bus 460.

In addition, the mobile phone 400 in which incorporates both the working press 419, the camcorder 416 on the basis of charge-coupled devices (CCD), liquid crystal display 418, a storage unit 423, the transceiver 463, antenna 414, 421 microphone and speaker 417.

When the user presses end call and power on the unit 451 power supplies power from the battery to each individual unit of the phone, thereby translating the mobile phone 400 in working condition.

The mobile phone 400 performs various operations, such as transmission/reception of the audio signal, the transmission/reception of e-mail or image data, the image reading or recording data in different modes, such as voice call or a data transmission mode, under control of the main unit 450 controls containing a processor CPU, a ROM, etc.

For example, a voice call, the mobile phone 400 converts audio to DECA 459 audio signal captured by the microphone 421, digital audio data, performs the expansion of the range of these digital audio data using a modulator/demodulator 458 and performs digital-to-analog conversion and frequency conversion using transceiver 463. This mobile phone 400 transmits transferable signal obtained by conversion to a base station, not showing what nnuu in the drawing, via the antenna 414. Specified transferable signal (audio signal)transmitted to the base station, send a mobile phone on the other end of the call through the telephone network of General use.

In addition, in the mode, for example, a voice call, the mobile phone 400 amplifies the received signal, which was adopted by the antenna 414, using the transceiver 463, and then performs frequency conversion and analog-to-digital conversion of the signal, performs a compression of the spectrum of the signal through the modulator/demodulator 458 and converts the resultant signal into an analog audio signal using the audio to the deck 459. The mobile phone 400 outputs obtained by converting an analog audio signal through a speaker 417.

Moreover, for example, in the case of transferring e-mail messages in the data transmission mode of the mobile telephone 400 receives using block 452 control input of the text data of the e-mail entered by operating the keys 419. The mobile phone 400 processes the text data using the main unit 450 controls and presents the data obtained in the form of an image on the liquid crystal display 418 via the unit 455 control LCD display.

Next, the mobile phone 400 generates usage is of main unit 450 controls e-mail data on the basis of the specified text data or user commands, taken by block 452 control input. The mobile phone 400 performs the expansion of spectrum for e-mail data using a modulator/demodulator 458 and performs digital-to-analog conversion and frequency conversion using transceiver 463. The mobile phone 400 transmits the resulting conversion of the signal to be transferred to the base station, not shown in the drawing, via the antenna 414. This transferable signal (e-mail)transmitted to the base station, sent to some address through the communication network and a mail server or similar system.

In addition, for example, when receiving e-mail messages in the data transmission mode of the mobile telephone 400 receives the signal transmitted from the base station via the antenna 414 using transceiver 463, amplifies this signal and then performs frequency conversion and analog-to-digital conversion. Next, the mobile phone 400 performs the compression spectrum of the received signal in the modulator/demodulator 458 in order to restore the original e-mail data. The mobile phone 400 is recovered email data on the liquid crystal display 418 via the unit 455 control this LCD display.

In addition, mobilenation 400 may also write (save) received e-mail data in the storage device 423 by block 462 recording/playback.

Storage device 423 represents any rewritable recording media. As a storage device 423 is possible to use a semiconductor memory device such as RAM or built-in flash memory, hard drive, magnetic disk, or a removable recording medium such as a magnetic disk, a magnetooptical disk, an optical disk, a USB memory or memory card. Of course, can also be used any other media type.

Moreover, in the case of, for example, transmission of the image data in the data transmission mode of the mobile telephone 400 generates image data by reading the image by using a CCD camera 416. CCD camera 416 has an optical device, such as a lens and aperture, and a charge-coupled device (CCD)that serves as a photoelectric conversion element, reads the image of the object, converts the intensity of received light into an electrical signal and generates image data of the subject. The encoder 453 image compresses and encodes these image data through the interface 454 camera using some encryption algorithm, such as MPEG2 or MPEG4, with the aim of converting them into encoded image data.

The mobile phone 400 uses encodes eliminate the STV 101, described above, as of encoder 453 image, performing such a procedure. Thus, as in the case of encoder 101, the encoder 453 performs image coding using the predicted image generated by the prediction filter, so that the projected image includes a large number of high-frequency components and has a small difference relative to the original image, resulting in the amount of code assigned to the residual error can be reduced, and the coding efficiency can be improved.

In addition, at the same time, the mobile phone 400 is performed using the audio codec 459 analog-to-digital conversion of the audio signal captured by the microphone 421 during reading of an image by a CCD camera 416, and then encodes this audio.

The mobile phone 400 multiplexes using the demultiplexer 457 coded image data supplied from the encoding device 453 image, and the digital audio data supplied from the audio codec 459, using some algorithm. The mobile phone 400 performs the expansion of the spectrum of the received multiplexed data using a modulator/demodulator 458 and performs digital-and Lugovoe conversion and frequency conversion using transceiver 463. The mobile phone 400 transmits the resulting conversion of the signal to be transferred to the base station, not shown in the drawing, via the antenna 414. This transferable signal (image data), which was transmitted to the base station, receives on the other end of the communication channel through the communication network or similar structure.

Note that when image data transfer is not necessary, the mobile phone 400 may represent image data, generated using a CCD camera 416, the liquid crystal display 418 via block 455 display control, and not through the encoder 453 image.

In addition, in the case of, for example, when you need to take the data file of the moving image, with a link to a simple web page or similar object, in the data transfer mode, the mobile phone 400 receives the signal transmitted by the base station via the antenna 414 using transceiver 463, amplifies this signal and then performs frequency conversion and analog-to-digital conversion. The mobile phone 400 performs the compression spectrum of the received signal to restore the original multiplexed data via the modulator/demodulator 458. Then the mobile phone 400 shares these multiplexed data to the coded data image is supply and audio data using demultiplexer 457.

The mobile phone 400 decodes using decoder 456 image encoded image data using the decoding algorithm corresponding to a certain encoding algorithm, such as MPEG2 or MPEG4, for generating data of a reproduced moving image and presents the data on the liquid crystal display 418 via the unit 455 control LCD display. Accordingly, data such as moving images included in the moving image file having a link to a simple web page that can be presented on the liquid crystal display 418.

The mobile phone 400 uses the above-described decoding device 1 as the decoder 456 image, performing such a procedure. Therefore, as in the case of the decoding device 1, the decoder 456 image performs decoding using the predicted image generated by the prediction filter, and thus, the decoded high-resolution image can be obtained more effectively by using the image correlation in time.

At this point, the mobile phone 400 simultaneously converts the digital Audi put into an analog audio signal using the audio codec 459, and outputs the audio signal through gromkogovorya the spruce 417. Accordingly, it is possible, for example, to reproduce the audio data included in the moving image file having a link to a simple web page.

Note that as in the case of e-mail, mobile telephone 400 is also able to record (save) the received data associated with the link with a simple web page in a storage device 423 through the block 462 recording/playback.

In addition, the mobile phone 400 may also analyze two-dimensional code obtained CCD camera 416 in the read image, and accept the information contained in this two-dimensional code using the main unit 450 controls.

Moreover, the mobile phone 400 can communicate with an external device via infrared radiation using block 481 infrared connection.

Through the use of encoder 101 as of encoder 453 image mobile phone 400 can increase the efficiency of encoding data generated by encoding the image data generated by, for example, a CCD camera 416, without increasing the complexity of the processing. As a result, the mobile phone 400 may transfer to another device encoded data (image data) with high coding efficiency.

In addition, when using a decoding of the mouth of the STS 1 as the decoder 456 image mobile phone 400 may generate the predicted image with high accuracy without increasing the processing load. As a result, for example, the mobile phone 400 may receive the decoded image with high resolution on the basis of the moving image file associated by reference with a simple web page, and present the image on the display.

Note that although the above description was given for the mobile phone 400, using a CCD camera 416, instead of such a CCD camera 416 can be used imager (CMOS imager), based on CMOS structures (complementary patterns of metal-oxide-semiconductor). In this case also, as in the case of use of a CCD camera 416, the mobile phone 400 can read the image of the object and to generate data of this image object.

In addition, although the above description was given with reference to the mobile phone 400, discussed the decoding device 1 and the encoder 101 can be applied in any device having a function for reading image and a communication function, similar to the mobile phone 400, such as, for example, a personal digital assistant (PDA), smart phone, ultra-mobile personal computer (UMPC), a netbook computer or a portable personal computer (notebook).

Fig is a block diagram illustrating an example of the basic configuration is trojstva write to the hard disk, using decoders and encoder, in which is applied the present invention.

The device 500 recording hard disk drive (HDD)shown in Fig, is a device that saves audio and video broadcasting programmes included in the broadcast signal (television signal)transmitted from the satellite, a terrestrial antenna or similar device and received by the tuner, built-in hard disk and providing the stored data to the user at the right time in accordance with the commands of this user.

The device 500 entries on the hard disk can select, for example, audio data and video data from the broadcast signal, to decode the data properly and to keep the built-in hard disk. In addition, the device 500 entries on the hard disk may also receive audio data and video data from another device via, for example, the communication network, decode the data properly and to keep the built-in hard disk.

Further, the device 500 entries on the hard disk can decode audio data and video data recorded on the built-in device hard disk space, transfer, for example, the decoded data to the monitor 560, to render an image on the screen this is the monitor 560 and output the audio signal through a speaker of the monitor 560. In addition, the device 500 entries on the hard disk may, for example, to decode audio data and video data extracted from the broadcast signal received through a tuner or audio data and video data received from another device via the communication network, to transmit the decoded data to the monitor 560, to render an image on the screen of the monitor 560 and output the audio signal through a speaker of the monitor 560.

Of course, we can also perform other operations.

As shown in Fig, the device 500 records on a hard magnetic disk includes a receiver 521, the demodulator 522, the demultiplexer 523, the audio decoder 524, the video decoder block 525 and 526 controls the recording device. The device 500 records on a hard magnetic disk drive further includes a memory 527 data of the electronic program guide (EPG), memory 528 programs, working memory 529, Converter 530 of the display unit 531 control on-screen menu (OSD), block 532 control display unit 533 recording/playback, digital-to-analog Converter 534 and the communication unit 535.

In addition, the Converter 530 display has a video encoder 541. Block 533 recording/playback contains the encoder 551 and the decoder 552.

The receiver 521 receives an infrared signal from a remote controller (not shown), converts it into an electrical signal and transmits nl the ku 526 controls the recording device. Block 526 controls the recording device contains, for example, a microprocessor or similar device and performs various operations in accordance with the program stored in memory 528 programs. At this time, the block 526 controls the recording device uses working memory 529 as necessary.

Block 535 links connected to the communication network and communicates with another device through the communication network. For example, block 535 connection that is managed by block 526 controls the recording device may communicate with a tuner (not shown) and to transmit it to the tuner main control signal channel selection.

The demodulator 522 performs demodulation of a signal from the tuner, and transmits the result to the demultiplexer 523. The demultiplexer 523 divides data received from the demodulator 522, audio data, video data and electronic program guide (EPG) and transmits the received data to audio decoder audio 524, the video decoder block 525 and 526 controls the recording device, respectively.

Audio decoder audio 524 decodes the input audio data using, for example, the MPEG algorithm and passes the result to block 533 recording/playback. The video decoder 525 performs the decoding of the input video data using, for example, the MPEG algorithm and passes the result to the Converter 530 of the display. Block 526 control the recording device transmits the input data of the electronic program guide (EPG) in the memory 527 EPG data for recording these EPG data in memory.

Converter 530 display encodes using the video encoding device 541 video data supplied from the video decoder 525, or from block 526 to control the recording device, to convert them into video data, for example, in the NTSC standard, and transmits the result of the conversion in block 533 recording/playback. In addition, the Converter 530 converts the display screen size of video data from the video decoder 525, or from block 526 to control the recording device, the screen size corresponding to the size of the monitor 560, converts video to NTSC with video encoding device 541, transforms this data into an analog signal and transmits in block 532, the control display.

Block 532 display control imposes a signal on screen menu (OSD) from block 531 control this menu on the video signal coming from the Converter 530 display command from block 526, the control device records and transmits the result to the monitor 560 for presentation on the screen.

In addition, the monitor 560 receives audio data output from the audio decoder 524, and then converted into an analog signal by digital-to-analog Converter 534. The monitor 560 outputs the audio signal via the loudspeaker.

The device 533 recording/playback has a rigid magnetic disk as the recording media, on which opisyvayut video audio data and other similar information.

Block 533 recording/playback encodes using the coding device 551 audio data supplied from the audio decoder 524, using the MPEG algorithm, for example. In addition, the block 533 recording/playback encodes using the coding device 551 video data coming from the video encoding device 541 from the Converter 530 of the display, using the MPEG algorithm. Block 533 recording/playback unites through multiplexer encode the audio data and the result of encoding video data. Block 533 recording/playback performs channel coding of the received complex data to enhance them and writes the data on a hard magnetic disk by the recording head.

Block 533 recording/playback reproduces data recorded on a hard magnetic disk, by a playback head, amplifies the data, and divides them into audio data and video data with use of the demultiplexer. Next, block 533 recording/playback decodes audio data and video data using the decoder 552 in accordance with the MPEG algorithm. Block 533 recording/playback performs digital-to-analog converting the decoded audio data and transmits the result to the speaker of the monitor 560. In addition, BC is to 533 recording/playback performs digital-to-analog converting the decoded video data and transmits the result for presentation on the display of the monitor 560.

Block 526, the control device reads the most recent data of the electronic program guide (EPG) from memory 527 EPG data based on the user commands, presents an infrared signal from remote control and received through the receiver 521, and transmits the EPG data block 531 control on-screen menu (OSD). Block 531 control OSD menu generates image data corresponding to the input data, electronic program guide, and transmits the data block 532 display control. Block 532 display control transmits video data received at its input from block 531 management OSD, the monitor 560 for presentation on the display screen. Accordingly, the display of the monitor 560 is shown an electronic program guide (EPG).

In addition, the device 500 entries on the hard disk can receive various data such as video data, audio data or the data of the electronic program guide (EPG)received from another device via a communication network such as the Internet.

Block 535 connection command block 526 controls the recording device receives an encrypted data such as video data, audio data, and EPG data transmitted from another device through the communication network and sends the data in block 526 controls the recording device. Block 526, the control condition is the device entry passes, for example, the coded data corresponding to the adopted video data and the audio data, the device 533 recording/playback, where the preservation of these data on a hard magnetic disk. At this point, block 526 controls the recording device and the device 533 recording/playback can, if necessary, to perform such operation as transcoding.

In addition, the block 526 controls the recording device decodes the received coded data corresponding to the video data and the audio data, and transmits the received video Converter 530 of the display. Converter 530 display processes the video data received from block 526 to control the recording device, similarly to the processing of video data coming from the video decoder 525, passes the result to the monitor 560 through the block 532 control the display and initiates the representation of the image on the display screen of the monitor.

Moreover, along with the representation of the image on the display unit 526 controls the recording device may transmit the decoded audio data to the monitor 560 through a digital-to-analog Converter 534 and output the sound signal output from the loudspeaker.

Next, block 526 controls the recording device decodes the received coded data corresponding to the electronic program guide (EPG), and transmits the decoded Dan is haunted EPG in memory 527 EPG data.

The device 500 entries on the hard disk, as described above, uses the decoding device 1 as the video decoder 525, the decoder 552 and decoder in block 526 controls the recording device. Therefore, the video decoder 525, the decoder 552 and the decoder in block 526, the control device records perform decoding using the predicted image generated by the prediction filter, as in the case of the decoding device 1, and thus, the decoded high-resolution image can be obtained using the image correlation in time more efficiently.

Accordingly, the device 500 entries on the hard disk can generate a projected image with high accuracy without increasing the processing load. As a result, the device 500 entries on the hard disk can obtain a decoded image with high resolution on the basis of, for example, encoded video data received through the tuner encoded video data read from the hard magnetic disk unit 533 recording/playback, or encoded video data received through the communication network, and present the resulting image on the screen of the monitor 560.

In addition, the device 500 entries on the hard disk uses to tiraumea device 101 as of encoder 551. Therefore, as in the case of encoder 101, the encoder 551 performs encoding using the predicted image generated by the prediction filter comprising a large number of high-frequency components and having a small difference relative to the original image, and thus, it is possible to reduce the amount of code assigned to the residual error, and improve the coding efficiency.

Accordingly, such a device 500 entries on the hard disk can, for example, to increase the efficiency of encoding data for recording on the hard disk without increasing the complexity of the processing. As a result, the device 500 entries on the hard disk can more effectively utilize the available space on the hard magnetic disk for recording information.

Note that, although the above description was given with reference to the device 500 entries on the hard disk, which records video and audio data on a hard magnetic disk, of course, you can use the recording media of any type. For example, such a decoding device 1 and the encoder 101 can be applied to the recording device using a recording medium other than a hard magnetic disk, such as flash memory, optical fiber is ski disk or magnetic tape, as in the case of device 500 records on a hard magnetic disk described above.

Fig is a block diagram illustrating an example of the basic configuration of a video camera that uses the decoding device and the encoder, in which is applied the present invention.

The camera 600, shown in Fig, reads the image of the object and represents the image of the subject on the LCD display 616, and writes this image as image data on the media 633 entries.

Block 611 lens directs the light flux (bearing the image of a subject on the imaging unit 612 of the image signals of the CCD/CMOS type. CCD/CMOS driver 612 is an imager that uses a CCD or a CMOS structure, which converts the intensity of received light into an electrical signal and transmits this signal in block 613 signal processing of the video camera.

Block 613 signal processing video camera converts the electric signal coming from the CCD/CMOS imaging unit 612, color difference signals Y, Cr and Cb, and sends these signals to the processor 614 of the image signal. The processor 614 of the image signal performs some image processing in accordance with the image signal coming from block 613 processing of the video signal, and encodes the village is edstam of encoder 641 the image signal with the use, for example, the MPEG algorithm on commands from the controller 621. The processor 614 for processing the image signal transmits the coded data generated by coding the image signal, the decoder 615. In addition, the processor 614 of the image signal receives the data for presentation on the display, generated by block 620 on-screen menu (OSD), and transmits the data to the decoder 615.

When the procedure described above, block 613 signal processing camcorder uses a properly dynamic storage device 618 random access (dynamic RAM), United with him via the bus 617, and writes image data, encoded data obtained by encoding these image data, or similar data to save in the dram 618 as necessary.

The decoder 615 performs decoding encoded data received from the processor 6 14 image signal, and transmits the received image data (data of the decoded image) liquid crystal display 616. In addition, the decoder 615 transmits the data for presentation on the display coming from the processor 614 of the image signal, the liquid crystal display 616. This liquid crystal display 616 sums properly the image represented by the decoded image, pic is Pausini from the decoder 615, and the image data for presentation on the display and presents the resulting composite image on the display.

Block 620 OSD transmit data for presentation on the display, such as a menu screen, educated characters, letters or numbers, and icons in the processor 614 of the image signal via the bus 617 command controller 621.

The controller 621 performs various procedures on the basis of the signal representing the contents of the command submitted by the user using the operating unit 622, and controls the processor 614 of the image signal, the dram 618, the external interface 619, block 620 OSD drive 623 the recording media and the like via the bus 617. Programs, data, and similar information needed by the controller 621 to perform a variety of procedures recorded in the flash ROM 624.

For example, the controller 621 can encode the image data recorded in the dram 618, or to decode the coded data written in the dram 618, "on behalf of" processor 614 of the image signal or the decoder 615. At this point, the controller 621 can carry out the procedure of encoding/decoding using an algorithm similar to the algorithm of the encoding/decoding processor 614 of the image signal or the decoder 615, or may perform the encoding procedure/decterov the Oia using the algorithm, not compatible with the processor 614 of the image signal or the decoder 615.

In addition, when, for example, the operation unit 622 gave the command to start the printing image, the controller 621 reads the image data from the dram 618 and transmits the data to the printer 634 connected to the external interface 619, via the bus 617 to print this image.

Moreover, when, for example, the operation unit 622 gave the command to write the image, the controller 621 reads the coded data from the dynamic OSU and transmits the data to the media 633 entries installed in the actuator 623 media, via the bus 617 to save this data.

Media 633 account represents any arbitrary read and rewritable removable recording media such as magnetic disk, a magnetooptical disk, an optical disk or a semiconductor memory. Of course, as such a carrier 633 account, you can use removable recording media of any type, which may be a device on the magnetic tape, disk or memory card. Naturally, the media 633 entries can also be contactless electronic card or similar device.

In addition, the actuator 623 the recording media and the media 633 recording can be performed at the same time as reportative recording media, such as su is rainny memory hard disk drive or solid state SSD drive.

The external interface 619 contains, for example, a USB terminal I/o or similar terminal, and is connected to the printer 634, if you want to print the image. In addition, the external interface 619, if necessary, connect the drive 631 on which you install the appropriate removable media 632 entries, such as magnetic disk, optical disk or magneto-optical disk, and read from the media of the computer program is installed as required in memory flash ROM 624.

Further, the external interface 619 has a network interface connected to any communication network such as a local network (LAN) or the Internet. The controller 621 can read encoded data from the dram 618 and to transfer these data from the external interface 619 to another device connected through a communication network, in accordance with commands from, for example, operational block 622. In addition, the controller 621 can take through the external interface 619 coded data or image data transmitted from another device through the communication network and store them in the dram 618 or be sent to the processor 614 of the image signal.

The camera 600, as described above, uses the decoding device 1 as the decoder 615. Therefore, as in the case of the decoding device 1, the decoder 615 performs zakodirovana is using the projected image, generated by predictive filtering. Thus, it is possible to obtain a decoded image with high resolution due to more efficient use of image correlation in time.

As a result, the camera 600 can generate a projected image with high accuracy without increasing the processing load. As a result, the camera 600 can receive the decoded image with high resolution on the basis of, for example, image data generated by using the CCD/CMOS imaging unit 612, the coded video data read from the dynamic OSU or media 633 recording, or encoded video data received through the communication network, and can present this image on the liquid crystal display 616.

In addition, the camera 600 uses the encoder 101 as a coding device 641. Therefore, as in the case of encoder 101, specified the encoder 641 performs encoding using the predicted image generated by the prediction filter comprising a large number of high-frequency components and has a small difference relative to the original image, and thus, the amount of code assigned to the residual error signal is to be reduced, and the coding efficiency can be increased.

Accordingly, the camera 600 can improve the coding efficiency, for example, coded data for recording on the hard disk without increasing the complexity of the processing. As a result, the camera 600 can also more effectively use the "space" entries in the dram 618 or substrate 633 entries.

In addition, the method of decoding used in the decoding device 1 may be applied in the decoding process performed by the controller 621. Similarly, the encoding method used in the encoding device 101, can be applied in the encoding process performed by the controller 621.

In addition, the image data read by the video camera 600 may be data of moving or still images.

Of course, the decoding device 1 and the encoder 101 can also be used in equipment or system that is different from the apparatus described above.

The list of positional notation

1 - decoding device 21 is a diagram of the prediction/motion compensation, 41 - scheme for determining the prediction mode, 42 diagram of the unidirectional prediction, 43 diagram of the bidirectional prediction, 44-prediction scheme, 45 - circuit filter 51 - schema calculating the difference, 52 - f is ltr lower frequencies, 53 diagram of the adjustment of the gear ratio, 54 - high-pass filter, 55 - scheme-adjustment of transfer ratio, 56 - adder 57 - adder 231 - partitioning scheme block 232 - limiter, 233 - buffer

1. An imaging device, comprising: means for determining to determine in accordance with the number of taps of the filter used for the filtering process, the number of pixels across the width of the strip placed outside of the macroblock including the reference block representing a block of a decoded reference frame and in contact with the specified reference block; means for receiving from the reference frame of the specified reference block and the band corresponding to the number of pixels, a specific means of determining if the reference block representing a block of the reference frame corresponding to the block included in the image filtered is in contact with the border of the specified macroblock including the reference block; and filtering means for filtering the image of the reference block and the band obtained by means of obtaining, with a filtration media includes a first filtering means for filtering lower frequency in relation to the differential image between multiple images, the second filtering means for filtering the top frequent is t to the image, the resulting filter the lower frequencies, perform the first filtering means, and an adder for adding the image obtained by filtering lower frequency, perform the first filtering means, and the image obtained by filtering the upper frequencies that are performed by the second filtering means, to any of the multiple images for generation of the predicted image in units of macroblocks.

2. The imaging device according to claim 1 in which the means of obtaining made with the possibility of obtaining a reference block from the reference frame, if the reference block is in contact with the boundary of the macroblock including the reference block.

3. The imaging device according to claim 2, in which the means for determining is configured to determine the number of pixels equal to the maximum integer that is less than or equal to the value obtained by dividing by two the number of taps of the filter used for filtering, and a certain number of pixels is the number of pixels across the width specified band.

4. The imaging device according to claim 1, further comprising: specifying means for specifying a reference block based on the motion vector.

5. The imaging device according to claim 4, in which a filter is a filter with finite impulse is a characteristic.

6. The method of image processing, comprising: a detection phase, which is determined in accordance with the number of taps of the filter used for filtering, the number of pixels across the width of the strip placed outside of the macroblock including the reference block representing a block of a decoded reference frame and in contact with the specified reference block; a step of receiving, which is obtained from the reference frame specified reference block and the band corresponding to the number of pixels found at the stage of determination, if the reference block representing a block of the reference frame corresponding to the block included in the image filtered is in contact with the border of the specified macroblock including the reference block; and a phase filter, which filter the image of the reference block and the band obtained at the stage of getting on stage filter performs low pass filtering in respect of a differential image between multiple images, perform high-pass filtering to the image obtained by filtering lower frequency, perform the first filtering means, and add an image obtained by filtering the low frequencies, perform the first filtering means, and an image obtained as a result of filtering, high-pass, perform the second filtering means, to any of the multiple images for generation of the predicted image in units of macroblocks.



 

Same patents:

FIELD: information technology.

SUBSTANCE: method for block interframe motion compensation for video encoders includes breaking down an image into hexagonal blocks. The blocks are grouped in three layers. For each block, a displacement vector is searched based on a search pattern defined for a layer. The search region is preliminarily shifted by a predicted vector calculated from vectors of neighbouring blocks from previous layers for which a displacement vector has been found already. Standard deviation is calculates using a special weighting function (mask) which takes into account the hexagonal shape of the block.

EFFECT: improved visual quality of a reconstructed video sequence by reducing displacement vector prediction error due to improved correlation properties of the used blocks.

4 dwg

FIELD: information technology.

SUBSTANCE: method of encoding a residual block comprises steps of generating a prediction block of a current block; generating a residual block based on the difference between the prediction block and the residual block; generating a transformation residual block by transforming the residual block to a frequency domain; splitting the transformation residual block into frequency band units; and encoding effective coefficient flags indicating frequency band units, in which nonzero effective transformation coefficients exist.

EFFECT: efficient encoding and decoding of a residual block.

15 cl, 36 dwg

FIELD: information technology.

SUBSTANCE: method for local adjustment of brightness and contrast of a reference frame for encoding a multi-view video sequence including: obtaining pixel values of the current encoded block belonging to the encoded frame, and pixel values of a reference block belonging to a reference frame; obtaining restored pixel values neighbouring with respect to the current block of the encoded frame, and pixel values neighbouring with respect to the reference block of the reference frame; determining numerical relationships between pixel values of the reference block and pixel values neighbouring with respect to the reference block, and relationships between the restored pixel values, neighbouring with respect to the current encoded block, and pixel values neighbouring with respect to the reference block; numerical relationships found at the previous step are used to determine parameters for adjusting brightness and contrast for adjusting differences in brightness and contrast for the reference block compared to the current encoded block; and adjusting differences in brightness and contrast for the reference block using the found adjustment parameters.

EFFECT: high encoding efficiency.

13 cl, 10 dwg

FIELD: information technology.

SUBSTANCE: method is carried out by realising automatic computer formation of a prediction procedure which is appropriately applied to an input image. The technical result is achieved by making an image encoding device for encoding images using a predicted pixel value generated by a predetermined procedure for generating a predicted value which predicts the value of a target encoding pixel using a pre-decoded pixel. The procedure for generating a predicted value, having the best estimate cost, is selected from procedures for generating a predicted value as parents and descendants, where the overall information content for displaying a tree structure and volume of code estimated by the predicted pixel value, obtained through the tree structure, is used as an estimate cost. The final procedure for generating a predicted value is formed by repeating the relevant operation.

EFFECT: high efficiency of encoding and decoding, and further reduction of the relevant volume of code.

12 cl, 14 dwg

FIELD: information technology.

SUBSTANCE: disclosed is use of a parent population which is generated via random formation of a procedure for generating a predicted value, each indicated by a tree structure, and a set of procedures for generating a predicted value is selected as a parent from such a population. The procedure for generating a predicted value is generated as a descendant based on a certain method of development of the tree structure which develops selected procedures for generating a predicted value, where the existing function for generating a predicted value can be a tree end node. The procedure for generating a predicted value, having the best estimate cost, is selected from procedures for generating a predicted value as a parent and a descendant, and overall information content for representing the tree structure and volume of the code, estimated by the predicted pixel value, is used as a cost estimate, and the final procedure for generating a predicted value is formed by repeating the relevant operation.

EFFECT: high encoding efficiency.

28 cl, 14 dwg

FIELD: information technologies.

SUBSTANCE: method for motion vector coding includes the following stages: selection of the first mode as the mode of information coding about a predictor of the motion vector in the current unit, and in this mode information is coded, which indicates the motion vector predictor at least from one motion vector predictor, or selection of the second mode, in which information is coded, which indicates generation of a motion vector predictor on the basis of units or pixels included into a pre-coded area adjacent to the current unit; determination of the motion vector predictor of the current unit in accordance with the selected mode, and coding of information on the motion vector predictor of the current unit; and coding of the vector of difference between the motion vector of the current unit and predictor of the motion vector of the current unit.

EFFECT: increased efficiency of coding and decoding of a motion vector.

15 cl, 19 dwg

FIELD: information technologies.

SUBSTANCE: share of cast combinations of optimal forecasting modes, which shall be selected for spatially corresponding units of upper and lower layers is identified on the basis of the optimal forecasting mode, which was selected in process of traditional coding, and a table of compliance is developed, which describes interconnections between them. Combinations of selected optimal forecasting modes in the compliance table are narrowed on the basis of the value of the share of casts, in order to create information of compliance for forecasting modes, which describes combinations of narrowed optimal forecasting modes. In process of upper layer unit coding, the version of searching for the forecasting mode, searching for which shall be carried out in process of coding, is identified by referral to information of compliance for forecasting modes using as the key the optimal forecasting mode selected in process of coding of the spatially corresponding unit of the lower layer.

EFFECT: reduced versions of searching for a forecasting mode of an upper layer using correlations of optimal forecasting modes between layers.

7 cl, 14 dwg

FIELD: information technology.

SUBSTANCE: displacement vectors are searched for by searching for global displacement, breaking up the image into multiple layers of blocks, successive processing of the layers using various search schemes, using displacement vector prediction, as well as selecting displacement vectors based on efficiency of their further entropy coding.

EFFECT: quality improvement of efficiency of a video compressing system, especially at low bit losses, high output thereof.

2 cl, 8 dwg

FIELD: information technologies.

SUBSTANCE: video coding device is a video coding device for exposure of a video image to forecasting coding with compensation of motion, comprising a detection module, in order to detect accessible blocks for blocks having vectors of motion, from coded blocks adjacent to a block to be coded, and a number of available blocks, a selection module, in order to select one selective block from coded accessible blocks, a coder of selection information, to code information of selection, indicating the selective block, using a coding table, corresponding to the number of accessible blocks, and a coder of images, to expose the block to be coded to forecasting coding with compensation of motion using a vector of motion of the selective block.

EFFECT: reduction of additional information by information of selection of a motion vector with increased extents of freedom for calculation of a motion vector by selection of one of coded blocks.

10 cl, 14 dwg

FIELD: information technology.

SUBSTANCE: each re-encoded frame of a multiview video sequence, defined according to a predetermined encoding sequence, is presented as a set of non-overlapping units; at least one of already encoded frame is determined, which corresponds to said view and denoted as reference; synthesised frames are generated for the encoded and reference frames, wherein for each non-overlapping unit of pixels of the encoded frame, denoted as the encoded unit, a spatially superimposed unit inside the synthesised frame is determined, which corresponds to the encoded frame, denoted as a virtual unit, for which the spatial position of the unit of pixels in the synthesised frame which corresponds to the reference frame is determined, so that the reference virtual unit thus determined is the most accurate numerical approximation of the virtual unit; for the determined reference virtual unit, the spatially superimposed unit which belongs to the reference frame, denoted as the reference unit, is determined, and the error between the virtual unit and the reference virtual unit is calculated, as well as the error between the reference virtual unit and the reference unit; the least among them is selected and based thereon, at least one differential encoding mode is determined, which indicates which of the units found at the previous should be used to perform prediction during the next differential encoding of the encoded unit, and differential encoding of the encoded unit is carried out in accordance with the selected differential encoding mode.

EFFECT: providing differential encoding of a frame using a small volume of service information by taking into account known spatial connections between neighbouring views at each moment in time, as well as information available during both encoding and decoding.

5 cl, 15 dwg

FIELD: information technology.

SUBSTANCE: method involves selecting the high-frequency component of pixel values of an image component; subtracting, from the initial pixel values of the image component, corresponding values of the low-frequency component thereof; calculating mathematical expectation and mean-square deviation of the high-frequency component of all pixels of the image component; dividing the matrix of the high-frequency component into columns or rows and calculating values of mathematical expectation and mean-square deviation of each column or values of mathematical expectation and mean-square deviation of each row; correcting the values of the high-frequency component; an image with reduced noise is formed by summation of values of the low-frequency component and the corrected values of the high-frequency component.

EFFECT: reduced noise in an electronic image.

3 cl, 15 dwg

FIELD: information technology.

SUBSTANCE: deblocking filter 113 adjusts the value of disable_deblocking_filter-idc, slice_alpha_c0_offset_div2 or slice_beta_offset_div2 based on the Activity of an image calculated by an activity calculation unit 141, the total sum of orthogonal transformation coefficients of the image calculated by an orthogonal transformation unit 142, Complexity of the image calculated by the rate control unit 119, or the total sum of prediction errors of the image calculated by a prediction error addition unit 120.

EFFECT: improved image quality through correct deblocking.

8 cl, 7 dwg

FIELD: information technology.

SUBSTANCE: method involves parallel processing of the component of each decomposition level; brightness-contract transformation parameters are determined by forming a function for correcting brightness levels and a function for correcting contrast, forming a matrix of correction factors for third level decomposition contrast using the function for correcting contrast, reconstructing the family of matrices of scaled contrast correction factors for spatial matching on each level of correction factors with values of the detail component.

EFFECT: high quality of displaying digital images.

8 dwg

FIELD: information technologies.

SUBSTANCE: method includes performance of the following operations: digital copy of initial printed document is produced in colour space of RGB, brightness difference is detected, and direction of maximum gradient is determined, current count of image is classified for its affiliation to area of brightness difference or uniform area without sharp changes of brightness, Gauss smoothening of current count is made, if it is classified as belonging to uniform area without sharp changes of brightness, current count is smoothened in anisotropic manner, if it is classified as belonging to the area of brightness difference.

EFFECT: invention makes it possible to carry out fast single-stage descreening of screen-type pattern images with preservation of contour differences and increased accuracy.

5 cl, 9 dwg

FIELD: information technologies.

SUBSTANCE: target image that forms video image is divided into multiple division areas (DA); pass band (PB) width applied to DA is determined; array of filtration ratios (FR) is calculated to realise frequency characteristics corresponding to limitation of band, with application of PB width; image data is filtered with application of FR array; error information value is produced between obtained data and data of initial image, and distribution ratio (DR) is calculated to be used to determine optimal width of PB, on the basis of produced value; optimal width of PB corresponding to DR is defined for each DA, and array of optimal FR is calculated to realise frequency characteristics corresponding to limitation of band, using optimal width of PB; image data of division area is filtered using array of optimal FR; and produced data of each DA are synthesised.

EFFECT: generation of filtered image with specified value of image quality assessment.

29 cl, 27 dwg

FIELD: information technology.

SUBSTANCE: first band pass (BP) is determined based on initial image data; a matrix of filter coefficients (FC) is calculated to obtain frequency characteristics corresponding to limitation of frequency band (FB) using the first BP; data of the first filtered image are generated by filtering data of the initial image using the matrix of first FC; an estimate value of the objective image quality of data of the first filtered image is obtained and the distribution coefficient (DC) is calculated, which is used to determine the optimum BP based on the estimate value of objective image quality; the optimum BP corresponding to the calculated DC is determined using a table in which the corresponding relationship between DC and optimum BP is defined; a matrix of optimum FC is calculated to obtain frequency characteristics corresponding to limitation of FB using the optimum BP; and data of the optimally filtered image is generated by filtering data of the initial image using the matrix of optimum FC.

EFFECT: adaptive image filtering process for providing high-quality image.

3 cl, 11 dwg

FIELD: information technology.

SUBSTANCE: filtration of noise from digital images is based on defining a local structure of the image and on non-local averaging in accordance with the defined structure. The local structure of the image is determined by successively rolling up predefined templates with neighbouring pixels and by selecting a RPC template which gives the least error after rolling up. Noise is filtered from the digital image through weighted averaging of pixel values in the search window.

EFFECT: fast filtration of noise in digital images which provides high quality of noise suppression without causing distortions.

4 cl, 16 dwg

FIELD: information technology.

SUBSTANCE: size of coding unit relative the required printer resolution is evaluated. If the size of the unit is discernible by the human eye, the following steps are carried out: the approximate metric of discernibility of a distortion caused by the Gibbs effect is determined for each coding unit and stored in memory; the approximate metric of discernibility of block distortion is determined for each border of the unit; if the corresponding element of the approximate metric of discernibility of unit distortion exceeds a predefined threshold value, a filter which can suppress block distortions is applied to the given border of the unit; for each coding unit, if the corresponding element of the approximate metric of discernibility of distortion caused by the Gibbs effect exceeds a predefined threshold value, a filter which can suppress distortions is applied to the coding unit in order to suppress distortions caused by the Gibbs effect.

EFFECT: preservation of image sharpness.

7 cl, 6 dwg

FIELD: physics.

SUBSTANCE: invention proposes to use an imaging model with separation of effects associated with reflecting power of the surface R and effects associated with scene illumination characteristics L, for which: quality of a recorded image is evaluated and if there is need to correct the image, noise is filtered off; a smaller copy of the image is formed; borders for subsequent contrast enhancement at the correction step are defined on the smaller copy; the luminance channel of the initial image is selected and filtered; the image is corrected in accordance with an empirical equation of the LR imaging model:

where A is the lower boundary of contrast enhancement of the smaller copy of the image; Φ and ψ are lower and upper boundaries of contrast enhancement of the converted image; JB is the brightness component of the initial image after bilateral filtering; γ is a non-linear conversion parameter and JF is the brightness component of the enhanced image; and the resultant image is converted to RGB colour space.

EFFECT: higher image quality.

7 cl, 19 dwg, 2 tbl

FIELD: information technology.

SUBSTANCE: coding device has definition apparatus for determining image area data meant for processing in order to counter reconstruction implied by granular noise arising in image data coded based on said image data and apparatus for countering reconstruction, designed for processing in order to counter reconstruction for image area data, defined using definition apparatus when coding image data, where when the said image data are coded in data unit data modules, the said definition apparatus determines unit data which form the said image data as the said image area data, and apparatus for countering reconstruction forcibly sets the orthogonal transformation coefficient to zero, which becomes equal to zero when quantisation is carried out using the said unit data, among orthogonal transformation coefficients of unit data defined using the said definition apparatus.

EFFECT: improved quality of the decoded image.

14 cl, 18 dwg

FIELD: digital processing of images, possible use for global and local correction of brightness of digital photographs.

SUBSTANCE: system and method for correcting dark tones in digital photographs contain global contrasting module, module for conversion from RGB color system, module for determining dark tone amplification coefficient, bilateral filtration module, dark tone correction module, module for conversion to RGB color system, random-access memory block, displaying device. Global contrasting module is made with possible correction of global image contrast, module for conversion from RGB color system is made with possible conversion of image from RGB color system to three-component color system, one component of which is image brightness, and two others encode color, module for conversion to RGB color system is made with possible conversion from three-component color system, one of components of which is image brightness, and two others encode color, back to RGB color system, module for determining dark tone amplification coefficient is made with possible computation of global image brightness bar graph and can determine dark tone amplification coefficient based on analysis of signs, calculated from global image brightness bar graph, bilateral filtration module is made with possible execution of bilateral filtration of image brightness channel, dark tone correction module is made with possible correction of dark tones in image brightness channel.

EFFECT: absence of halo-effect.

2 cl, 17 dwg

Up!