Method of coding images

FIELD: physics; image processing.

SUBSTANCE: invention relates to a method of buffering multimedia information, as well as a method of decoding a coded stream of images in a decoder, in which the coded stream of images is received in form of transmission blocks which contain multimedia data. A system for processing multimedia data is proposed, which contains a coder for coding images and a buffer for buffering multimedia data. Multimedia data are included in the data transmission blocks. The data transmission blocks are ordered in the transmission sequence, which at least partially differs from the sequence of decoding multimedia data in transmission blocks. There is also definition block, which can set a parametre which indicates the maximum number of data transmission blocks which precede any data transmission block in a stream of packets in the transmission sequence and that data transmission block is tracked in the decoding sequence.

EFFECT: more efficient compression when buffering multimedia information.

32 cl, 7 dwg

 

The technical field to which the invention relates.

The present invention relates to a method of buffering multimedia information. The invention also relates to a method for decoding the encoded stream of images in the decoder, in which the encoded stream of images taken in the form of transfer units containing multimedia data. The invention additionally relates to a system, transmitting device, receiving device, a computer software product for buffering the encoded images, a computer program product for decoding a stream of coded images, the signal module, the processor for processing multimedia data in the transmitting device, the processor for processing multimedia data in the receiving device, the encoder and decoder.

The level of technology

Published video coding standards include ITU-T H.261, ITU-T H.263, ISO/IEC MPEG-1, ISO/IEC MPEG-2 and ISO/IEC MPEG-4 Part 2. These standards are specified in this document as the traditional video coding standards.

Transmission system video

The video transmission system can be divided into spoken and nerazgadannie system. Conversational systems include video conferencing and video telephony. Examples of such systems include ITU-T Recommendations H.320, H.323 and H.324, which specify system the him video conferencing/telephony working ISDN, IP and PSTN, respectively. Conversational systems are characterized by the desire to minimize end-to-end delay (from capture audio/video to view audio/video on the remote end)in order to improve user experience.

Nerazgadannie systems include playback of stored content, such as digital versatile disks (DVD) or video files stored in a storage device of large capacity playback device, a digital TV and streaming data. The following is a brief overview of the most important standards in these technological fields.

The main standard in digital consumer video equipment today is MPEG-2, which includes specifications compression video compression, audio, preservation and transfer. The preservation and transmission of the encoded video based on the concept of elementary flux. Elementary stream comprises encoded data from one source (for example, video) plus additional data required for synchronization, identification and determination of parameters of the original information. Elementary stream is packaged in packages of either fixed length or variable length to form a packetized elementary stream (PES). Each PES packet consists of a header, followed by streaming Dan is s, called payload. The PES packets from different elementary streams are combined to form either a program stream (PS) or the flow spread (TS). PS demanded in applications with low transmission error, for example in applications such as "save and play". TS-demand applications that allow transmission errors. However, TS implies that the network bandwidth is only guaranteed to be constant.

Joint Video Team (JVT) of organizations ITU-T and ISO/IEC has released a draft standard, which includes the text of the same standard that ITU-T Recommendation H.264 and ISO/IEC International Standard 14496-10 (MPEG-4 Part 10). The draft standard is called in this document JVT coding standard, and the codec according to the draft standard is called JVT codec.

The very specification of the codec conceptually different between the level of video encoding (VCL) and the level of abstraction from the network (NAL). VCL contains the functionality of the signal processing codec, such things as transformation, quantization, search/motion compensation and loop filter. It follows the basic concept of most modern video codecs based on the macroblock encoder, which uses the prediction of the external image motion compensation and encoding conversion differential signal. The output of the VCL are quanta: a bit string that is which contains the data of the macroblock of an integer number of macroblocks and header information quantum (containing the spatial address of the first macroblock in the quantum the original quantization parameter and the like). The macroblocks in the quanta are arranged in the scan sequence, unless you specify a different allocation of macroblocks using the so-called syntax flexible macroblock ordering. The prediction of external images is used only in the framework of quantum.

NAL encompasses conclusion quanta VCL in blocks of abstraction from the network (NALU), which is suitable for transmission over networks with packet switching or use-oriented packet multiplex environments. Appendix B JVT specifies the encapsulation process to pass the NALU under-oriented bit streams networks.

An additional mode of selection of the reference image on H.263 and tool coding NEWPRED of MPEG-4 Part 2 give you the choice of reference frame for motion compensation on each segment of the image, for example, each quantum in H.263. Moreover, the more advanced mode of the reference image on H.263 and the JVT coding standard give you the choice of reference frame for each macroblock separately.

The choice of the reference image makes it possible for many types of circuits temporal scalability. Figure 1 illustrates an example of the arrangement of temporal scalability, which in this document is called a recursive temporal scalability. A typical scheme can is t to be decoded with the help of three constant speeds frame. Figure 2 illustrates the scheme, called redundant encoding video in which the image sequence is divided into two or more independently encoded stream of alternating way. The arrows on these and all subsequent drawings show the direction of motion compensation, and the values under the frames correspond to the relative time capture and display of frames.

The sequence of transmission

In the traditional standards of coding sequence decoding images similar sequence display, except for B-pictures. Block in the traditional B-picture may be bidirectional predicted time of the two reference images, and the one reference image precedes in time, and another reference image you time after in the sequence display. Only the most recent reference image in the sequence decoding can follow B-picture in the sequence is displayed (exception: interlacing in H.263, where both field image subsequent in time the reference frame may be preceded by the B-picture in the sequence decoding). Traditional B-picture can be used as a reference picture for the temporal prediction and therefore traditional Izobrajenie can be accommodated without affecting the decoding of any other images.

The JVT coding standard includes the following new technical features compared to earlier standards:

Sequence of decoded images is separated from the sequence display. The image number shows the sequence of decoding, and the counter image sequence shows the sequence display.

- Reference image for a block in a B-picture can be either before or after the B-image in the sequence display. Therefore, B-picture means depressive image instead of the bidirectional image.

Images that are not used as reference pictures that are marked explicitly. The image of any type (internal, external, B, etc. may be either the reference image or unsampled image. (Thus, the B-picture can be used as a reference picture for the temporal prediction of other images).

- The image may contain quanta, which are encoded using different encoding type. In other words, the encoded image may consist of, for example, from internally encoded quantum and B-coded quantum.

Department of sequence display on sequence decoding may be advantageous since t is his view of the compression efficiency and robustness.

An example of the structure predictions, potentially increasing the compression efficiency presented in figure 3. The boxes show the image, capital letters in the boxes show the encoding types, the numbers in the rectangles numbers of images according to the JVT coding standard, and the arrows show the dependence of the predictions. Note that the image B17 is a reference image for image B18. The compression efficiency is potentially higher than with traditional encoding, as a reference image for image B18 closer in time compared to traditional coding with templates coded images PBBP or PBBBP. The compression efficiency is potentially increased in comparison with the traditional pattern of coded images PBP, as part of the reference images is bidirectional predictable.

Figure 4 shows an example of a method of delaying the internal images that can be used to increase the error tolerance. Traditionally, the internal image is encoded immediately after a shot or, for example, in response to the preceding renewal period the internal image. With the method of delaying the internal image of the internal image is not encoded as soon as the need arises C is to encode the internal image, instead, the next time the image is selected as an internal image. Each image between the encoded internal image and the traditional location of the internal image is predicted from the following time-image. As illustrated in figure 4, a method of delaying the internal image generates two independent chains predictions of external images, whereas the traditional coding algorithms generate a chain of external images. It is intuitively clear that the approach with two chains more robust against errors erase than the traditional approach with one chain. If one chain is experiencing packet loss, the other chain can still be correctly accepted. In traditional encoding packet loss always causes the distribution of the error on the remaining part of the chain predictions of external images.

Two types of information about the ordering and timing were traditionally associative associated with digital video: the sequence decoding and presentation. Below are the details related technologies.

Timestamp decoding (DTS) shows the time relative to the reference clock signal, in which, as expected, is decoded block encoded data. If TS is encoded and transmitted, it serves two purposes. First, if the sequence of decoded images is different from the sequence of their output, DTS shows the sequence of decoding in an explicit form. Secondly, DTS guarantees the functional behavior of the buffer pre-decoder, provided that the receiving speed close to the speed of the transmission at any point in time. In networks where end-to-end delay varies, the second option is the use of DTS does not play or plays a minor role. Instead, the received data are decoded as quickly as possible, provided that there is a reserve in the buffer post-decoder for uncompressed images.

Migrating DTS depends on communication systems and standard video encoding. In systems MPEG-2 DTS does not need to be transmitted as one element in the header of the PES packet. In the JVT coding standard DTS may not be transferred as part of the additional information extension (SEI) and it is used when the optional nominal reference decoder. In the basic format of media files ISO DTS dedicated to the native type of the block, the block decoding time per sample. In many systems, for example, based RTP systems streaming, DTS not tolerated, because it is believed that the sequence decode the approaches are similar in the transfer sequence, and the exact time of decoding does not play an important role.

Optional application U and W.6.12 H.263 specify the image number, which is incremented by 1 relative to the previous reference image in the sequence decoding. In the JVT coding standard element coding frame number is set similarly to the image number H.263. The JVT coding standard specifies a specific type of internal image called instant image update decoder (IDR). None of the subsequent image may not rely on images that follow earlier than the IDR image in the sequence decoding. IDR-image is often encoded as a reaction to replace the frames. In the JVT coding standard, the frame number is reset to 0 when the IDR image to increase the error tolerance in the event of loss IDR-image that presented on figa and 5b. However, it should be noted that SEI message information about the frame of the JVT coding standard can also be used for detection of the frame.

The image number H.263 can be used to restore the sequence of decoding the reference image. Similarly, the frame number JVT can be used to restore the sequence of decoded frames between IDR-picture (inclusive) and the next IDR-image is available (excluding) in the sequence decoding. However, because additional anchor pairs of fields (serial image, encoded as fields that have different parity) share the same frame number, the sequence of decoding cannot be restored from frame numbers.

Room H.263 image or frame number JVT unsampled image is set to be equal to the number of the image or frame of the previous reference image in the sequence decoding plus 1. If several unsampled image are performed sequentially in the sequence of decoding, they share the same room image or frame. The image number or frame unsampled image similar to the image number or frame of the next reference image in the sequence decoding. The sequence decoding serial unsampled image can be restored using the reference time (TR) element coding in H.263 or concept counter in the sequence of images (POC) of the JVT coding standard.

Timestamp representation (PTS) shows the time relative to the reference clock signal, when considered, the image should be displayed. Timestamp representation is also called the timestamp display temporarily the label output and composite timestamp.

Transfer PTS depends on communication systems and standard video encoding. In systems MPEG-2 PTS does not need to be transmitted as one element in the header of the PES packet. In the JVT coding standard PTS may not be transferred as part of as part of additional information extension (SEI). In the basic format of media files ISO DTS devoted to their own block type composite unit time per sample, while the timestamp representation is encoded relative to the corresponding timestamp decoding. In the RTP timestamp of the RTP header of the RTP packet corresponds to PTS.

Traditional video coding standards include the reference time (TR) element encoding that is similar to PTS in many aspects. In some of the traditional coding standards such as MPEG-2 TR is reset to zero at the beginning of the group of pictures (GOP). In the JVT coding standard has no notion of time in video encoding. The counter image sequence (POC) is set for each frame and the field and will be used as a TR, for example, in direct temporal prediction of B-rays. POC is reset to 0 V IDR image.

Buffering

Clients streaming typically have a buffer of a receiving device, which can store a relatively large quantity d is the R. In the beginning, when you have established a session streaming, the client does not start playing the stream immediately, but instead he typically buffers the incoming data for a few seconds. This buffering helps to maintain smooth playback, as in the case of coincidence of increased transmission delays or crashes the network bandwidth, the client can decode and play back the buffered data. Otherwise, without the initial buffering, the client must freeze the display, stop the decoding and wait for incoming data. Buffering is also required either for automatic or for selective retransmission at any Protocol level. If any part of the image is lost, the retransmission mechanism can be used to resend the lost data. If the retransmitted data is received up to the appointed time of the decoding or playback, the loss is recovered.

The encoded image can be ranked according to their importance in the subjective quality of the decoded sequence. For example, unsampled image, such as a traditional B-image, subjectively less important because their absence does not affect the decoding of any other images. Subjective ranking so the e can be made on the basis of the data section or group of quanta. Encoded quanta and data sections, which is subjectively more important, can be sent before the sequence decoding will show, while the encoded quantum and data sections, which are subjectively less important, can be sent after will show their natural sequence encoding. Therefore, any re-transmitted part of the most important quanta and data sections are more likely to be adopted before their appointed time decoding or playback compared to less important quanta and data sections.

Buffer pre-decoder

Buffer pre-decoder refers to buffering the encoded data before they are decoded. Initial buffering refers to the buffer pre-decoder at the beginning of the session streaming. Initial buffering has been traditionally done for two reasons described below.

In traditional multimedia systems with packet switching, for example, in IP-based video conference systems, different media types are normally transferred in separate packages. Moreover, the packages are typically transferred over the network "greatest effort", which cannot guarantee a constant transmission delay, and instead, the delay can vary from package to package. Therefore, the packets in the region have the same timestamp presentation (playback), can not be accepted at the same time, and the reception interval of the two packages can be not the same as the interval (in time units). Thus, in order to maintain the synchronized playback between different media types and to maintain the correct speed playback, multimedia terminal typically buffers the received data in a short period of time (for example, less than half a second)to smooth the variation of the delay. In this document, this type of buffer component is called a volatile buffer delay. Buffering may occur before and/or after the recording of multimedia data.

Buffering unstable delay is also applied to streaming. Given the fact that streaming is neriskovanny application, the desired volatile buffer delay may be significantly larger than in conversational applications. When the player streaming has established a connection to the server and requested multimedia stream to be downloaded, the server begins to transmit the desired flow. The player starts playing the stream immediately, but instead typically buffers the incoming data for a certain period, typically a few seconds. In this document buffering is also called the initial buffering. The initial Boo who arizala provides the ability to smooth out the variations of delays in the transfer method, similar provided by buffering unstable delay in conversational applications. In addition, it can enable the use of the communication channel, distribution and/or retransmissions at the application layer of the lost Protocol (PDU) data blocks. The player can decode and play back the buffered data, although the retransmitted PDU can be taken in time to be decoded and reproduced at the scheduled time.

Initial buffering clients streaming provides another advantage that cannot be achieved in a conversational systems: it enables the transmission speed multimedia data transmitted from the server to vary. In other words, the media packets can be transmitted faster or slower in time than the playback speed, as long as the buffer of the receiving device is not full or empty. Fluctuation speed data transfer can occur from two sources.

First, the compression efficiency achieved in some media types such as video, depends on the content of the source data. Therefore, if you want stable quality, the speed of the resulting compressed bit stream varies. Typically stable audiovisual quality is objective more attractive, than varying quality. Thus, the initial buffering allows to be made more attractive audiovisual quality in comparison with the system without the initial buffering, such as a videoconferencing system.

Secondly, it is well known that packet loss in a dedicated IP networks occur at the peaks. To avoid pulsing errors and high peak bit rate and packet well-designed streaming servers carefully schedule the transmission of packets. Packages can be sent exactly with the speed with which they are reproduced by the receiving party, but, instead, the server may attempt to achieve a steady interval between transmitted packets. The server can also adjust the transmission rate of packets in accordance with the dominant characteristics of the network, for example, reducing the rate of transmission of packets when the network becomes overloaded, and increasing, if network characteristics.

The transmission of the streaming media

System for streaming media comprises streaming server and a number of players who access the server via the network. The network typically focused on packages and provides little or no provides funds guaranteed quality service is. Players receive either pre-stored or live content from the server and render it in real time at the time when the content is loaded. The type of transmission data can be either point-to-point or multilateral. When point-to-point streaming server provides a separate connection for each player. When multilateral streaming server transmits one data packet number of players, and network elements duplicate thread only if necessary.

When the player has established a connection to the server and the requested media stream, the server begins to transmit the desired flow. The player starts playing the stream immediately, but instead typically buffers the incoming data for a few seconds. In this document buffering is also called the initial buffering. Initial buffering helps to maintain smooth playback, as in the case of coincidence of increased transmission delays or crashes the network bandwidth, the player can decode and play back the buffered data.

To avoid unlimited transmission delay, rarely prefer a reliable transport protocols in systems streaming. Instead, the system prefer nenadi the s transport protocols, for example UDP, which, on the one hand, inherit more stable latency, but, on the other hand, also experience damage or loss of data.

The RTP and RTCP can be used on top of UDP to control the exchange of data in real-time. RTP provides a means to determine packet loss transfer, re-assemble the correct sequence of packets on the receiving side and an associative link to the timestamp of the sample with each batch. RTCP sends information about how a large part of the packets have been correctly received, and therefore it can be used for flow control.

In the traditional standards of coding a video sequence decoding combined with sequence output. In other words, the sequence of decoding the I - and P-pictures is the same as the sequence of their output, and the sequence of decoding a B-picture immediately follows the sequence of decoding the last reference image of the B-pictures in the sequence output. Consequently, it is possible to restore the sequence of the decoding based on the known sequence of the output. The sequence output is typically transmitted in the elementary video stream of bits in the "reference time" (TR), as well as in the system multipl the SPE level, for example in the RTP header.

Some specification of RTP payloads enable transmission of the encoded data out of sequence decoding. The magnitude of the disorder is typically characterized by one value, which is similar in many relevant specifications. For example, in the draft RTP Payload Format for Transport of MPEG-4 Elementary Streams" option maxDisplacement specified as follows:

The maximum time shift block access (AU)corresponding to the encoded image is the maximum difference between the timestamp of the AU in the template and the timestamp of the earliest AU, which has not yet been received. In other words, when considering a sequence of alternating AU:

Max. shift = max{TS(i) - TS(j)}, for any i and any j>i

where i and j indicate the index of the AU in the pattern of alternation, and TS denotes the time stamp of the AU.

In the present invention, it was observed that in this way there are some problems and it provides a value that is too large for the buffer.

The invention

Example circuit, which sets the maximum offset is wrong when setting requirements for buffering (in terms of buffer space and duration of the initial buffering) below. The sequence of quantized into pieces of 15 AU, and AU in the last days the sequences of decoding and output in this part of the 15 AU is transmitted first, and all other AU are transmitted in sequence decoding and output. Thus, the transmitted sequence AU the following:

14 0 1 2 3 4 5 6 7 8 9 10 11 12 13 29 15 16 17 18 19...

The maximum shift for this sequence is equal to 14 for AU (14 + k * 15) for all nonnegative integers k).

However, the sequence space for the buffer and the initial buffering only for the first AU.

The draft document on the format of the payload for RTP H.264 (draft-ietf-avt-rtp-h264-01.txt) the parameter num-reorder-VCL-NAL-units specified as follows: this parameter can be used to signal the properties of stream NAL units or the characteristics of the transmitting device, or the implementation of a receiver. This parameter sets the maximum size of blocks VCL NAL, which precede any block VCL NAL stream NAL units in the sequence decoding NAL units and follow the block VCL NAL in the sequence numbers of the RTP sequence or in the sequence of link packet aggregation containing block VCL NAL. If the parameter is absent, should be applied to the value of num-reorder-VCL-NAL-units equal to 0. The value of num-reorder-VCL-NAL-units must be a whole number in the range 0-32767, inclusive.

According to the H.264 standard blocks VCL NAL set as the NAL units having nal_unit_type equal to from 1 to 5, inclusive. The standard specifies the following types of components the NAL 1-5:

1 - encoded quantum non-IDR-image;

2 - encoded quantum data partition A;

3 - encoded quantum data partition B;

4 - encoded quantum, section C data;

5 - encoded quantum IDR-image.

The parameter num-reorder-VCL-NAL-units causes a problem similar to the problem encountered in the above setting, the maximum shift. I.e. it is impossible to decide about requirements for buffering space and time initial buffering on the basis of this parameter.

The invention enables alarms for the size of the receive buffer to the decoder.

It independent GOP consists of images from the IDR-picture (inclusive) until the next IDR-image (excluding) in the sequence decoding.

In the present invention, the parameter indicating the maximum value of the required buffering, is specified more accurately than in systems of the prior art. In the following description the invention is described using based on the encoder-decoder system, however, it is obvious that the invention may also be implemented in systems in which the video signals are stored. The stored signals can be unencoded signals stored prior to encoding, the encoded signals are stored after encoding, or the decoded signals stored after Cody the Finance and decoding. For example, the encoder generates bit streams in the transfer sequence. The file system receives the bit streams of audio and/or video, which are, for example, in the sequence decoding and saved as a file. The file can be saved in the database from which the streaming server can read blocks of NAL and make them into RTP packets.

Moreover, in the following description the invention is described using based on the encoder-decoder system, however, it is obvious that the invention may also be implemented in systems in which the encoder outputs and transmits the coded data to another component, such as a streaming server in the first sequence, in which another component writes the encoded data from the first sequence in another sequence, specifies the required size of the buffer to a different sequence, and forwards the encoded data is reorganized in the form of a decoder.

According to the first aspect of the present invention, a method of buffering the media data in the buffer, when the multimedia data included in the blocks of data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in units of predecoding, wherein the set parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

According to the second aspect of the present invention, a method for decoding a stream of coded images in the decoder, in which the stream of coded images is in the form of blocks of data containing multimedia data, and the blocks in the form of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, perform buffering of blocks of data, characterized in that the requirements for buffering indicate the decoding process in the form of a parameter that indicates the maximum number of blocks of data that precede any transmission unit of the data stream packets in the sequence of transmission and follow this block of data in the sequence decoding.

According to a third aspect of the present invention proposed a system containing an encoder for encoding images, and a buffer for buffering the multimedia data, the multimedia, the applications data included in the blocks of data, moreover, the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, wherein the block contains the definitions made with the ability to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

According to a fourth aspect of the present invention proposed a transmitting device for transmitting multimedia data included in the blocks of data, the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, wherein the block contains the definitions made with the ability to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

According to the fifth aspect of the present invention proposed pickup is a great device for receiving a stream of coded images in the form of blocks of data, containing multimedia data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

According to the sixth aspect of the present invention provides a computer software product containing Mashinostroenie stages for buffering the encoded images in the buffer, when the multimedia data included in the blocks of data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it further comprises Mashinostroenie steps to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence is follow this block of data in the sequence decoding.

According to the seventh aspect of the present invention offer a computer software product containing Mashinostroenie stages for decoding the stream of encoded image, the encoded stream of images accepted in the form of transfer units containing multimedia data, and Mashinostroenie stages for buffering blocks transmission, characterized in that it contains Mashinostroenie steps to set the buffering requirements for the decoding process by using a parameter that indicates the maximum number of transfer units, containing multimedia data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the transmission unit containing multimedia data, in the sequence decoding.

According to the eighth aspect of the present invention proposed a signal-carrying media data included in the blocks of data, the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it includes a parameter that indicates the maximum number of blocks of data preceding the block of the transmission data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

According to the ninth aspect of the present invention proposed a module for receiving a stream of coded images in the form of blocks of data containing multimedia data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

According to the tenth aspect of the present invention proposed a processor for processing multimedia data in the transmitting device, characterized in that it contains the definition block, configured to set the parameter that indicates the maximum number of transfer units, containing multimedia data that precede any transmission unit containing multimedia data packet stream in the sequence of transmission units transmission and follow the transmission unit containing multimedia data, in the sequence decterov the Oia.

According to the eleventh aspect of the present invention proposed a processor for processing multimedia data in a receiving device for receiving a stream of coded images in the form of transfer units containing quantized data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of transfer units, containing multimedia data that precede any transmission unit containing multimedia data packet stream in the transfer sequence and follow the transmission unit containing multimedia data, in the sequence decoding.

According to the twelfth aspect of the present invention proposed an encoder for encoding the multimedia data included in the blocks of data, the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, wherein the block contains the definitions made with the ability to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the beginning of a data sequence decoding.

According to the thirteenth aspect of the present invention proposed a decoder for decoding the encoded stream of images included in the blocks of data containing multimedia data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of transfer units that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

In a typical embodiment of the present invention a transmission unit that contains media data unit is a VCL NAL.

The present invention increases the efficiency of the buffering systems of coding. By the present invention can be used right in fact the desired amount of buffering. So there is no need to allocate more memory than is necessary to buffer the coding device coding and buffer pre-decoding in the decoding device. You can also avoid overflow in the morning is a pre-encoding.

Description of the drawings

Figure 1 illustrates an example diagram of a recursive temporal scalability.

Figure 2 illustrates the scheme, called redundant encoding video in which the image sequence is divided into two or more independently encoded stream alternating method.

Figure 3 is an example of the structure predictions, potentially increasing the efficiency of compression.

Figure 4 shows an example of a method of delaying the internal images that can be used to improve robustness.

Figure 5 illustrates the preferential option for the implementation of the system according to the present invention.

6 illustrates a preferential option for the implementation of the encoder according to the present invention.

Fig.7 illustrates the preferential option for the implementation of the decoder according to the present invention.

Detailed description of the invention

Hereinafter the invention is described in more detail with reference to the system of figure 5, the encoder 1, 6 and decoder 2, 7. Images that need to be encoded can be, for example, images, video and video source 3, for example, camera, VCR, etc. Images (frames) of the video stream can be divided into smaller parts, such as quanta. Quanta can be further divided into blocks. In the encoder 1 videophoto is encoded, to reduce the amount of information that must be transmitted by channel 4 transfer or media storage (not shown). Image of the video stream are entered in the encoder 1. The encoder has a buffer 1.1 coding (6) for temporarily storing some of the images that need to be encoded. Encoder 1 also includes a memory 1.3 and CPU 1.2, which can be applied to the task of encoding according to the invention. Memory 1.3 and CPU 1.2 can be shared with the transmitting device 6 or the transmitting device 6 may have a different processor and/or memory (not shown) for other functions transmitting device 6. Encoder 1 performs motion estimation and/or some other task to compress the video stream. When the motion estimation searches for similarities between the image to be encoded (the current image), and previous and/or subsequent image. If similarities are found, comparing the image or its part can be used as a reference image for the image to be encoded. In JVT display sequence and the sequence of decoded images are not necessarily the same, while the reference image must be stored in the buffer (for example, in the buffer 1.1 encoding) as long as it is used as pornoho image. Encoder 1 also inserts information about the sequence of display images in the stream.

From the encoding process, the encoded image, if necessary, transferred to the buffer 5.2 encoded images. The encoded image is transmitted from the encoder, 1 decoder 2 through channel 4 assists. In the decoder 2, the encoded image is decoded to generate uncompressed image, corresponding to the maximum extent encoded images.

Decoder 1 also includes a 2.3 memory and processor 2.2, which can be applied to the task of decoding according to the invention. The 2.3 memory and processor 2.2 can be shared with the receiving device 8 or the receiving device 8 may have a different processor and/or memory (not shown) for other functions of the transmitting to the receiving device 8.

Encoding

Let us now consider the process of encoding/decoding more. Images from video source 3 are entered in the encoder 1 and preferably stored in the buffer 1.1 encoding. The encoding process is not necessarily begins immediately after the first image entered in the encoder, and after a certain number of images available in the buffer 1.1 encoding. Then, the encoder 1 is trying to find suitable candidates of images to be used in the reference frames. After that, the encoder 1 performs coding to generate coded image. Encoded images can be, for example, predicted pictures (P), Dvoretskaya image (B) and/or internally encoded image (I). Internally encoded image can be decoded without the use of any other images, but another type of image requires at least one reference image before it can be decoded. Images of any of the above types of images can be used as a reference image.

Encoder mainly attaches two temporary tags to images: time stamp decoding (DTS) and the timestamp output (OTS). The decoder can use the timestamps to determine the correct time decoding and output (display) images. However, these timestamps are not necessarily transmitted to the decoder, or it does not use them.

The NAL units can be delivered in a different packet type. In this preferred embodiment, the different packet formats include simple packets and packet aggregation. The packet aggregation can be further divided into single-use packet aggregation and reusable packages aggregation.

The format of the payload Pak is tov RTP is defined as the number of different structures payload depending on the needs. However, what structure contains the received RTP packet, it is obvious from the first byte of the payload. This byte is always structured as a header of the NAL unit. The field type of the NAL unit indicates which structure is used. Possible structure: package one NAL unit, the packet aggregation and packet fragmentation. Package one NAL unit contains only one NAL unit in the payload. Field type header NAL will be equal to the source type of the NAL unit, i.e. in the range from 1 to 23 inclusive. The type of packet aggregation is used to aggregate multiple NAL units into a single RTP payload. This package exists in four versions, disposable bag aggregation type A (STAP-A), disposable bag aggregation type B (STAP-B), multi-service aggregation (MTAP) offset in 16-bit (MTAP16) and multi-service aggregation (MTAP) with an offset of 24 bits (MTAP24). Rooms type of NAL unit assigned to the STAP-A, STAP-B, MTAP16 and MTAP24 is 24, 25, 26 and 27, respectively. Block fragmentation is used to fragment a single NAL unit across multiple RTP packets. It exists in two versions, specified using the type of the NAL unit 28, and 29.

There are three cases modes of packaging specified for RTP data:

- mode a single NAL unit,

- non-interleaved mode,

- alternating mode.

The single NAL unit is designed for conversational systems, which according to the frame of ITU-T Recommendation H.241. Non-interleaved mode is designed for conversational systems that may not conform to ITU-T Recommendation H.241. In non-interleaved mode, the NAL units are transmitted in sequence decoding NAL units. Alternating mode is suitable for systems that do not require a very small end-to-end delay. Alternating mode allows transmission of NAL units out of sequence decoding NAL units.

The mode used packaging can be presignalisation by the values of the additional MIME parameter regime packaging or external means. Used mode of batching controls which types of NAL units permitted in the RTP payload.

In alternating mode, the stacking sequence of the transmission of NAL units is allowed to differ from the sequence decoding NAL units. The sequence number decode (DON) is a field in the structure of the payload or derived variable that indicates the sequence decoding NAL units.

The Union of the sequence of transmission and decoding is controlled by additional MIME parameter depth rotation in the following way. When the value of the additional MIME-depth setting rotation to 0, and the transmission of NAL units out of sequence decoding is prohibited by an external tool, the latter is the sequence of transmission of NAL units corresponds to the sequence decoding NAL units. When the value of the additional MIME parameter depth rotation is greater than 0 or transmission of NAL units out of their sequence decoding resolved by external means,

sequence of NAL units in multi-service aggregation 16 (MTAP16) and reusable service aggregation 24 (MTAP24) does not necessarily have to be a sequence of decoding NAL units, and

sequence of NAL units, composed by concluding disposable packet aggregation B (STAP-B), MTAP and blocks aggregation (FU) in two successive packet does not necessarily have to be a sequence of decoding NAL units.

Patterns of RTP payloads for a single package units NAL, STAP-A and FU-A does not include DON. Patterns STAP-B and FU-B include DON, and the structure of MTAP allows the output of the DON.

If the transmitting device wants to include one unit of NAL in the package and send the package outside of their sequence decoding, can be used type of package STAP-B.

Mode of packaging one unit of NAL sequence of transmission of NAL units similar to their sequence decoding NAL units. In non-interleaved mode batching sequence of transmission of NAL units in packages of a single NAL unit and STAP-A and FU-A is equivalent to the sequence decoding NAL units. The NAL units in the STAP are displayed in sequence decoding : CTCSS / DCS the project NAL units.

Given the fact that H.264 allows sequence decoding different from the sequence display, the RTP timestamp values may not be not monotonically decreasing as a function of the numbers of the RTP sequence.

The value of DON for the first NAL unit in the transfer sequence can be set to any value. Value DON is in the range 0-65535, inclusive. After reaching the maximum value is DON minimized to 0.

The video sequence according to this specification may be any type of flow NALU, which cannot be decoded independently of the other parts of the NALU stream.

The following is an example of a reliable planning packages.

On the drawings the following example, time moves from left to right I mean IDR-image, R is the reference image, N means unsampled image, and the number indicates the relative output time proportional to the previous IDR image in the sequence decoding. Values below image sequences show timestamps scaled system clock generator and they are initialized randomly in this example. Each I-, R - and N-image is tied to the same time scale as compared with the previous stage of processing, if available, provided that codiovan is, transmission and decoding does not take time.

A subset of images in several sequences shown below in the sequence o

The sequence encoding (and decoding) of these images from left to right as follows:

The sequence number decode (DON) for the image equal to the value of DON for the previous image in the sequence decoding plus one.

For simplicity, we assume that:

- frequency frame transmission sequence is constant,

- each image consists of only one quantum,

each quantum is enclosed in the package of one NAL unit,

images are transmitted in sequence decoding, and

- images are sent at regular intervals (which is equal to 1/frequency transfer frames).

Thus, images are taken in sequence decoding:

The value of the parameter num-reorder-VCL-NAL-units is set to 0, because it does not require buffering to restore the correct sequence decoding of the transmit (or receive sequence).

The decoder need to buffer for an interval of one image is initially in its buffer the decoded images to crobiological image from a sequence of decoding, to output the sequence, as shown below:

The initial value of the required buffering in the buffer of decoded images can be signaled in the SEI message about the buffering period or in the value of num_reorder_frames syntax element information for H.264 video. Num_reorder_frames shows the maximum number of frames, complementary field pairs or unpaired fields that precede any frame, additional a couple of fields or unpaired field in the sequence decode and follow it in the sequence output.

For simplicity, it is assumed that num_reorder_frames is used to show the initial buffer in the buffer of decoded images. In this example, num_reorder_frames is equal to 1.

You can see that if the IDR-picture I00 lost during transmission and a request for retransmission is issued when the value of the system clock generator 62 is provided interval of time into a single image (as long as the timestamp of the system clock reaches 63 to receive the retransmitted IDR-picture I00.

Next, assume that IDR-images are sent to two frame interval earlier than their position decoding, for example, images are transmitted in the following order:

/p>

Let the variable id1 is set according to the prior art (described in draft-ietf-avt-rtp-h264-01.txt), i.e. it specifies the maximum size of blocks VCL NAL, which precede any block VCL NAL stream NAL units in the sequence decoding NAL units and follow the block VCL NAL in the sequence numbers of the RTP sequence or in the sequence of link packet aggregation containing block VCL NAL. Let the variable id2 will be set according to the present invention, i.e. it specifies the maximum size of blocks VCL NAL, which precede any block VCL NAL stream NAL units in the transfer sequence and follow the block VCL NAL sequence decoding.

In the example, the value of id1 is equal to 2, and id2 is equal to 1. As already shown in section 2, the value of id1 is not proportional to time or space decoding required for initial decoding, to rearrange the packets from the receive sequence in the sequence decoding. In this example, the initial buffering equal to the interval of one image is required to restore the sequence, as illustrated below (the figure represents the output of the buffering process of the receiving device). This example also demonstrates that the time value of the initial buffering and buffering space mo is et to be obtained according to the invention.

In addition, the delay in initial buffering in the interval of one image is needed to organize images from the sequence of decoded into a sequence of output, as shown below:

You can see that the maximum delay that IDR-image can be transferred with the transfer, including the possible application, transmission or re-transmission at the data link level of num_reorder_frames + id2. Thus, the error tolerance IDR images increased in systems that support retransmission.

The receiving device can arrange the images in the sequence decoding based on the value of DON, associative associated with each image.

Transfer

Transmitting and/or storing the encoded images (and optional virtual decoding) can be started immediately after finished the first encoded image. This image does not need to be first in the sequence of output of the decoder, since the sequence decoding and the sequence of output may not match.

When the first image of the video stream is encoded, the transmission can be started. The encoded image is not necessarily stored in the buffer 1.2 encoded image is s. The transmission can also be started at a later stage, for example, once encoded, a certain part of the video stream.

Decoder 2 also need to output the encoded image in the correct sequence, for example by using the ordering counter image sequence.

Diacetylene

The process of diacetylene depends on the implementation. Therefore, further description is not limiting example of a suitable implementation. Can also be used by other schemas. Probably possible optimization regarding the described algorithms.

The General concept regarding these rules deacetylase to reorganize the NAL units of the transfer sequence in the sequence of delivery of NAL units.

Decoding

The following describes the operation of the receiving device 8. The receiving device 8 collects all packets belonging to the image, leading them to a rational sequence. The accuracy of the sequence depends on the used profile. Received packets stored in the buffer 9.1 (receive buffer pre-coding). The receiving device 8 discards all useless and passes the rest to the decoder 2. The packet aggregation is handled by unloading their payload in a separate RTP packets carrying NALU. These NALU is obrabatyvayutsya thus, as if they were taken in separate RTP packets, in the order in which they were placed in service aggregation.

Further let N be the value of the optional parameter num-reorder-VCL-NAL-units (parameter depth rotation), which sets the maximum size of blocks VCL NAL, which precede any block VCL NAL in the packet stream NAL sequence of transmission of NAL units and follow the block VCL NAL sequence decoding. If the parameter is absent, can be applied to a value of 0.

When installed in the transmission of video streams, the receiving device 8 allocates memory buffer 9.1 reception to save at least N pieces of blocks VCL NAL. After that, the receiving device starts to receive the video stream and stores the received blocks VCL NAL in the receive buffer. Initial buffering continues:

up until at least N pieces of blocks VCL NAL stored in the buffer 9.1 or reception

- if the MIME parameter max-don-diff is present, as long as the value function don_diff(m,n) is greater than the value max-don-diff, where n corresponds to the NAL unit having the greatest value of AbsDON received NAL units, and m corresponds to the NAL unit having the smallest value of AbsDON received NAL units, or

up until the initial buffering has not lasted for a period equal to or greater than the value of the long is a high MIME-init-buf-time.

Function don_diff(m,n) is defined as follows:

If DON(m)==DON(n), don_diff(m,n)=0

If (DON(m)<DON(n) and DON(n)-DON(m)<32768),

don_diff(m,n)=DON(n)-DON(m)

If (DON(m)>DON(n) and DON(m)-DON(n)>=32768),

don_diff(m,n)=65536-DON(m)+DON(n)

If (DON(m)<DON(n) and DON(n)-DON(m)>=32768),

don_diff(m,n)=-(DON(m +65536-DON(n))

If (DON(m)>DON(n) and DON(m)-DON(n)<32768),

don_diff(m,n)=-(DON(m)-DON(n)),

where DON(i) the sequence number of the decoding of the NAL unit having index i in the transfer sequence.

A positive value don_diff(m,n) indicates that the NAL unit having the index of the transfer sequence of n should be in the sequence decoding for the NAL unit having the index of the transfer sequence m.

AbsDON means the sequence number of the decoding of the NAL unit, which is not minimized to 0 after 65535. In other words, AbsDON is calculated as follows:

let m and n consecutive NAL units in the transfer sequence. For the first NAL unit in the transfer sequence (whose index is equal to 0), AbsDON(0) = DON(0). For the remaining NAL units AbsDON is calculated as follows:

If DON(m)==DON(n), AbsDON(n)=AbsDON(m)

If (DON(m)<DON(n) and DON(n)-DON(m)<32768),

AbsDON(n)=AbsDON(m)+DON(n)-DON(m)

If (DON(m)>DON(n) and DON(m)-DON(n)>=32768),

AbsDON(n)=AbsDON(m)+65536-DON(m)+DON(n)

If (DON(m)<DON(n) and DON(n)-DON(m)>=32768),

AbsDON(n)=AbsDON(m)-DON(m)+65536-DON(n))

If (DON(m)>DON(n) and DON(m)-DON(n)<32768),

AbsDON(n)=AbsDON(m)-DON(m)-DON(n)),

where DON(i) the sequence number of decoding the Loka NAL, having index i in the transfer sequence.

When the buffer 9.1 reception contains at least N blocks VCL NAL, NAL units are removed from the buffer 9.1 receiving one by one and passed to the decoder 2. The NAL units are not necessarily removed from the buffer 9.1 reception in the same sequence in which they were saved, but according to DON NAL units, as described below. The delivery of packets to the decoder 2 continues until the buffer contains fewer than N blocks VCL NAL, i.e. N-1 blocks VCL NAL.

The NAL units that must be removed from the buffer of the receiving device are defined as follows:

If the receive buffer contains at least N blocks VCL NAL, NAL units are removed from the buffer of the receiving device and transmitted to the decoder in the sequence specified below, up until the buffer contains N-1 blocks VCL NAL.

- If max-don-diff is present, all NAL units m for which don_diff(m,n) is greater than max-don-diff, removed from the buffer of the receiving device and transmitted to the decoder in the sequence specified below. While n corresponds to the NAL unit having the greatest value of AbsDON received NAL units.

Variable ts is set to the value of the system timer, which was initialized to 0 when the first packet of stream NAL units was adopted. If the buffer of the receiving device contains a NAL unit whose reception time tr satisfies the condition that ts - tr > init-buf-time, block the NAL transmitted to the decoder (and removed from the buffer of the receiving device) in order specified below, up until the buffer of the receiving device does not contain the NAL block whose reception time tr satisfies a specified condition.

The sequence in which the NAL units are passed to the decoder is specified as follows.

Let PDON is a variable that is initialized to 0 at the beginning of the RTP session. For each NAL unit, associative associated with the value of the DON, the duration is calculated as follows. If the value of the DON of the NAL unit is greater than the value of PDON, duration equal DON DON - PDON. Otherwise, the duration is equal to 65535 DON - PDON + DON + 1.

The NAL units are delivered to the decoder in an increasing sequence length DON. If multiple units have the same NAL value for the duration of the DON, they can be transferred to the decoder in any sequence. When the desired number of NAL units was transferred to the decoder, the value of PDON is set equal to the value of DON for the last NAL unit, transmitted to the decoder.

DPB 2.1 contains the digits in the memory for storing a number of images. These discharges in this specification, also referred to as a frame memory. The decoder 2 decodes the received image in the correct sequence.

The present invention can be applied to many kinds of systems and devices. The transmitting device 6, which includes an encoder 1, mainly includes peredauschiesya 7, to transmit the encoded image in the channel 4 assists. The receiving device 8 includes a receiving unit 9, to receive the encoded image, the decoder 2 and the display 1, which can be displayed encoded image. The transmission channel may be, for example, channel ground connection and/or wireless channel. The transmitting device and the receiving device also includes one or more processors, 1.2, 2.2, which can perform the necessary steps to manage the process of encoding/decoding of the video stream according to the invention. Therefore, the method according to the present invention can mainly be implemented in the form of mashinostryenia stages processors. Buffering of images can be implemented in memory 1.3, 2.3 devices. Code 1.4 encoder can be saved in memory 1.3. Therefore, the code 2.4 decoder can be saved in memory 2.3.

1. The way the buffering multimedia data in the buffer, when the multimedia data included in the blocks of data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, wherein the set parameter that indicates the maximum h is word blocks of data, that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

2. The method according to claim 1, characterized in that the said multimedia data includes at least one of the following video data, audio data.

3. The method according to claim 1 or 2, characterized in that the said multimedia data contains quantum encoded image.

4. The method according to claim 1, characterized in that the said transmission unit containing media data unit is a level of abstraction from the network containing the data level video encoding.

5. Method of decoding a stream of coded images in the decoder, namely, that the stream of coded images is in the form of blocks of data containing multimedia data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, perform buffering of blocks of data, characterized in that the requirements for buffering indicate the decoding process in the form of a parameter that indicates the maximum number of blocks of data that precede any block transfer the ACI data in the packet stream in the transfer sequence and follow this the block of data in the sequence decoding.

6. The method according to claim 5, characterized in that the analyze mentioned the option and reserve bits of memory for buffering in accordance with the above parameter.

7. The method according to claim 6, characterized in that the decode image from a received stream of coded images and bufferinput encoded image using the reserved bits of memory.

8. System for processing multimedia data containing an encoder for encoding images, and a buffer for buffering the multimedia data, the multimedia data included in the blocks of data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, wherein the block contains the definitions made with the ability to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

9. The system of claim 8, characterized in that it contains a decoder for decoding encoded images and memory for buffering the decoded image, and the above-mentioned parameter is used for ODA is dividing the required number of bits in the memory, that must be reserved in memory for buffering the decoded images.

10. The system of claim 8, characterized in that the said multimedia data contains quantum encoded image.

11. The system of claim 8, characterized in that the said unit data unit is a level of abstraction from the network containing the data level video encoding.

12. A transmitting device for transmitting multimedia data included in the blocks of data, the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, wherein the block contains the definitions made with the ability to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

13. The transmitting device according to item 12, characterized in that the said multimedia data contains quantum encoded image.

14. The transmitting device according to item 12, characterized in that the said transmission unit, containing multimedia data is block level abstra the financing from the network, contains the data level video encoding.

15. A receiving unit for receiving the encoded stream of images in the form of blocks of data containing multimedia data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of transfer units that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

16. The receiving device according to item 15, characterized in that it contains the memory, and the aforementioned tool that uses the parameter block contains the definitions for the analysis of the mentioned parameter and to reserve bits of memory for buffering in said memory according to said parameter.

17. The receiving device according to item 16, characterized in that it contains a decoder for decoding pictures from a received stream of coded images and the means for using the reserved bits of memory for buffering the encoded image.

18. Computer-cheat is emy media, contains commands that when executed by a computer instruct the computer to perform steps for buffering the encoded images in the buffer, when the multimedia data included in the blocks of data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it further contains commands to perform steps to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

19. Computer-readable storage media that contains commands that when executed by a computer instruct the computer to perform the steps for decoding a stream of coded images, and the stream of coded images accepted in the form of transfer units containing multimedia data, and stages for buffering blocks transmission, characterized in that it further contains commands to perform steps to set the buffering requirements for the decoding process by using the parameter showing the Maxim the aspects of the number of transfer units, containing multimedia data that precede any transmission unit data containing multimedia data packet stream in the sequence of transmission units transmission and follow the transmission unit containing multimedia data, in the sequence decoding.

20. A module for receiving a stream of coded images in the form of blocks of data containing multimedia data, and the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

21. A processor for processing multimedia data in the transmitting device, characterized in that it contains the definition block, configured to set the parameter that indicates the maximum number of transfer units, containing multimedia data that precede any transmission unit containing media is haunted data in the stream of packets in the sequence of transmission units transmission and follow the transmission unit containing multimedia data, in the sequence decoding.

22. The processor according to item 21, characterized in that the said multimedia data contains quantum encoded image.

23. The processor according to item 21, wherein the said transmission unit containing media data unit is a level of abstraction from the network containing the data level video encoding.

24. A processor for processing multimedia data in a receiving device for receiving a stream of coded images in the form of transfer units containing quantized data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of transfer units, containing multimedia data that precede any transmission unit containing multimedia data packet stream in the sequence of transmission units transmission and follow the transmission unit containing multimedia data, in the sequence decoding.

25. The processor according to paragraph 24, characterized in that it contains the memory, and the aforementioned tool that uses the parameter block contains the definitions for the analysis of the mentioned parameter and reserves the financing of bits of memory for buffering in said memory according to said parameter.

26. The processor according to paragraph 24, characterized in that it contains a decoder for decoding pictures from a received stream of coded images and the means for using the reserved bits of memory for buffering the encoded image.

27. An encoder for encoding the multimedia data included in the blocks of data, the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, wherein the block contains the definitions made with the ability to set the parameter that indicates the maximum number of blocks of data that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

28. The encoder according to item 27, characterized in that the said multimedia data contains quantum encoded image.

29. The encoder according to item 27, wherein the said transmission unit containing media data unit is a level of abstraction from the network containing the data level video encoding.

30. A decoder for decoding the encoded stream of images included in the blocks of data, provided the multimedia data, while the blocks of data arranged in a transfer sequence that is at least partially differs from the sequence of decoding multimedia data in blocks of data, characterized in that it contains means to specify requirements for buffering through the use of a parameter that indicates the maximum number of transfer units that precede any transmission unit of data in the packet stream in the transfer sequence and follow the block of data in the sequence decoding.

31. The decoder according to item 30, characterized in that it contains the memory, and the aforementioned tool that uses the parameter block contains the definitions for the analysis of the mentioned parameter and to reserve bits of memory for buffering in said memory according to said parameter.

32. Decoder p, characterized in that it contains means for use of reserved bits of memory for buffering the encoded images.



 

Same patents:

FIELD: information technologies.

SUBSTANCE: method and device are suggested for multilevel integration used for elimination of errors. Error is detected in multimedia data on the basis of the first level protocol, and then error detected in multimedia data is masked on the basis of the second level protocol. In one aspect error in multimedia data is eliminated on the basis of communication level protocol, and it is controlled on the basis of transport level protocol. Further distribution of controlled error is determined on the basis of synchronisation level protocol, then error detected in multimedia data is masked on the basis of applied level protocol. Further stage of error elimination and scaling stage are provided.

EFFECT: increased efficiency of multimedia data stream processing by reception of multiple streams of coded multimedia data, eliminating errors in erroneous part of stream and recovering multimedia data from multiple streams.

40 cl, 10 dwg

FIELD: information technology.

SUBSTANCE: invention relates to buffering packets of a media stream during transmission from a transmission device to a receiving device. Media packets are generated from at least one type of media information in a stream generator; at least one transmission frame is generated based on transmitted media packets; transmitted packets are generated from at least one transmission frame and a transmission schedule is generated for transmitted packets. In addition, the first and second steps of hypothetical decoding are executed. The first step of hypothetical decoding is executed in accordance with the transmission schedule, and involves buffering the transmitted packets in accordance with the transmission schedule in the first buffer for hypothetical decoding and output of packets from the first buffer for hypothetical decoding based on the transmission frame. The second step of hypothetical decoding involves controlling occupance rate of the first and second buffer for hypothetical decoding by controlling at least one of the following: operation of the stream generator, generation of at least one transmission frame, transmission schedule.

EFFECT: more efficient buffering of media stream packets.

20 cl, 7 dwg

FIELD: image transferring equipment engineering, possible use in multimedia communications.

SUBSTANCE: in accordance to method, when error codes are detected on receiving side, data of code stream of image with error codes are refused prior to decoding of data of code stream of image, and refused data of code stream of image are replaced with data of code stream of image, positioned in appropriate position of previous frame, and data of code stream of image are encoded continuously. Also, an array of marks is set up for data of code stream of image prior to encoding on receiving side, to perform recording of positions, where error codes have been detected.

EFFECT: possible avoidance of transfer of internal frame images on transmitting side and of frozen images on receiving side, or decrease of their occurrence periods, thus improving quality of image.

7 cl, 2 dwg

FIELD: re-synchronization.

SUBSTANCE: method can be used in decoding channel according to MPEG-4 standard. To provide proper decoding of pressed video data signal, the re-synchronization word RW differs from known words of variable length code VLC as well as from start code of plane of video object and has at least 17 sequent zeros, after which the unit follows, for plane of video object coded to provide two-directional prediction. After error in transmission in pressed video signal is detected, the pressed video data signal can be re-synchronized.

EFFECT: higher efficiency of re-synchronization.

4 cl, 2 dwg

The invention relates to encoding and decoding digital data divided into blocks of digits, in order of importance digits

The invention relates to television, in particular to the processing of the image data, and in particular to a method and apparatus for loop-filtering the image data

FIELD: physics; image processing.

SUBSTANCE: invention relates to a method of buffering multimedia information, as well as a method of decoding a coded stream of images in a decoder, in which the coded stream of images is received in form of transmission blocks which contain multimedia data. A system for processing multimedia data is proposed, which contains a coder for coding images and a buffer for buffering multimedia data. Multimedia data are included in the data transmission blocks. The data transmission blocks are ordered in the transmission sequence, which at least partially differs from the sequence of decoding multimedia data in transmission blocks. There is also definition block, which can set a parametre which indicates the maximum number of data transmission blocks which precede any data transmission block in a stream of packets in the transmission sequence and that data transmission block is tracked in the decoding sequence.

EFFECT: more efficient compression when buffering multimedia information.

32 cl, 7 dwg

FIELD: information technologies.

SUBSTANCE: method includes the following: operation of coding for formation of coded images in coder, and also operation of specified coded images transfer in decoder in the form of transmission units, operation of buffering for buffering of transmission units sent to decoder, in buffer and operation of decoding for decoding of coded images with production of decoded images. Buffer size is specified by means of determining overall size of at least two transmission units and setting maximum size of buffer on the basis of this overall size.

EFFECT: improved efficiency of coded images buffering.

22 cl, 16 dwg

FIELD: information technologies.

SUBSTANCE: method for decoding of compressed video sequence, at the same time image frames are introduced into buffer memory related to decoding. Video sequence includes indication related to at least one gap in numbering of image frames, besides this indication is decoded from video sequence. Further, in response to this indication, buffer memory is configured so that it provides for number of images frames corresponding to gap in numbering of image frames, and images frames in buffer memory are used in process of decoding. Preferentially, specified indication informs about the fact that at least one gap in numbering of image frames in video sequence is deliberate, and specified number of image frames is used in buffer memory instead of image frames, which are not available in decoder.

EFFECT: provision of the possibility for decoder to account for image frames, which were deleted deliberately by coder.

31 cl, 14 dwg

FIELD: re-synchronization.

SUBSTANCE: method can be used in decoding channel according to MPEG-4 standard. To provide proper decoding of pressed video data signal, the re-synchronization word RW differs from known words of variable length code VLC as well as from start code of plane of video object and has at least 17 sequent zeros, after which the unit follows, for plane of video object coded to provide two-directional prediction. After error in transmission in pressed video signal is detected, the pressed video data signal can be re-synchronized.

EFFECT: higher efficiency of re-synchronization.

4 cl, 2 dwg

FIELD: image transferring equipment engineering, possible use in multimedia communications.

SUBSTANCE: in accordance to method, when error codes are detected on receiving side, data of code stream of image with error codes are refused prior to decoding of data of code stream of image, and refused data of code stream of image are replaced with data of code stream of image, positioned in appropriate position of previous frame, and data of code stream of image are encoded continuously. Also, an array of marks is set up for data of code stream of image prior to encoding on receiving side, to perform recording of positions, where error codes have been detected.

EFFECT: possible avoidance of transfer of internal frame images on transmitting side and of frozen images on receiving side, or decrease of their occurrence periods, thus improving quality of image.

7 cl, 2 dwg

FIELD: information technology.

SUBSTANCE: invention relates to buffering packets of a media stream during transmission from a transmission device to a receiving device. Media packets are generated from at least one type of media information in a stream generator; at least one transmission frame is generated based on transmitted media packets; transmitted packets are generated from at least one transmission frame and a transmission schedule is generated for transmitted packets. In addition, the first and second steps of hypothetical decoding are executed. The first step of hypothetical decoding is executed in accordance with the transmission schedule, and involves buffering the transmitted packets in accordance with the transmission schedule in the first buffer for hypothetical decoding and output of packets from the first buffer for hypothetical decoding based on the transmission frame. The second step of hypothetical decoding involves controlling occupance rate of the first and second buffer for hypothetical decoding by controlling at least one of the following: operation of the stream generator, generation of at least one transmission frame, transmission schedule.

EFFECT: more efficient buffering of media stream packets.

20 cl, 7 dwg

FIELD: information technologies.

SUBSTANCE: method and device are suggested for multilevel integration used for elimination of errors. Error is detected in multimedia data on the basis of the first level protocol, and then error detected in multimedia data is masked on the basis of the second level protocol. In one aspect error in multimedia data is eliminated on the basis of communication level protocol, and it is controlled on the basis of transport level protocol. Further distribution of controlled error is determined on the basis of synchronisation level protocol, then error detected in multimedia data is masked on the basis of applied level protocol. Further stage of error elimination and scaling stage are provided.

EFFECT: increased efficiency of multimedia data stream processing by reception of multiple streams of coded multimedia data, eliminating errors in erroneous part of stream and recovering multimedia data from multiple streams.

40 cl, 10 dwg

FIELD: physics; image processing.

SUBSTANCE: invention relates to a method of buffering multimedia information, as well as a method of decoding a coded stream of images in a decoder, in which the coded stream of images is received in form of transmission blocks which contain multimedia data. A system for processing multimedia data is proposed, which contains a coder for coding images and a buffer for buffering multimedia data. Multimedia data are included in the data transmission blocks. The data transmission blocks are ordered in the transmission sequence, which at least partially differs from the sequence of decoding multimedia data in transmission blocks. There is also definition block, which can set a parametre which indicates the maximum number of data transmission blocks which precede any data transmission block in a stream of packets in the transmission sequence and that data transmission block is tracked in the decoding sequence.

EFFECT: more efficient compression when buffering multimedia information.

32 cl, 7 dwg

FIELD: physics, communications.

SUBSTANCE: invention relates to transmission of a media stream over an error-prone digital video broadcasting - handheld (DVB-H) channel in which media datagrams are labelled according to a priority, packed in a multiprotocol encapsulation section, unequally protected using forward error-correction codes packed into a traffic stream and transmitted into the channel using packets with time-division. A system and a method are proposed for transmitting a multiplexed service stream over a DVB-H channel. Media IP packets are priority labelled. For each packet with time division, the IP packets are grouped based upon the priority labels. Multi protocol encapsulation - forward error correction (MPE-FEC) matrices are made for different priority labels in each packet with time division. Reed-Solomon code data table (RSDT) columns are computed such that the average service bit rate does not overshoot the maximum allowed bit rate, and protection increases with priority. The application data table (ADT) and RSDT of the MPE-FEC matrices are then encapsulated into MPE-FEC sections.

EFFECT: shorter start delay during reception of an unequally protected priority service bit stream.

21 cl, 10 dwg

FIELD: physics, communications.

SUBSTANCE: invention relates to multimedia transmission systems, specifically to methods and a device for acquiring services. Proposed is a service acquisition device which has a source coder configured to generate one or more channel switch video (CSV) signals, which is an independently decoded version of a low-resolution video for the selected channel in a received multiplex transmission and associated one or more multimedia signals, an error coder configured to code CSV signals and multimedia signals for formation of coded error blocks, and a linker configured to encapsulate coded error blocks into a multiplex transmission signal.

EFFECT: fast acquisition of a service and/or switching between services in multiplex transmission.

60 cl, 23 dwg

FIELD: information technologies.

SUBSTANCE: video data is coded, packet is formed with coded video data, and packet is transferred via wireless channel into access network. Level of access control to transfer medium (MAC) receives negative notice from the access network (NAK). It is identified whether received NAK is associated with packet, which contains video data. If received NAK is associated with packet, which contains video data, errors are corrected.

EFFECT: improved efficiency of video data errors correction.

36 cl, 5 dwg

FIELD: information technologies.

SUBSTANCE: method for transmission/reception of signal and device for transmission/reception of signal. Device for transmission of signal includes coder with forward error correction (FEC), which executes FEC-coding of input data for detection and correction of data errors, interleaver, which interleaves FEC-coded data, and unit of symbols display, which displays interleaved data to data of symbol according to method of transmission.

EFFECT: improved efficiency of channel bandwidth use, increased speed of data transmission and increased distance of signal transmission, reduced cost of network development for signal transmission-reception.

15 cl, 33 dwg

FIELD: information technologies.

SUBSTANCE: several various VLC-tables are stored in coding devices, in process of coding and decoding, one of VLC-tables is selected and used to do coding of SVR for this video unit. Table may be selected on the basis of number of neighbouring video units for current video unit, which include non-zero transformation ratios.

EFFECT: increased coefficients of coding of SVR video units, in which structures of coefficients occurring with higher probability, are coded with the help of shorter codes, while structures of coefficients, which occur with lower probability, are coded with the help of longer codes, which is especially useful in coding of video units of improved layers in coding of scalable video.

25 cl, 7 dwg, 1 tbl

Up!