Flexible Macroblock Ordering

H.264/MPEG-4 AVC is a new standard for digital video compression jointly developed by ITU-T’s Video Coding Experts Group (VCEG) and ISO/IEC’s Moving Picture Experts Group (MPEG).Next to efficient coding algorithms, the H.264/AVC specifications define a set of error resilience tools. The most striking one is Flexible Macroblock Ordering or FMO, a technique which is described in this article.

Description

One of the new characteristics of the H.264/AVC standard is the possibility of dividing an image in regions called slice groups. Each slice group can also be divided in several slices; therefore, we need to define the word slice as a sequence of macroblocks that belong to the same slice group. These macroblocks are processed in a scan order (left to right and top to bottom). A slice can be decoded independently.

FMO consists of deciding to which slice each macroblock of the image belongs.Each macroblock can be assigned freely to a slice group using an MBAmap (MacroBlock Allocation map). The MBAmap consists of an identification number for each macroblock of the image that specifies to which slice group that macroblock belongs. The number of slice groups is limited to 8 for each picture to prevent complex allocation schemes.

If we deactivate the FMO, the images will be composed of a single slice with the macroblocks in a scan order. The use of FMO is totally compatible with any type of inter-frame prediction.

With this technique, we can correct errors easily by exploiting the spatial redundancy of the images. It’s a good idea to choose the slice groups in a way that no macroblock and its neighbors belong to the same group. Therefore, if a slice is lost during transmission, it’s easy to reconstruct the lost blocks with the information of the neighboring blocks. We have to consider the transmission characteristics of these slices: each slice is transmitted independently in separate units called packets. Each packet contains in its own header the information to decode itself without any other packet’s information (if the images used as reference are the same in the encoder and the decoder side).The use of FMO together with advanced error resilience tools can keep the visual quality even with a packet loss rate of 10%.

Type

When using FMO, the image can be divided in different scan patterns of the macroblocks. FMO consists of 7 different types, labeled Type 0 to Type 6. Type 6 is the most random one and allows full flexibility to the user. All the other ones contain a certain pattern. These patterns can be exploited when storing and transmitting the MBA

* Type 0: uses runlengths which are repeated to fill the frame. Therefore only those runlengths have to be known to rebuild the image on the decoder side.

* Type 1: also known as scattered slices; it uses a mathematical function, which is known in both the encoder and the decoder, to spread the macroblocks. The distribution in the figure, in which the macroblocks are spread forming a chess board, is very common (see Applications).

* Type 2: is used to mark rectangular areas, so called regions of interest. In this case the coordinates top-left and bottom-right of the rectangles are saved in the MBAmap.

* Type 3-5: are dynamic types that let the slice groups grow and shrink over the different pictures in a cyclic way. Only the growth rate, the direction and the position in the cycle have to be known.

"Note: Each color represents a slice group"

Applications

In this section we describe some of the possible applications of FMO in H.264/AVC video coding:

▪ The FMO Type 1 can be useful to maintain privacy in videoconferences. The image is divided into 2 slice groups with the macroblocks spread forming a chess board. Each slice group is sent in a different packet. This way, if a hacker wants to decode the videoconference, he or she will have to know exactly in which two packets the information is being sent.

▪ The FMO Type 1 can also be used in transmission environments with a high packet loss rate (example seen in "Experimental Results").

▪ The FMO Type 2 can also be very useful. Let’s imagine we want to transmit a news bulletin video with a low bit rate. If we encode the whole image with the bit rate distributed equally for all the macroblocks we'll get some bad results. Humans are particularly sensitive to image errors in faces, so we can mark the region of the newsreader’s face so it's encoded with more bits. Therefore, the background of the set will be encoded with less bits and the newsreader with a lot more bits (maintaining the total bit rate as we had before). This way, the subjective visual quality will be enhanced significantly.

Experimental Results

To do these experiments we have used reference software of the H.264/AVC standard [1] .In the images below we can see two frames of the "Stefan.yuv" sequence, the first one without using FMO and the second one using FMO Type 1:

We can clearly see in the bottom-right corner of the image that some packets were lost during transmission and the macroblocks of the image couldn’t be decoded. But when we use FMO Type 1, we have two slice groups dividing the image like a chess board. Therefore, when a packet of the slice group 0 is lost, we can recover that part of the image interpolating the neighboring macroblocks that have not been lost because they belong to slice group 1.

Conclusions

This overview of FMO and the experimental results appearing in this article clearly show that FMO is a very useful error resilience tool. Its use can enhance visual quality, both subjective and objective (SNR or MSE), when the packets are being transmitted in an environment with a certain packet loss rate.Even though we’ve been exposing the advantages of FMO, we also have to consider its main disadvantage: we have to send the information of which FMO we’re using in the header of the packet. That means we have a significant increase in the number of bits we have to send. Therefore, the use of FMO is only recommended when we know that the transmission environment is likely to have a rather high packet loss rate.

See also

# Video coding
# MPEG-2

External links

# [1] Software H.264: http://iphome.hhi.de/suehring/tml/download/
# T.Wiegand, G.J. Sullivan, G. Bjøntegaard, A.Luthra: "Overview of the H.264/AVC Video Coding Standard". IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, July 2003
# Y.Dhondt, P.Lambert: "Flexible Macroblock Ordering, an error resilience tool in H.264/AVC". Fifth FTW PhD Symposium, Faculty of Engineering, Ghent University, Desembre 2004, Paper No. 106