Contents Page

7.5 Inverse DCT

7.5.1 Non-coded blocks and skipped macroblocks

However, if pattern_code[i] is zero, or if the macroblock is skipped, then that block contains no coefficient data. The sample domain coefficients f[y][x] for such a block shall all take the value zero.

7.6 Motion compensation

In general up to four separate predictions are formed for each block which are combined together to form the final prediction block p[y][x].

In the case of intra coded macroblocks no prediction is formed so that p[y][x] will be zero. The saturation shown in Figure 7-5 is still required in order to remove negative values from f[y][x]. Intra coded macroblocks may carry motion vectors known as “concealment motion vectors”. Despite this no prediction is formed in the normal course of events. This motion vector information is intended for use in the case that bitstream errors preclude the decoding of coefficient information. The way in which a decoder shall use this information is not specified. The only requirement for these motion vectors is that they shall have the correct syntax for motion vectors. A description of the way in which these motion vectors may be used can be found in 7.6.3.9.

In the case where a block is not coded, either because the entire macroblock is skipped or the specific block is not coded there is no coefficient data. In this case f[y][x] is zero and the decoded samples are simply the prediction, p[y][x].

7.6.1 Prediction modes

In field prediction, predictions are made independently for each field by using data from one or more previously decoded fields. Frame prediction forms a prediction for the frame from one or more previously decoded frames. It must be understood that the fields and frames from which predictions are made may themselves have been decoded as either field pictures or frame pictures.

Within a field picture all predictions are field predictions. However in a frame picture either field predictions or frame predictions may be used (selected on a macroblock-by macroblock basis).

In addition to the major classification of field or frame prediction two special prediction modes are used:

• 16x8 motion compensation. In which two motion vectors are used for each macroblock. The first motion vector is used for the upper 16x8 region, the second for the lower 16x8 region. In the case of a bidirectionally predicted macroblock a total of four motion vectors will be used since there will be two for the forward prediction and two for the backward prediction. In this specification 16x8 motion compensation shall only be used with field pictures.

• Dual-prime. In which only one motion vector is encoded (in its full format) in the bitstream together with a small differential motion vector. In the case of field pictures two motion vectors are then derived from this information. These are used to form predictions from two reference fields (one top, one bottom) which are averaged to form the final prediction. In the case of frame pictures this process is repeated for the two fields so that a total of four field predictions are made. This mode shall only be used in P-pictures where there are no B-pictures between the predicted and reference fields or frames.

7.6.2 Prediction field and frame selection

7.6.2.1 Field prediction

NOTES -

2 In the case of predicting a field picture, the field being predicted may be either the top field or the bottom field.

The case when predicting the second field picture of a coded frame is more complicated because the two most recently decoded reference fields shall be used, and in this case, the most recent reference field was obtained from decoding the first field picture of the coded frame. Figure 7-7 illustrates the situation when this second picture is the bottom field. Figure 7-8 illustrates the situation when this second picture is the top field.

NOTE - The earlier reference field may itself have been reconstructed by decoding a field picture or a frame picture.



Figure 7-7. Prediction of the second field-picture when it is the bottom field



Figure 7-8. Prediction of the second field-picture when it is the top field

Field prediction in B-pictures shall be made from the two fields of the two most recently reconstructed reference frames. Figure 7-9 illustrates this situation.

NOTE - The reference frames may themselves have been reconstructed from two coded field-pictures or a single coded frame-picture.



Figure 7-9. Field-prediction of B field pictures or B frame pictures

7.6.2.2 Frame prediction

NOTE - The reference frame may itself have been reconstructed from two field pictures or a single frame picture.

Similarly frame prediction in B-pictures shall be made from the two most recently reconstructed reference frames as illustrated in Figure 7-11.

NOTE - The reference frames themselves may each have been reconstructed from two field pictures or a single frame picture.



Figure 7-11. Frame-prediction for B-pictures

7.6.3 Motion vectors

7.6.3.1 Decoding the motion vectors

delta = motion_code[r][s][t] ;

else {

delta = ( ( Abs(motion_code[r][s][t]) - 1 ) * f ) + motion_residual[r][s][t] + 1;

delta = - delta;

}
prediction = PMV[r][s][t];

if ( (mv_format == “field”) && (t==1) && (picture_structure == “Frame picture”) )

prediction = PMV[r][s][t] DIV 2;
vector’[r][s][t]= prediction + delta;

if (vector’[r][s][t] < low )

vector’[r][s][t] = vector’[r][s][t] + range;

if (vector’[r][s][t] > high)

vector’[r][s][t] = vector’[r][s][t] - range;
if ( (mv_format == “field”) && (t==1) && (picture_structure == “Frame picture”) )

PMV[r][s][t] = vector’[r][s][t] * 2;

else

PMV[r][s][t] = vector’[r][s][t];

motion_code[r][s][t] and motion_residual[r][s][t] are fields recovered from the bitstream. mv_format is recovered from the bitstream using Table 6 17 and Table 6 18.

7.6.3.2 Motion vector restrictions

7.6.3.3 Updating motion vector predictors

The motion vector predictors shall be updated as specified in Table 7-9 and 7-10. The rules for updating motion vector predictors in the case of skipped macroblocks are specified in 7.6.6.

NOTE - It is possible for an implementation to optimise the updating (and resetting) of motion vector predictors depending on the picture type. For example in a P-picture the predictors for backwards motion vectors are unused and need not be maintained.

Table 7-9. Updating of motion vector predictors in frame pictures

frame_motion_-	macroblock_motion_-			macroblock_-
type	forward	backward	intra	Predictors to Update
Frame-based‡	-	-	1	PMV[1][0][1:0] = PMV[0][0][1:0]◊
Frame-based	1	1	0	PMV[1][0][1:0] = PMV[0][0][1:0]
				PMV[1][1][1:0] = PMV[0][1][1:0]
Frame-based	1	0	0	PMV[1][0][1:0] = PMV[0][0][1:0]
Dual prime	1	0	0	PMV[1][0][1:0] = PMV[0][0][1:0]
NOTE - PMV[r][s][1:0] = PMV[u][v][1:0] means that; PMV[r][s][1] = PMV[u][v][1] and PMV[r][s][0] = PMV[u][v][0] ◊ If concealment_motion_vectors is zero then PMV[r][s][t] is set to zero (for all r, s and t). ‡ frame_motion_type is not present in the bitstream but is assumed to be Frame-based § (Only occurs in P-picture) PMV[r][s][t] is set to zero (for all r, s and t). See 7.6.3.4

Table 7-10. Updating of motion vector predictors in field pictures

field_motion_-	macroblock_motion_-			macroblock_-
type	forward	backward	intra	Predictors to Update
Field-based‡	-	-	1	PMV[1][0][1:0] = PMV[0][0][1:0]◊
Field-based	1	1	0	PMV[1][0][1:0] = PMV[0][0][1:0]
				PMV[1][1][1:0] = PMV[0][1][1:0]
Field-based	1	0	0	PMV[1][0][1:0] = PMV[0][0][1:0]
Dual prime	1	0	0	PMV[1][0][1:0] = PMV[0][0][1:0]
NOTE - PMV[r][s][1:0] = PMV[u][v][1:0] means that; PMV[r][s][1] = PMV[u][v][1] and PMV[r][s][0] = PMV[u][v][0] ◊ If concealment_motion_vectors is zero then PMV[r][s][t] is set to zero (for all r, s and t). ‡ field_motion_type is not present in the bitstream but is assumed to be Field-based § (Only occurs in P-picture) PMV[r][s][t] is set to zero (for all r, s and t). See 7.6.3.4

7.6.3.4 Resetting motion vector predictors

• At the start of each slice.

• Whenever an intra macroblock is decoded which has no concealment motion vectors.

• In a P-picture when a non-intra macroblock is decoded in which macroblock_motion_forward is zero.

• In a P-picture when a macroblock is skipped.

7.6.3.5 Prediction in P-pictures

• The prediction type shall be “Field-based”

• The (field) motion vector shall be zero (0;0)

• The motion vector predictors shall be reset to zero

• Predictions shall be made from the field of the same parity as the field being predicted.

If this occurs in a P frame picture the following apply;

• The prediction type shall be “Frame-based”

• The (frame) motion vector shall be zero (0;0)

• The motion vector predictors shall be reset to zero

In the case that a P field picture is used as the second field of a frame in which the first field is an I field picture a series of semantic restrictions apply. These ensure that prediction is only made from the I field picture. These restrictions are;

• There shall be no macroblocks that are coded with macroblock_motion_forward zero and macroblock_intra zero.

• Dual prime prediction shall not be used.

• Field prediction in which motion_vertical_field_select indicates the same parity as the field being predicted shall not be used.

• There shall be no skipped macroblocks.

7.6.3.6 Dual prime additional arithmetic

In order to form a motion vector for the opposite parity (vector’[r][0][1:0]) the existing motion vector is scaled to reflect the different temporal distance between the fields. A correction is made to the vertical component (to reflect the vertical shift between the lines of top field and bottom field) and then a small differential motion vector is added. This process is illustrated in Figure 7-12 which shows the situation for a frame picture.

+ e[parity_ref][parity_pred] + dmvector[1];

In the case of field pictures only one such motion vector is required and here r=2. Thus the (encoded) motion vector used for the same parity prediction is vector’[0][0][1:0] and the motion vector used for the opposite parity prediction is vector’[2][0][1:0].

In the case of frame pictures two such motion vectors are required. Both fields use the encoded motion vector (vector’[0][0][1:0]) for predictions of the same parity. The top field shall use vector’[2][0][1:0] for opposite parity prediction and the bottom field shall use vector’[3][0][1:0] for opposite parity prediction.

7.6.3.7 Motion vectors for chrominance components

vector[r][s][t] = vector’[r][s][t] (for all r, s and t)

For each of the two chrominance components the motion vectors shall be scaled as follows:

4:2:0 Both the horizontal and vertical components of the motion vector are scaled by dividing by two:

vector[r][s][0] = vector’[r][s][0] / 2;

vector[r][s][1] = vector’[r][s][1] / 2;

4:2:2 The horizontal component of the motion vector is scaled by dividing by two, the vertical component is not altered:

vector[r][s][0] = vector’[r][s][0] / 2;

vector[r][s][1] = vector’[r][s][1];

4:4:4 The motion vector is unmodified:

vector[r][s][0] = vector’[r][s][0];

vector[r][s][1] = vector’[r][s][1];

7.6.3.8 Semantic restrictions concerning predictions

NOTE - As explained in 6.1.2 it is, in general, not necessary for the slices to cover the entire picture. However in many defined levels of defined profiles the “restricted slice structure” is used in which case the slices do cover the entire picture. In this case the semantic rule may be more simply stated: “it is a restriction on the bitstream that reconstructed motion vectors shall not refer to samples outside the boundary of the coded picture.”

7.6.3.9 Concealment motion vectors

In the normal course of events no prediction shall be formed for such macroblocks (as would be expected since macroblock_intra = 1). This specification does not specify how error recovery shall be performed. However it is a recommendation that concealment motion vectors are suitable for use by a decoder that performs concealment by forming predictions as if field_motion_type and frame_motion_type (from which the prediction type is derived) have the following values:

• In a field picture; field_motion_type = “Field-based”

• In a frame picture; frame_motion_type = “Frame-based”

NOTE - If concealment is used in an I-picture then the decoder should perform prediction in a similar way to a P-picture.

Concealment motion vectors are intended for use in the case that a data error results in information being lost. There is therefore little point in encoding the concealment motion vector in the macroblock for which it is intended to be used since if the data error results in the need for error recovery it is very likely that the concealment motion vector itself would be lost or corrupted As a result the following semantic rules are appropriate.

• For all macroblocks except those in the bottom row of macroblocks concealment motion vectors should be appropriate for use in the macroblock that lies vertically below the macroblock in which the motion vector occurs.

• When the motion vector is used with respect to the macroblock identified in the previous rule a decoder must assume that the motion vector may refer to samples outside of the slices encoded in the reference frame or reference field.

For all macroblocks in the bottom row of macroblocks the reconstructed concealment motion vectors will not be used. Therefore the motion vector (0;0) may be used to reduce unnecessary overhead.

7.6.4 Forming predictions

A positive value of the horizontal component of a motion vector indicates that the prediction is made from samples (in the reference field/frame) that lie to the right of the samples being predicted.

A positive value of the vertical component of a motion vector indicates that the prediction is made from samples (in the reference field/frame) that lie the below the samples being predicted.

All motion vectors are specified to an accuracy of one half sample. Thus if a component of the motion vector is odd, the samples will be read from mid-way between the actual samples in the reference field/frame. These half-samples are calculated by simple linear interpolation from the actual samples.

In the case of field-based predictions it is necessary to determine which of the two available fields to use to form the prediction. In the case of dual-prime this is specified in that a motion vector is derived for both of the fields and a prediction is formed from each. In the case of field-based prediction and 16x8 MC an additional bit, motion_vertical_field_select, is encoded to indicate which field to use.

If motion_vertical_field_select is zero then the prediction is taken from the top reference field.

If motion_vertical_field_select is one then the prediction is taken from the bottom reference field.

For each prediction block the integer sample motion vectors int_vec[t] and the half sample flags half_flag[t] shall be formed as follows;

for (t=0; t<2; t++) {

int_vec[t] = vector[r][s][t] DIV 2;

if ((vector[r][s][t] - (2 * int_vec[t]) != 0)

half_flag[t] = 1;

else

half_flag[t] = 0;

Then for each sample in the prediction block the samples are read and the half sample prediction applied as follows;

if ( (! half_flag[0] )&& (! half_flag[1]) )

pel_pred[y][x] = pel_ref[y + int_vec[1]][x + int_vec[0]] ;

pel_pred[y][x] = ( pel_ref[y + int_vec[1]][x + int_vec[0]] +

pel_ref[y + int_vec[1]+1][x + int_vec[0]] ) // 2;
if ( half_flag[0]&& (! half_flag[1]) )

pel_pred[y][x] = ( pel_ref[y + int_vec[1]][x + int_vec[0]] +

pel_ref[y + int_vec[1]][x + int_vec[0]+1] ) // 2;
if ( half_flag[0]&& half_flag[1] )

pel_pred[y][x] = ( pel_ref[y + int_vec[1]][x + int_vec[0]] +

pel_ref[y + int_vec[1]][x + int_vec[0]+1] +

pel_ref[y + int_vec[1]+1][x + int_vec[0]] +

pel_ref[y + int_vec[1]+1][x + int_vec[0]+1] ) // 4;

where pel_pred[y][x] is the prediction sample being formed and pel_ref[y][x] are samples in the reference field or frame.

7.6.5 Motion vector selection

7.6.6 Skipped macroblocks

The handling of skipped macroblocks is different between P-pictures and B-pictures. In addition the process differs between field pictures and frame pictures.

There shall be no skipped macroblocks in I-pictures except when:

either picture_spatial_scalable_extension() follows the picture_header() of the current picture.

or sequence_scalable_extension() is present in the bitstream and scalable_mode = “SNR scalability”.

7.6.6.1 P field picture

• The prediction shall be made from the field of the same parity as the field being predicted.

• Motion vector predictors shall be reset to zero.

• The motion vector shall be zero.

7.6.6.2 P frame picture

• Motion vector predictors shall be reset to zero.

• The motion vector shall be zero.

7.6.6.3 B field picture

• The prediction shall be made from the field of the same parity as the field being predicted.

• The direction of the prediction forward/backward/bi-directional shall be the same as the previous macroblock.

• Motion vector predictors are unaffected.

• The motion vectors are taken from the appropriate motion vector predictors. Scaling of the motion vectors for colour components shall be performed as described in 7.6.3.7.

7.6.6.4 B frame picture

• The direction of the prediction forward/backward/bi-directional shall be the same as the previous macroblock.

• Motion vector predictors are unaffected.

• The motion vectors are taken directly from the appropriate motion vector predictors. Scaling of the motion vectors for colour components shall be performed as described in 7.6.3.7.

7.6.7 Combining predictions

It is also necessary to organise the data into blocks that are either field organised or frame organised in order to be added directly to the decoded coefficients.

The transform data is either field organised or frame organised as specified by dct_type.

7.6.7.1 Simple frame predictions

pel_pred[y][x] = (pel_pred_forward[y][x] + pel_pred_backward[y][x])//2;

The predictions for chrominance components of 4:2:0, 4:2:2 and 4:4:4 formats shall be of size 8 samples by 8 lines, 8 samples by 16 lines and 16 samples by 16 lines respectively.

7.6.7.2 Simple field predictions

In the case of simple field prediction in a frame picture the predictions for chrominance components of 4:2:0, 4:2:2 and 4:4:4 formats for each field shall be of size 8 samples by 4 lines, 8 samples by 8 lines and 16 samples by 8 lines respectively.

In the case of simple field prediction in a field picture the predictions for chrominance components of 4:2:0, 4:2:2 and 4:4:4 formats for each field shall be of size 8 samples by 8 lines, 8 samples by 16 lines and 16 samples by 16 lines respectively.

7.6.7.3 16x8 Motion compensation

The predictions for chrominance components, for each 16x8 region, of 4:2:0, 4:2:2 and 4:4:4 formats shall be of size 8 samples by 4 lines, 8 samples by 8 lines and 16 samples by 8 lines respectively.

7.6.7.4 Dual prime

pel_pred[y][x] = (pel_pred_same_parity[y][x] + pel_pred_opposite_parity[y][x])//2;

In the case of dual prime prediction in a frame picture, the predictions for chrominance components of each field of 4:2:0, 4:2:2 and 4:4:4 formats shall be of size 8 samples by 4 lines, 8 samples by 8 lines and 16 samples by 8 lines respectively.

In the case of dual prime prediction in a field picture, the predictions for chrominance components of 4:2:0, 4:2:2 and 4:4:4 formats shall be of size 8 samples by 8 lines, 8 samples by 16 lines and 16 samples by 16 lines respectively.

7.6.8 Adding prediction and coefficient data

The transform data f[y][x] shall be added to the prediction data and saturated to form the final decoded samples d[y][x] as follows;

for (y=0; y<8; y++) {

for (x=0; x<8; x++) {

d[y][x] = f[y][x]+p[y][x];

if (d[y][x] < 0) d[y][x] = 0;

if (d[y][x] > 255) d[y][x] = 255;

}

}