Deblocking filter
Of the common digital video compression methods, most are
essentially lossless. The most significantly lossy compression
method is quantization either of prediction residual
differences or the transformed frequency domain representation
of such. To a lesser extent, approximations in the spatial to
frequency domain forward and inverse transforms also introduce
errors.
Because the prediction source data for adjacent blocks might
not be from adjacent locations, predicted block data may
exhibit discontinuities across block boundaries. These
discontinuities would be perfectly corrected if residuals were
coded without loss. Due to the information lost as a result of
quantization and transform approximation, residuals do not
perfectly correct block data to avoid such discontinuities. In
fact, if the quantization and transform approximation errors
for residuals in adjacent blocks conspire to affect the data in
opposite directions then the artificial apparent discontinuity
at block boundaries will be made worse.
Greater quantization lends itself to a higher compression
ratio, but naturally results in greater artificial
discontinuities at block boundaries.
The purpose of a deblocking filter is to smooth artificial
discontinuities at block boundaries. This must be performed
both horizontally and vertically. Sometimes video source data
has a discontinuity that happens to fall at a block boundary.
A deblocking filter algorithm should not perform filtering on
such boundaries. Such real video data boundaries can be
detected if the discontinuous difference across the block edge
is greater than the sum of the maximum amount of error possible
for the quantization level used in each block. In cases where
the discontinuous difference is less than the maximum possible
error the amount of filtering might be varied based on the
amount of discontinuity difference. Determining how strongly
to filter based on the amount of discontinuity difference
across block boundaries is an art the can distinguish the
quality of one decoder from another.
For video at low bit rates, with high quantization, the
possible discontinuous difference across block boundaries is
large and therefore the deblocking filter often can not
distinguish real discontinuities in video data from artificial
discontinuities. As a result, the filter will leave block
boundaries unfiltered or lightly filtered, leaving a blocky
looking video result.
In-loop artifact filtering
Filtering, if done effectively, makes the decoded video image
look better. However, filtering does so at the expense of some
loss of original data. Artifact filtering algorithms were
traditionally left to be implemented in different ways by
individual video decoder designers. This meant that prediction
of inter-predicted frames would use different prediction source
data in different decoder implementations. The longer the
series of dependent predictions, the greater the possible
divergence of results between decoders and worse yet, the
greter the divergence between the prediction source used by the
decoder and that expected by an encoder if the encoder was
created by a different designer. Furthermore, verification of
decoder correctness is impossible since for any given bitstream
there are different possible valid results from one decoder to
another.
In newer video coding standards, such as H.264 (AVC), the
deblocking filter is defined by the standard. A decoder
compliant to the standard must deblock reference frames using
the standard deblocking algorithm and none other. All compliant
decoders will generate the same bit exact results. In other
words, for any given bitstream there is exactly one valid set
of decoded video frame data.
Decoder designers are not precluded by any standard from
performing further deblocking and deringing filter operations
on decoded frames before display. Such frames, though, must not
be used for interprediction of future frames.
