[0001] The invention relates to a method and to an apparatus for storing and transmitting
audio-visual data.
Background
[0002] In the field of compressing audio-visual (AV) data the technological trend goes towards
increasingly parameter-oriented descriptions. Seen from an encoding point of view,
compression parameters in general specify, in which of several possiblemodes certain
parts of the encoder input (AV data) shall be processed. Correspondingly, from a decoding
point of view, those same compression parameters indicate, according to which of several
allowable rules a section of a given compressed bit stream shall be converted back
into a piece of AV data. Compression parameters can be of different nature; assuming
a compression scheme where pictures are subdivided into slices, they may be "global
parameters (GP)", "picture layer parameters (PLP)" and "slice layer parameters (SLP)".
It is expected that GP are mostly constant, since they express parameters such as
picture size or color space. However, they could be changed at the start of a random
access unit, such as a GOP. Therefore, a small set of different GP sets will be referenced
from within an AV bit stream. PLPs express parameters that might change more frequently,
at a maximum on a per-picture basis. Hence, each coded picture will reference one
out of a potentially larger set of such PLPs. Similarly, different SLPs could be referenced
by each coded slice within a coded picture. It is expected that GP are mostly predetermined
by the nature of the specific AV material being compressed, whereas PLP and SLP can
be freely chosen as part of the encoder optimization. For the latter, the selection
is likely to be based on some kind of compression efficiency measure quantifying either
the coding error or the compression bit rate or both. The above-mentioned technology
will be used in the developing MPEG-4 Part 10, also known as H.26L or JVT codec.
Invention
[0003] The invention is based on the recognition of the following facts.
[0004] In addition to specifying and appropriately selecting the compression parameters,
they themselves must in most cases be transmitted or stored along with the AV data
proper, in order to allow a meaningful decoding. This task needs to be tackled separately
for each application domain, wherein the transmission or storage of compression parameters
has to fulfil several, partly conflicting requirements.
- Compactness:
On a global as well as local level, compression parameter description must be kept
compact, such that the total bandwidth needed for AV plus parameters does not outweigh
any gain in compression efficiency achieved by a parameterized approach. This includes
a strong motivation to avoid any unnecessary repetition or other redundancy present
in the compression parameters.
Consequently, the indexing mechanism from a coded picture or slice to the parameter
set(s) will limit the number of parameter sets of each type that can be addressed,
in order to reach the compactness goal. Hence, it must be specified whether or when
it is possible that two different parameter sets are identified by the same parameter
set identifier (numeric or other label). In other words, a name scope for the parameter
set identifiers needs to be established.
- Decodability:
With AV data, especially on a storage medium, being prone to be accessed in non-sequential,
random ways, any parameter set allocation scheme must ensure that all required compression
parameters are made available when entering and decoding a bit stream at any of its
entry points. This includes the important cases of any kind of trick play, and may
entail allowing certain controlled forms of data redundancy.
- Compatibility:
When designing an extension of an existing specification, such as DVD for optical
storage, it is desirable to maintain as many as possible of the well-understood and
established system concepts. Solutions taking this into account are likely to be judged
favorably in the standardization process.
[0005] Therefore, a problem to be solved by the invention is to store/transmit compression
parameter sets optimized for bit rate and compatibility to the logical structures
used in the application domain.
[0006] This problem is solved by the method disclosed in claim 1. An apparatus that utilizes
this method is disclosed in claim 9. A corresponding decoding method and apparatus
is claimed in claim 10 and 11, respectively.
[0007] According to the inventive method a number of compression parameter sets being referenced
from within the coded data used. The compression parameter sets are stored in auxiliary
data files that are uniquely associated to the file or stream containing the main
AV bit stream. Identifiers of the compression parameter sets and the corresponding
references to these identifiers in the main AV bit stream are unique within said auxiliary
data files and associated file or stream containing the main AV bit stream.
[0008] Advantageously, the existing auxiliary data files related to the file or stream containing
the main AV bit stream are extended to carry the compression parameter sets.
[0009] Advantageously, an additional isomorphic set of auxiliary data files for said file
or stream containing the main AV bit stream is created to carry additional information
such as said compression parameter sets, as to not sacrifice compatibility with older
equipment not being able to process extensions to existing auxiliary data files.
[0010] Furthermore, it is of advantage that the validity period of said compression parameter
set identifiers is explicitly signalled.
[0011] Advantageously, the signalling of the validity period of the compression parameter
set identifiers is based on the presentation or decoding time information associated
to portions of the main AV bit stream. However, the signalling of the validity period
of the compression parameter set identifiers may also be based on the file names associated
to portions of the main AV bit stream. Also, the signalling of the validity period
of the compression parameter set identifiers may be based on an existing identification
of the underlying transport or storage entities, such as packets or sectors, associated
to portions of the main AV bit stream carried therein.
[0012] According to a further advantageous embodiment
the compression parameter sets are referenced indirectly, wherein
a. the parameter set identifiers in the AV bitstream of each of a set of parts of
AV material are defined to refer to entries of a secondary table;
b. the secondary table is individually available per each part, and is stored in auxiliary
files in a data structure which is repeated per each part;
c. entries of all secondary tables of said AV material are defined to be themselves
identifiers referring in turn to entries of a common parameter set table provided
once per said AV material.
[0013] Further advantageous embodiments of the invention result from the following description.
Drawing
[0014] Exemplary embodiments of the invention are described with reference to the accompanying
drawings, which show in:
- Fig. 1
- an AV stream with separate parameter sets;
- Fig.2.
- an AV stream with out-of-band parameter sets modelled after the AV data;
- Fig.3.
- the internal structure of PGCI Files existing in DVD;
- Fig.4.
- the internal structure of PGCI Files extended to encompass parameter sets;
- Fig.5.
- the internal structure of PGCI File and a parallel File to encompass parameter sets
;
- Fig.6.
- parameter Sets with Scope Period information added;
- Fig.7.
- an indirect referencing mechanism.
Exemplary embodiments
[0015] For video compression schemes such as MPEG-4 Part 10, compression parameter sets
are generated during the video encoding process, are stored separate from the main
AV bit stream and then are being referenced from there, as shown in Figure 1.
[0016] As depicted in Figure 2 compression parameter sets are stored out-of-band, i.e. in
a file or set of files separate from the main AV file (equivalently: to transmit the
parameter sets separately and prior to any subsequent transmission of AV data). Preferably,
the internal structure of such a parameter set file/stream should closely follow that
of the AV data it is related to, as shown in Figure 2. Typically, the parameter set
identifiers must remain unique within an "encoding session". In other words, the same
identifier cannot be used for two different parameter sets within the encoding session.
Hence, the encoding session constitutes the scope of these identifiers.
[0017] An "encoding session" is mapped to meaningful entities on a storage medium, such
as DVD, as follows: DVD identifies a hierarchy of semantic entities, among them "Program
Chain", "Cell" and "Video Object Unit (VOBU)". In order to minimize unnecessary repetition
of parameter sets, the scope of the parameter set identifiers is limited to the biggest
possible of these entities. In DVD, Information about entities constitutes part of
Navigation Data, and is assembled in the "Video Manager Information (VMGI)" and "Video
Title Set Information (VTSI)" files, to be commonly called PGCI files hereafter. Figure
3 shows the internal structure of these PGCI files.
[0018] Furthermore, either the PGCI files may be extended as defined in DVD by including
the compression parameter sets directly in them (Figure 4) or, advantageously in order
to maintain compatibility with existing DVD players, to create separate files with
an internal structure analogous to that of the PGCI files but just containing the
compression parameter sets (Figure 5).
[0019] During authoring, different parts of a Program Chain may have been encoded separately
prior to recording on a storage medium. In that case, it is very likely that, between
these parts, the same parameter set identifier happens to be assigned to different
parameter sets. In order to support easy authoring of such compound content, explicit
identification about the scope period of the parameter set identifiers may be added.
It is not a good option to add such identification to the AV bit stream where compression
parameter sets are being referenced, since that would require more parsing of the
bit stream and would increase the bit rate. Therefore, it is proposed to amend each
compression parameter set or groups of such parameter sets as stored in out-of-band
files with information specifying its scope period (Figure 6). Such scope period information
can be based on the time stamps within the AV material, on file names identifying
different parts of the AV bitstream (VOB files on DVD), or on sector numbers on the
disc.
[0020] In cases of compound content as described above, although parameter set identifier
assignment is likely to vary between separately encoded parts of an overall AV material,
it is nevertheless likely that many if not all of these parameter sets, despite bearing
different identifiers, are actually identical; since the employed encoding equipment
may internally use and choose from a limited set of predefined parameter sets. In
this case, as another way to avoid unnecessary repetition of compression parameter
sets, it is additionally proposed to employ an indirect mechanism of referencing parameter
sets, where the parameter set identifiers in the AV bitstream of each part are defined
to be referring to a small secondary table individually available per each part, where
this secondary table is stored in the out-of-band files in a data structure which
is repeated per each part, and where the entries of all the secondary tables of the
overall AV material are defined to be themselves identifiers referring in turn to
entries of a common parameter set table provided once (Figure 7).
[0021] The invention has the advantage of ease of formatting authored AV bitstreams onto
a storage medium as well as bit rate efficiency. It is expected that authoring tools
will store the parameter sets separate from the bit stream of AV data, as this is
the original goal of the usage of parameter sets. Each parameter set needs only be
stored once and can be referenced multiple times from any coded picture on the storage
medium. Employing an indirect referencing scheme, parameter set management remains
efficient even if AV assets come from heterogenious sources.
[0022] The invention is especially useful for transport or storage of AV bit streams. However,
the invention is also applicable to other kinds of bit streams.
1. Video coding method using a number of compression parameter sets being referenced
from within the coded data, wherein said compression parameter sets are stored in
auxiliary data files that are uniquely associated to the file or stream containing
the main AV bit stream, and wherein identifiers of said compression parameter sets
and the corresponding references to these identifiers in the main AV bit stream are
unique within said auxiliary data files and associated file or stream containing the
main AV bit stream.
2. Video coding method according to claim 1, wherein existing auxiliary data files related
to said file or stream containing the main AV bit stream are extended to carry said
compression parameter sets.
3. Video coding method according to claim 1, wherein an additional isomorphic set of
auxiliary data files for said file or stream containing the main AV bit stream is
created to carry additional information such as said compression parameter sets, as
to not sacrifice compatibility with older equipment not being able to process extensions
to existing auxiliary data files.
4. Video coding method according to any of claim 1 to 3, wherein the validity period
of said compression parameter set identifiers is explicitly signalled.
5. Video coding method according to claim 4, wherein the signalling of the validity period
of the compression parameter set identifiers is based on the presentation or decoding
time information associated to portions of the main AV bit stream.
6. Video coding method according to claim 4, wherein the signalling of the validity period
of the compression parameter set identifiers is based on the file names associated
to portions of the main AV bit stream.
7. Video coding method according to claim 4, wherein the signalling of the validity period
of the compression parameter set identifiers is based on an existing identification
of the underlying transport or storage entities, such as packets or sectors, associated
to portions of the main AV bit stream carried therein.
8. Video coding method according to any of claims 4 to 7, wherein said compression parameter
sets are being referenced indirectly, and wherein
d. the parameter set identifiers in the AV bitstream of each of a set of parts of
AV material being defined to refer to entries of a secondary table;
e. said secondary table being individually available per each said part, and being
stored in auxiliary files in a data structure which is repeated per each said part;
f. entries of all said secondary tables of said AV material being defined to be themselves
identifiers referring in turn to entries of a common parameter set table provided
once per said AV material.
9. Video coding apparatus using a method according to any of claims 1 to 8.
10. Video decoding method for decoding a video signal coded according to any of claims
1 to 8.
11. Video decoding apparatus using a method according to claim 10.