METHOD, APPARATUS AND SYSTEM FOR IMPLEMENTING AUDIO MIXING

Description

FIELD OF THE INVENTION

[0001] The present invention relates to the field of communications technologies, and in particular, to a method, an apparatus, and a system for implementing audio mixing.

BACKGROUND OF THE INVENTION

[0002] At present, with development of communications technologies, a video conference has been widely developed and applied. The video conference may be understood as a television conference service in a general sense. By means of multimedia communication, a conference is held by using a television device and a communication network, so that a function of exchanging images, voice, and data between two sites or among multiple sites at the same time may be implemented. A video conference system generally includes a video terminal device, a communication network, and a multipoint control unit (Multipoint Control Unit, MCU).

[0003] Generally, a conventional conference terminal has merely a mono channel or a dual channel, so that a sense of space and direction is generally absent or only a left/right direction can be distinguished. A next generation conference terminal generally adopts a multi-screen solution, in which an image is as large as a real person. In order to experience a strong sense of presence and immersion, a strong sense of direction and space of a sound is generally required. However, a conventional dual channel can no longer satisfy the requirement. In order to reflect a strong sense of direction and space, two solutions are adopted in the prior art. One is to perform encoding and transmission based on a multi channel method. The other is to perform encoding and transmission based on an audio protocol of an audio object. In this way, a sense of direction and space of a sound can be carried in the case that a bit rate is increased slightly. An MCU audio mixing method in the prior art is usually an audio channel-based audio mixing solution.

[0004] The prior art has the following disadvantages:

With an existing audio channel-based audio mixing method, in the same conference, only a conventional conference terminal is compatible, but a next generation multi channel-based terminal and a next generation audio object-based terminal are not compatible.

[0005] EP1855455A1(GLOBAL IP SOLUTIONS GIPS AB [SE]; GLOBAL IP SOLUTIONS INC [US]) 14 November 2007 (2007-11-14) relates to a method and an apparatus for mixing audio signals. The invention is based on the idea to base selection of what audio channels to mix on the dynamic behaviour of the audio channels in terms of whether they are active or not, rather than basing the selection on qualitative measures of the audio channels. According to the invention, a set of audio channels are arranged in a mixing stack and a mixed audio signal is produced from the topmost channels above a predetermined threshold level in the stack.

[0006] WO 2008/148591 A1 (SKYPE LTD [IE]); BEVIN MICHAEL [EE]; SORENSEN KARSTEN VANDBORG [SE]; VA) 11 December 2008 (2008-12-11) relates to a method of transmitting a combined audio signal to at least one of a plurality of participants in a communication event comprising; receiving an audio signal from each of said plurality of participants together with audio activity information associated with each of said received audio signals;analysing a measure of audio activity for each received audio signal based on the audio activity information associated with each of said received signals, wherein said measure of audio activity allows audio signals comprising audio activity to be compared; selecting a set of audio signals from said received audio signals based on the analysed measure of audio activity for each signal; decoding said set of audio signals; and combining said set of audio signals to generate said combined audio signal to be transmitted to said at least one of said plurality of participants.

[0007] WO 2009/050896 A1 (PANASONIC CORP [JP]; ISHIKAWA TOMOKAZU; NORIMATSU TAKESHI; KATAYAMA TA) 23 April 2009 (2009-04-23) relates to a stream generating device comprises an input section for inputting two or more encoding signals including the first down-mixed acoustic signals in which the acoustic signals where two or more sound signals are down-mixed are encoded and expansion signals for acquiring the two or more sound signals from the first down-mixed acoustic signals, an encoding signal generating section for generating a second down-mixed acoustic signal for acquiring each of the first down-mixed acoustic signals and the expansion signal for acquiring each of the first down-mixed acoustic signals from the second down-mixed acoustic signals on the basis of each of the encoding signals inputted by the input section and generating an encoding signal including the generated second down-mixed acoustic signals, the generated expansion signals, and each of the expansion signals of each of the inputted encoding signals, and an output section for outputting the generated encoding signal.

[0008] US 2010/063828 A1 (ISHIKAWA TOMOKAZU [JP] ET AL) 11 March 2010 (2010-03-11) relates to a stream synthesizing device includes an input unit which inputs at least two coded signals each including a first downmix acoustic signal and an extended signal, each of first downmix acoustic signals being obtained by coding an acoustic signal into which at least two sound signals are downmixed, and the extended signal being for obtaining the at least two sound signals out of the first downmix acoustic signal; a coded signal generating unit which generates: a second downmix acoustic signal and an extended signal based on each of coded signals inputted by the input unit, the second downmix acoustic signal being for obtaining each of the first downmix acoustic signals, and the generated extended signal being for obtaining each of the first downmix acoustic signals out of the second downmix acoustic signal; and generate a coded signal including the generated second downmix acoustic signal, the generated extended signal, and each of extended signals included in the corresponding inputted coded signal; and an output unit which outputs the generated coded signal.

SUMMARY OF THE INVENTION

[0009] Embodiments of the present invention provide a method, an apparatus, and a system for implementing audio mixing, so that compatibility with different conference terminals is improved. Technical solutions adopted in the embodiments of the present invention are:

A method for implementing audio mixing includes:

receiving an audio signal sent by each sending conference site, where the received audio signals include an audio channel-based audio signal sent by an audio channel-based sending conference site and an audio object-based audio signal sent by an audio object-based sending conference site;

selecting audio signals for each receiving conference site from the received audio signals;

processing the selected audio signals according to types of receiving conference sites, where the receiving conference sites include an audio channel-based receiving conference site and an audio object-based receiving conference site; and

sending the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site;

wherein when the receiving conference site is an audio channel-based receiving conference site, the processing the selected audio signals according to the types of the receiving conference sites comprises: converting the selected audio signals into audio signals wherein number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference site; and performing, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals; and

when the receiving conference site is an audio object-based receiving conference site, the processing the selected audio signals according to the types of the receiving conference sites comprises: converting the selected audio signals into audio objects according to a presentation mode of the receiving conference site; and combining the converted audio objects into one audio object stream.

[0010] An apparatus for implementing audio mixing is specifically a multipoint control unit, and includes:

a receiving module, configured to receive an audio signal sent by each sending conference site, where the received audio signals include an audio channel-based audio signal sent by an audio channel-based sending conference site and an audio object-based audio signal sent by an audio object-based sending conference site;

a selecting module, configured to select audio signals for each receiving conference site from the received audio signals;

a processing module, configured to process the selected audio signals according to types of receiving conference sites, where the receiving conference sites include an audio channel-based receiving conference site and an audio object-based receiving conference site; and

a sending module, configured to send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site;

wherein the processing module comprises: a first converting unit, configured to convert the selected audio signals into audio signals wherein number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference site; and an audio mixing unit, configured to perform, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals; or

the processing module further comprises: a second converting unit, configured to convert the selected audio signals into audio objects according to a presentation mode of the receiving conference site; and a combining unit, configured to combine the converted audio objects into one audio object stream.

[0011] A system for implementing audio mixing includes multiple sending conference sites, multiple receiving conference sites, and a multipoint control unit, where
the sending conference sites are configured to send audio signals to the multipoint control unit, where the audio signals include an audio channel-based audio signal sent by an audio channel-based sending conference site and an audio object-based audio signal sent by an audio object-based sending conference site;
the multipoint control unit is configured to receive the audio signals sent by the sending conference sites, select audio signals for each receiving conference site from the received audio signals, process the selected audio signals according to types of receiving conference sites, and send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site, where the receiving conference sites include an audio channel-based receiving conference site and an audio object-based receiving conference site; and
the receiving conference sites are configured to receive the processed audio signals from the multipoint control unit;
wherein when the receiving conference site is an audio channel-based receiving conference site, the processing the selected audio signals according to the types of the receiving conference sites comprises: converting the selected audio signals into audio signals wherein number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference site; and performing, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals; and
when the receiving conference site is an audio object-based receiving conference site, the processing the selected audio signals according to the types of the receiving conference sites comprises: converting the selected audio signals into audio objects according to a presentation mode of the receiving conference site; and combining the converted audio objects into one audio object stream.

[0012] With the method, the apparatus, and the system for implementing audio mixing that are provided in the embodiments of the present invention, after receiving an audio channel-based audio signal and an audio object-based audio signal, where the audio channel-based audio signal and the audio object-based audio signal are sent by each sending conference site, a multipoint control unit selects an audio signal for each receiving conference site, processes the selected audio signals according to types of receiving conference sites, and sends the processed audio signal to each receiving conference site respectively. Compared with the prior art, the multipoint control unit can perform audio mixing on the audio channel-based audio signal and the audio object-based audio signal, so as to be compatible with a conventional conference terminal, a next generation multi channel-based terminal, and a next generation audio object-based terminal in the same multipoint conference, thus improving quality of user experience.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To describe the technical solutions in the embodiments of the present invention or in the prior art more clearly, the accompanying drawings required for describing the embodiments or the prior art are introduced briefly in the following. Apparently, the accompanying drawings in the following description are only some embodiments of the present invention, and persons of ordinary skill in the art may also derive other drawings from these accompanying drawings without creative efforts.

FIG. 1 is a flowchart of a method for implementing audio mixing according to a first embodiment of the present invention;

FIG. 2 and FIG 3 are flowcharts of a method for implementing audio mixing according to a second embodiment of the present invention;

FIG. 4 is a flowchart of converting a selected audio signal into an audio signal that is consistent with the number of audio channels at an audio channel-based receiving conference site according to the second embodiment of the present invention;

FIG 5 is a schematic diagram of an audio presentation mode of a TelePresence terminal according to the second embodiment of the present invention;

FIG 6 is a schematic structural diagram of an apparatus for implementing audio mixing according to a third embodiment of the present invention;

FIG 7 and FIG 8 are schematic structural diagrams of an apparatus for implementing audio mixing according to a fourth embodiment of the present invention; and

FIG 9 is a schematic structural diagram of a system for implementing audio mixing according to a fifth embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] The technical solutions in the embodiments of the present invention are clearly and fully described in the following with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the embodiments to be described are only a part rather than all of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0015] In order to make the advantages of the technical solutions of the present invention clearer, the present invention is described in detail in the following with reference to the accompanying drawings and embodiments.

Embodiment 1

[0016] This embodiment provides a method for implementing audio mixing. As shown in FIG. 1, the method for implementing audio mixing includes:

101: Receive an audio signal sent by each sending conference site, where the received audio signals include an audio channel-based audio signal and an audio object-based audio signal.

102: Select an audio signal for each receiving conference site from the received audio signals.

103: Process the selected audio signals according to types of receiving conference sites, where the receiving conference sites include an audio channel-based receiving conference site and an audio object-based receiving conference site.

104: Send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site.

[0017] The same conference site may send an audio signal, and may also receive an audio signal. That is to say, a sending conference site and a receiving conference site may be the same conference site. With the method for implementing audio mixing in this embodiment of the present invention, after receiving an audio channel-based audio signal and an audio object-based audio signal, where the audio channel-based audio signal and the audio object-based audio signal are sent by each sending conference site, a multipoint control unit selects an audio signal for each receiving conference site, processes the selected audio signals according to types of receiving conference sites, and sends the processed audio signal to each receiving conference site respectively. Compared with the prior art, the multipoint control unit can perform audio mixing on the audio channel-based audio signal and the audio object-based audio signal, so as to be compatible with a conventional conference terminal, a next generation multi channel-based terminal, and a next generation audio object-based terminal in the same multipoint conference, thus improving quality of user experience.

Embodiment 2

[0018] In this embodiment, a multipoint conference system has multiple sending conference sites and multiple receiving conference sites. The sending conference sites include an audio channel-based sending conference site and an audio object-based sending conference site. An MCU performs audio mixing on an audio signal sent by a sending conference site. The same conference site may send an audio signal, and may also receive an audio signal. That is to say, a sending conference site and a receiving conference site may be the same conference site.

[0019] As shown in FIG 2 and FIG. 3, the method for implementing audio mixing includes:

201: An MCU receives an audio signal sent by each sending conference site, where the received audio signals include an audio channel-based audio signal and an audio object-based audio signal. A type of the audio signal is determined according to a type of a sending conference site. If the sending conference site is an audio channel-based sending conference site, an audio signal sent by the sending conference site is an audio channel-based audio signal. If the sending conference site is an audio object-based sending conference site, an audio signal sent by the sending conference site is an audio object-based audio signal. The audio channel-based sending conference site may be a mono channel-based or multi channel-based sending conference site. Accordingly, the audio channel-based audio signal may be a mono channel-based or multi channel-based audio signal. The audio object refers to an audio source that is regarded as an object. In addition to an audio signal, one audio object further includes auxiliary information. The auxiliary information includes information of the audio object, such as maximum absolute energy, an energy ratio, space information, position information, different playback modes, and corresponding parameters of the different playback modes.

202: The MCU selects an audio signal for each receiving conference site from the received audio signals.

[0020] The MCU may select an audio signal for each receiving conference site according to energy magnitude of each audio signal. A specific selection process may include:

202a: The MCU respectively calculates energy of the audio channel-based audio signal and/or energy of the audio object-based audio signal.

(1) Calculate the energy of the audio channel-based audio signal.

[0021] When the audio channel is a mono channel, the energy of the audio channel-based audio signal is energy of the mono channel.

[0022] When the audio channel is a multi channel, energy of each audio channel is calculated respectively, and maximum audio channel energy is taken as the energy of the audio channel-based audio signal, or an average value of the energy of each audio channel is taken as the energy of the audio channel-based audio signal.

(2) Calculate the energy of the audio object-based audio signal.

[0023] Auxiliary information of the audio object carries maximum absolute energy and an energy ratio of the audio object. The energy ratio of the audio object is the ratio of energy of the audio object to the maximum absolute energy. Absolute energy of the audio object is calculated according to the maximum absolute energy in the audio object and the energy ratio of the audio object. For example, the maximum absolute energy is Emax, energy ratios of audio objects S1, S2, and S3 are respectively a1, a2, and a3, and then absolute energy of the three audio objects is respectively Emax×a1, Emax×a2, and Emax×a3.

[0024] 202b: The MCU selects, according to magnitude of the energy of the audio channel-based audio signal and/or the energy of the audio object-based audio signal, multiple audio signals that have higher audio signal energy for each receiving conference site. The audio signals selected by the MCU for each receiving conference site may be the same, and may also be different.

[0025] For example, the multipoint conference system has five conference sites A, B, C, D, and E. The five conference sites may receive an audio signal, and may also send an audio signal. Audio signals sent by the five conference sites A, B, C, D, and E are A1, B1, C1, D1, and E1 correspondingly. The MCU selects three audio signals B1, C1, and D1 according to energy of the audio signal sent by each conference site. Since each conference site generally does not receive an audio signal sent by the conference site itself, audio signals selected by the MCU for each conference site are shown in Table 1.

Table 1

Receiving conference site	Selected audio signals
A	B1, C1, D1
B	C1, D1
C	B1, D1
D	B1, C1
E	B1, C1, D1

[0026] Furthermore, in order to implement a private conversation between two conference sites in a multipoint conference, audio signals sent by two conference sites that need to have a private conversation can be selected by only the other party of the private conversation, but cannot be selected by other conference sites. Therefore, audio signals selected by the MCU for each conference site are shown in Table 2.

Table 2

Receiving conference site	Selected audio signals
A	B1 C1, D1
B	A1, C1, D1
C	D1
D	D C1
E	C1, D1

[0027] 203: Process the selected audio signals according to types of receiving conference sites, where the receiving conference site may be an audio channel-based receiving conference site or an audio object-based receiving conference site.

(1) When the receiving conference site is an audio channel-based receiving conference site, as shown in FIG 2, the processing the selected audio signals according to the type of the receiving conference site includes:

203a: Convert the selected audio signals into audio signals that are consistent with the number of audio channels of the receiving conference site.

[0028] As shown in FIG 4, the converting the selected audio signals into the audio signals that are consistent with the number of audio channels of the receiving conference site may specifically include:

L1: Judge a type of the selected audio signals, and if the selected audio signals are audio channel-based audio signals, perform step L2; and if the selected audio signals are audio object-based audio signals, perform step L3.

L2: Convert the audio channel-based audio signals into signals that are consistent with the number of audio channels of the receiving conference site.

[0029] The number of audio channels of the audio channel-based conference site may be mono or multi. For a multi channel, generally, microphones are put at different spatial locations, and a signal collected by each of the microphones is respectively encoded by one audio channel. Since space information is already included between audio channels, sound space information of an original sending conference site can be presented when sound is played at a receiving conference site by using the same number of loudspeakers.

[0030] For example, the receiving conference site supports a dual channel signal, and audio channel-based audio signals and selected by the MCU for the receiving conference site are mono channel signals and three-channel signals. Then the MCU copies the mono channel signals to a left channel and a right channel of the receiving conference site at the same time. That is, signal content of the left channel is the same as that of the right channel. For the three-channel signals, the MCU copies a first channel signal to the left channel of the receiving conference site, copies a third channel signal to the right channel of the receiving conference site, and multiplies a second channel signal by a gain of 0.707, and then adds the second channel signal multiplied by the gain of 0.707 to the left channel and the right channel of the receiving conference site at the same time. In this way, the mono channel signals and the three-channel signals are both converted into dual channel signals.

[0031] L3: Judge whether auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site, and if the auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site, perform step L4; and if the auxiliary information of the audio object does not carry a playback mode that is consistent with the number of audio channels of the receiving conference site, perform step L5.

[0032] The auxiliary information of the audio object carries multiple playback modes and corresponding parameters of multiple playback modes. The playback mode refers to performing playback by using how many audio channels, for example, a dual channel or a five-channel. A parameter of each playback mode represents energy distribution of the audio object in each audio channel. The energy distribution may change with time.

[0033] L4: Convert, according to a parameter of a corresponding playback mode in the auxiliary information, the audio object into an audio signal that is consistent with the number of audio channels of the receiving conference site.

[0034] For example, the receiving conference site is a dual channel and the auxiliary information carries a dual-channel playback mode, and then the MCU extracts a parameter corresponding to the dual-channel playback mode, and distributes a signal of the audio object to a left channel and a right channel of the receiving conference site according to the parameter.

[0035] L5: Convert, according to position information of the audio object, the audio object into a signal that is consistent with the number of audio channels of the receiving conference site, where the position information is carried in the auxiliary information of the audio object.

[0036] For example, the auxiliary information has only a dual channel playback mode and a five-channel playback mode but the receiving conference site is a six-channel, and then the MCU converts, according to the position information of the audio object, the audio object into a six-channel audio signal.

[0037] The converting, according to the position information of the audio object, the audio object into the signal that is consistent with the number of audio channels of the receiving conference site may specifically be: determining, according to a position of the audio object and a location of a loudspeaker that is corresponding to each audio channel of the receiving conference site, a loudspeaker that is closest to the audio object; and copying the audio object-based audio signal to an audio channel corresponding to the loudspeaker that is closest to the audio object, while other channels are not given any signal.

[0038] 203b: Perform, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals.

(2) When the receiving conference site is an audio object-based receiving conference site, as shown in FIG 3, the processing the selected audio signals according to the type of the receiving conference site includes:

203c: Convert the selected audio signals into audio objects according to a presentation mode of the receiving conference site.

[0039] The converting the selected audio signals into the audio objects according to the presentation mode of the receiving conference site may specifically include:

S1: Set a presentation mode of the selected audio signals according to the presentation mode of the receiving conference site.

[0040] For example, FIG 5 shows an audio presentation mode of a TelePresence terminal. An image display screen is made up of three screens. Two loudspeakers exist at the bottom of each screen and one loudspeaker exists on both sides of the screen. A total of eight loudspeakers exist. In order to match an image and a sound, when a person at a location of the screen speaks, a loudspeaker near the bottom of the location makes a sound, so as to match the sound and the image. In a multipoint conference, a conference site where a speech is being made may not be a conference site that is currently shown in the screen, at this time, a sound of the conference site may be distributed to two loudspeakers on the two sides of the screen. For example, three audio signals s1, s2, and s3 are selected. A video signal of a conference site corresponding to the audio signal s1 is being viewed by the receiving conference site, and then it may be set that the audio signal s1 is played by six loudspeakers at the bottom of the screen. Video signals of conference sites corresponding to the audio signals s2 and s3 are not viewed currently, and then it may be designated that the audio signals s2 and s3 are respectively played by the two loudspeakers on the two sides of the screen. S2: When the selected audio signals are audio channel-based audio signal, convert the selected audio signals into audio objects according to the set presentation mode.

[0041] S3: When the selected audio signals are audio object-based audio signals, modify an original relevant parameter of the audio object according to the set presentation mode, so as to satisfy a requirement of the set presentation mode.

[0042] 203d: Combine the converted audio objects into one audio object stream.

[0043] 204: Send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site.

[0044] With the method for implementing audio mixing in this embodiment of the present invention, after receiving an audio channel-based audio signal and an audio object-based audio signal, where the audio channel-based audio signal and the audio object-based audio signal are sent by each sending conference site, a multipoint control unit selects audio signals for each receiving conference site, processes the selected audio signals according to types of receiving conference sites, and sends the processed audio signals to each receiving conference site respectively. Compared with the prior art, the multipoint control unit can perform audio mixing on the audio channel-based audio signal and the audio object-based audio signal, so as to be compatible with a conventional conference terminal, a next generation multi channel-based terminal, and a next generation audio object-based terminal in the same multipoint conference, thus improving quality of user experience.

Embodiment 3

[0045] This embodiment provides an apparatus for implementing audio mixing. As shown in FIG 6, the apparatus for implementing audio mixing includes:

[0046] A receiving module 61 is configured to receive an audio signal sent by each sending conference site, where the received audio signals include an audio channel-based audio signal and an audio object-based audio signal.

[0047] A type of the audio signal is determined according to a type of a sending conference site. If the sending conference site is an audio channel-based sending conference site, an audio signal sent by the sending conference site is an audio channel-based audio signal. If the sending conference site is an audio object-based sending conference site, an audio signal sent by the sending conference site is an audio object-based audio signal. The audio channel-based sending conference site may be a mono channel-based or multi channel-based sending conference site. Accordingly, the audio channel-based audio signal may be a mono channel-based or multi channel-based audio signal. The audio object refers to an audio source that is regarded as an object. In addition to an audio signal, one audio object further includes auxiliary information. The auxiliary information includes information of the audio object, such as maximum absolute energy, an energy ratio, space information, position information, difference playback modes, and corresponding parameters of the different playback modes.

[0048] A selecting module 62 is configured to select an audio signal for each receiving conference site from the received audio signals.

[0049] A processing module 63 is configured to process the selected audio signals according to types of receiving conference sites, where the receiving conference sites include an audio channel-based receiving conference site and an audio object-based receiving conference site.

[0050] A sending module 64 is configured to send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site.

[0051] The same conference site may send an audio signal, and may also receive an audio signal. That is to say, a sending conference site and a receiving conference site may be the same conference site.

[0052] With the apparatus for implementing audio mixing in this embodiment of the present invention, after receiving an audio channel-based audio signal and an audio object-based audio signal, where the audio channel-based audio signal and the audio object-based audio signal are sent by each sending conference site, a multipoint control unit selects an audio signal for each receiving conference site, processes the selected audio signals according to types of receiving conference sites, and sends the processed audio signal to each receiving conference site respectively. Compared with the prior art, the multipoint control unit can perform audio mixing on the audio channel-based audio signal and the audio object-based audio signal, so as to be compatible with a conventional conference terminal, a next generation multi channel-based terminal, and a next generation audio object-based terminal in the same multipoint conference, thus improving quality of user experience.

Embodiment 4

[0053] As shown in FIG 7 and FIG 8, an apparatus for implementing audio mixing includes:

[0054] A receiving module 61 is configured to receive an audio signal sent by each sending conference site, where the received audio signals include an audio channel-based audio signal and an audio object-based audio signal.

[0055] A type of the audio signal is determined according to a type of a sending conference site. If the sending conference site is an audio channel-based sending conference site, an audio signal sent by the sending conference site is an audio channel-based audio signal. If the sending conference site is an audio object-based sending conference site, an audio signal sent by the sending conference site is an audio object-based audio signal. The audio channel-based sending conference site may be a mono channel-based or multi channel-based sending conference site. Accordingly, the audio channel-based audio signal may be a mono channel-based or multi channel-based audio signal. The audio object refers to an audio source that is regarded as an object. In addition to an audio signal, one audio object further includes auxiliary information. The auxiliary information includes information of the audio object, such as maximum absolute energy, an energy ratio, space information, position information, different playback modes, and corresponding parameters of the different playback modes.

[0056] A selecting module 62 is configured to select an audio signal for each receiving conference site from the received audio signals.

[0057] A processing module 63 is configured to process the selected audio signals according to types of receiving conference sites, where the receiving conference sites include an audio channel-based receiving conference site and an audio object-based receiving conference site.

[0058] A sending module 64 is configured to send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site.

[0059] The same conference site may send an audio signal, and may also receive an audio signal. That is to say, a sending conference site and a receiving conference site may be the same conference site. The selecting module 62 includes:

a calculating unit 621, configured to respectively calculate energy of the audio channel-based audio signal and/or energy of the audio object-based audio signal; and

a selecting unit 622, configured to select audio signals according to the energy of the audio channel-based audio signal and/or the energy of the audio object-based audio signal, where the selecting unit 622 may select, according to magnitude of the energy of the audio channel-based audio signal and/or the energy of the audio object-based audio signal, multiple audio signals that have higher audio signal energy for each receiving conference site; and the audio signals selected by the selecting unit 622 for each receiving conference site may be the same, and may also be different.

[0060] When the receiving conference site is an audio channel-based receiving conference site, as shown in FIG 7, the processing module 63 may include:

a first converting unit 631, configured to convert the selected audio signals into audio signals that are consistent with the number of audio channels of the receiving conference site; and

an audio mixing unit 632, configured to perform, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals.

[0061] Furthermore, the first converting unit 631 may include:

a first judging sub-unit 6311, configured to judge a type of the selected audio signals;

a first converting sub-unit 6312, configured to, when the selected audio signals are audio channel-based audio signals, convert the audio channel-based audio signals into signals that are consistent with the number of audio channels of the receiving conference site;

a second judging sub-unit 6313, configured to, when the selected audio signals are audio object-based audio signals, judge whether auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site;

a second converting sub-unit 6314, configured to, when the auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site, convert, according to a parameter of a corresponding playback mode in the auxiliary information, the audio object into an audio signal that is consistent with the number of audio channels of the receiving conference site; and

a third converting sub-unit 6315, configured to, when the auxiliary information of the audio objects does not carry a playback mode that is consistent with the number of audio channels of the receiving conference site, convert, according to position information of the audio object, the audio object into a signal that is consistent with the number of audio channels of the receiving conference site, where the position information is carried in the auxiliary information of the audio object.

[0062] When the receiving conference site is an audio object-based receiving conference site, as shown in FIG 8, the processing module 63 may include:

a second converting unit 633, configured to convert the selected audio signals into audio objects according to a presentation mode of the receiving conference site; and

a combining unit 634, configured to combine the converted audio objects into one audio object stream.

[0063] Furthermore, the second converting unit 633 may include:

a setting sub-unit 6331, configured to set a presentation mode of the selected audio signals according to the presentation mode of the receiving conference site;

a fourth converting sub-unit 6332, configured to, when the selected audio signals are audio channel-based audio signals, convert the selected audio signals into audio objects according to the set presentation mode; and

a modifying sub-unit 6333, configured to, when the selected audio signals are audio object-based audio signals, modify a parameter of the audio object according to the set presentation mode.

[0064] After receiving an audio channel-based audio signal and an audio object-based audio signal, where the audio channel-based audio signal and the audio object-based audio signal are sent by each sending conference site, the apparatus for implementing audio mixing in this embodiment of the present invention selects an audio signal for each receiving conference site, processes the selected audio signals according to types of receiving conference sites, and sends the processed audio signal to each receiving conference site respectively. Compared with the prior art, the apparatus can perform audio mixing on the audio channel-based audio signal and the audio object-based audio signal, so as to be compatible with a conventional conference terminal, a next generation multi channel-based terminal, and a next generation audio object-based terminal in the same multipoint conference, thus improving quality of user experience.

Embodiment 5

[0065] This embodiment provides a system for implementing audio mixing, as shown in FIG 9, which includes multiple sending conference sites 91, multiple receiving conference sites 93, and a multipoint control unit 92.

[0066] The sending conference sites 91 are configured to send audio signals to the multipoint control unit 92, where the audio signals include an audio channel-based audio signal and an audio object-based audio signal.

[0067] The multipoint control unit 92 is configured to receive the audio signals sent by the sending conference sites 91, select an audio signal for each receiving conference site 93 from the received audio signals, process the selected audio signals according to types of receiving conference sites 93, and send the processed audio signal to each receiving conference site 93 respectively according to the type of the receiving conference site, where the receiving conference sites 93 include an audio channel-based receiving conference site and an audio object-based receiving conference site.

[0068] The receiving conference sites 93 are configured to receive the processed audio signals from the multipoint control unit 92.

[0069] The same conference site may send an audio signal, and may also receive an audio signal. That is to say, a sending conference site and a receiving conference site may be the same conference site.

[0070] In the system for implementing audio mixing in this embodiment of the present invention, after receiving an audio channel-based audio signal and an audio object-based audio signal, where the audio channel-based audio signal and the audio object-based audio signal are sent by each sending conference site, a multipoint control unit selects an audio signal for each receiving conference site, processes the selected audio signals according to types of receiving conference sites, and sends the processed audio signal to each receiving conference site respectively. Compared with the prior art, the multipoint control unit can perform audio mixing on the audio channel-based audio signal and the audio object-based audio signal, so as to be compatible with a conventional conference terminal, a next generation multi channel-based terminal, and a next generation audio object-based terminal in the same multipoint conference, thus improving quality of user experience.

[0071] The apparatus and the system for implementing audio mixing that are provided in the embodiments of the present invention can implement the method embodiments provided in the preceding. The method, the apparatus, and the system for implementing audio mixing that are provided in the embodiments of the present invention may be applied to the same multipoint conference where a conventional conference terminal, a next generation multi channel-based terminal, and a next generation audio object-based terminal are compatible, but the present invention is not limited to this.

[0072] Persons of ordinary skill in the art may understand that, all or a part of processes in the method in the preceding embodiments may be accomplished by a computer program instructing relevant hardware. The program may be stored in a computer-readable storage medium. When the program is executed, processes of the method in the preceding embodiments may be performed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).

[0073] The preceding descriptions are merely specific embodiments of the present invention, but are not intended to limit the protection scope of the present invention. Variations or replacements that may be easily thought by persons skilled in the art within the technical scope disclosed by the present invention shall all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for implementing audio mixing, comprising:

receiving (101) an audio signal sent by each sending conference site, wherein the received audio signals comprise an audio channel-based audio signal sent by an audio channel-based sending conference site and an audio object-based audio signal sent by an audio object-based sending conference site;

selecting (102) audio signals for each receiving conference site from the received audio signals;

processing (103) the selected audio signals according to types of receiving conference sites, wherein the receiving conference sites comprises an audio channel-based receiving conference site and an audio object-based receiving conference site; and

sending (104) the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site;

wherein when the receiving conference site is an audio channel-based receiving conference site, the processing of the selected audio signals according to the types of the receiving conference sites comprises: converting the selected audio signals into audio signals wherein the number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference site; and performing, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals; and

when the receiving conference site is an audio object-based receiving conference site, the processing of the selected audio signals according to the types of the receiving conference sites comprises: converting the selected audio signals into audio objects according to a presentation mode of the receiving conference site; and combining the converted audio objects into one audio object stream.

2. The method for implementing audio mixing according to claim 1, wherein the selecting the audio signal for each receiving conference site from the received audio signals specifically is:

respectively calculating energy of the audio channel-based audio signal and/or energy of the audio object-based audio signal; and

selecting an audio signal according to the energy of the audio channel-based audi.o signal and/or the energy of the audio object-based audio signal.

3. The method for implementing audio mixing according to claim 2, wherein the calculating the energy of the audio channel-based audio signal comprises:

when the audio channel is a mono channel, taking the energy of the audio channel-based audio signal as energy of the mono channel; and

when the audio channel is a multi channel, calculating energy of each audio channel respectively, and taking maximum audio channel energy as the energy of the audio channel-based audio signal, or taking an average value of energy of each audio channel as the energy of the audio channel-based audio signal.

4. The method for implementing audio mixing according to claim 2, wherein the calculating the energy of the audio object-based audio signal comprises:

calculating absolute energy of the audio object according to maximum absolute energy and a ratio of energy of the audio object to the maximum absolute energy, wherein the maximum absolute energy and the ratio of energy of the audio object to the maximum absolute energy are carried in auxiliary information of the audio object.

5. The method for implementing audio mixing according to claim 1, wherein the converting the selected audio signals into the audio signals wherein the number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference site comprises:

judging a type of the selected audio signals;

if the selected audio signals are audio channel-based audio signals, converting the audio channel-based audio signals into signals wherein the number of channels of each converted signal is consistent with the number of audio channels of the receiving conference site;

if the selected audio signals are audio object-based audio signals, judging whether auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site;

if the auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site, converting, according to a parameter of a corresponding playback mode in the auxiliary information, the audio object into an audio signal wherein the number of channels of the converted audio signal is consistent with the number of audio channels of the receiving conference site; and

if the auxiliary information of the audio object does not carry a playback mode that is consistent with the number of audio channels of the receiving conference site, converting, according to position information of the audio object, the audio object into a signal wherein the number of channels of the converted signal is consistent with the number of audio channels of the receiving conference site, wherein the position information is carried in the auxiliary information of the audio object.

6. The method for implementing audio mixing according to claim 5, wherein the converting, according to the position information of the audio object, the audio object into the signal wherein the number of channels of the converted signal is consistent with the number of audio channels of the receiving conference site comprises:

determining, according to a position of the audio object and a location of a loudspeaker that is corresponding to each audio channel of the receiving conference site, a loudspeaker that is closest to the audio object; and

copying the audio object-based audio signal to an audio channel corresponding to the loudspeaker that is closest to the audio object.

7. The method for implementing audio mixing according to claim 1, wherein the converting the selected audio signals into the audio objects according to the presentation mode of the receiving conference site comprises:

setting a presentation mode of the selected audio signals according to the presentation mode of the receiving conference site;

when the selected audio signals are audio channel-based audio signals, converting the selected audio signals into audio objects according to the set presentation mode; and

when the selected audio signals are audio object-based audio signals, modifying a parameter of the audio object according to the set presentation mode.

8. An apparatus for implementing audio mixing, specifically a multipoint control unit, wherein the apparatus comprises:

a receiving module(61), configured to receive an audio signal sent by each sending conference site, wherein the audio signals comprise an audio channel-based audio signal sent by an audio channel-based sending conference site and an audio object-based audio signal sent by an audio object-based sending conference site;

a selecting module(62), configured to select audio signals for each receiving conference site from the received audio signals;

a processing module(63), configured to process the selected audio signals according to types of receiving conference sites, wherein the receiving conference sites comprise an audio channel-based receiving conference site and an audio object-based receiving conference site; and

a sending module(64), configured to send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site;

wherein

the processing module (63) comprises: a first converting unit(631), configured to convert the selected audio signals into audio signals wherein the number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference, site; and an audio mixing unit(632), configured to perform, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals;

or

the processing module (63) comprises: a second converting unit(633), configured to convert the selected audio signals into audio objects according to a presentation mode of the receiving conference site; and a combining unit(634), configured to combine the converted audio objects into one audio object stream.

9. The apparatus for implementing audio mixing according to claim 8, wherein the selecting module(62) comprises:

a calculating unit(62 1), configured to respectively calculate energy of the audio channel-based audio signal and energy of the audio object-based audio signal; and

a selecting unit(622), configured to select audio signals according to the energy of the audio channel-based audio signal and the energy of the audio object-based audio signal.

10. The apparatus for implementing audio mixing according to claim 9, wherein when the processing module (63) comprises a first converting unit (631), the first converting unit(631) comprises:

a first judging sub-unit(6311), configured to judge a type of the selected audio signals;

a first converting sub-unit(6312), configured to, when the selected audio signals are audio channel-based audio signals, convert the audio channel-based audio signals into signals wherein the number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference site;

a second judging sub-unit(6313), configured to, when the selected audio signals are audio object-based audio signals, judge whether auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site;

a second converting sub-unit(6314), configured to, when the auxiliary information of the audio object carries a playback mode that is consistent with the number of audio channels of the receiving conference site, convert, according to a parameter of a corresponding playback mode in the auxiliary information, the audio object into an audio signal wherein the number of channels of the converted audio signal is consistent with the number of audio channels of the receiving conference site; and

a third converting sub-unit(6315), configured to, when the auxiliary information of the audio object does not carry a playback mode that is consistent with the number of audio channels of the receiving conference site, convert, according to position information of the audio object, the audio object into a signal wherein the number of channels of the signal is consistent with the number of audio channels of the receiving conference site, wherein the position information is carried in the auxiliary information of the audio object.

11. The apparatus for implementing audio mixing according to claim 8, wherein when the processing module (63) comprises a second converting unit (633), the second converting unit(633) comprises:

a setting sub-unit(6331), configured to set a presentation mode of the selected audio signals according to the presentation mode of the receiving conference site;

a fourth converting sub-unit(6332), configured to, when the selected audio signals are audio channel-based audio signals, convert the selected audio signals into audio objects according to the set presentation mode; and

a modifying sub-unit(6333), configured to, when the selected audio signals are audio object-based audio signals, modify a parameter of the audio object according to the set presentation mode.

12. A system for implementing audio mixing, comprising multiple sending conference sites(91), multiple receiving conference sites(93), and a multipoint control unit(92), wherein:

the sending conference sites(91) are configured to send audio signals to the multipoint control unit(92), wherein the audio signals comprise an audio channel-based audio signal sent by an audio channel-based sending conference site and an audio object-based audio signal sent by an audio object-based sending conference site;

the multipoint control unit(92) is configured to receive the audio signals sent by the sending conference sites(91), select audio signals for each receiving conference site from the received audio signals, process the selected audio signals according to types of receiving conference sites(93), and send the processed audio signal to each receiving conference site respectively according to the type of the receiving conference site, wherein the receiving conference sites(93) comprise an audio channel-based receiving conference site and an audio object-based receiving conference site; and

the receiving conference sites(93) are configured to receive the processed audio signals from the multipoint control unit(92);

wherein when the receiving conference site is an audio channel-based receiving conference site, the processing of the selected audio signals according to the types of the receiving conference sites(93) comprises: converting the selected audio signals into audio signals wherein the number of channels of each converted audio signal is consistent with the number of audio channels of the receiving conference site; and performing, based on the audio channel of the receiving conference site, audio mixing on the converted audio signals; and

when the receiving conference site is an audio object-based receiving conference site, the processing of the selected audio signals according to the types of the receiving conference sites(93) comprises: converting the selected audio signals into audio objects according to a presentation mode of the receiving conference site; and combining the converted audio objects into one audio object stream.

Ansprüche

1. Verfahren zum Implementieren einer Audiomischung, das Folgendes umfasst:

Empfangen (101) eines Audiosignals, das von jedem sendenden Konferenzstandort gesendet wird, wobei die empfangenen Audiosignale ein durch einen audiokanalbasierten sendenden Konferenzstandort gesendetes audiokanalbasiertes Audiosignal und ein durch einen audioobjektbasierten sendenden Konferenzstandort gesendetes audioobjektbasiertes Audiosignal umfassen;

Auswählen (102) von Audiosignalen für jeden empfangenden Konferenzstandort aus den empfangenen Audiosignalen;

Verarbeiten (103) der ausgewählten Audiosignale gemäß Typen von empfangenden Konferenzstandorten, wobei die empfangenden Konferenzstandorte einen audiokanalbasierten empfangenden Konferenzstandort und einen audioobjektbasierten empfangenden Konferenzstandort umfassen; und

jeweiliges Senden (104) des verarbeiteten Audiosignals zu jedem empfangenden Konferenzstandort gemäß dem Typ des empfangenden Konferenzstandorts;

wobei dann, wenn der empfangende Konferenzstandort ein audiokanalbasierter empfangender Konferenzstandort ist, das Verarbeiten der ausgewählten Audiosignale gemäß den Typen der empfangenden Konferenzstandorte Folgendes umfasst: Umsetzen der ausgewählten Audiosignale in Audiosignale, wobei die Anzahl von Kanälen jedes umgesetzten Audiosignals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist; und Ausführen einer Audiomischung auf den umgesetzten Audiosignalen basierend auf dem Audiokanal des empfangenden Konferenzstandorts; und

dann, wenn der empfangende Konferenzstandort ein audioobjektbasierter empfangender Konferenzstandort ist, das Verarbeiten der ausgewählten Audiosignale gemäß den Typen der empfangenden Konferenzstandorte Folgendes umfasst: Umsetzen der ausgewählten Audiosignale in Audioobjekte gemäß einem Präsentationsmodus des empfangenden Konferenzstandorts; und Kombinieren der umgesetzten Audioobjekte in einen Audioobjektdatenstrom.

2. Verfahren zum Implementieren einer Audiomischung nach Anspruch 1, wobei das Auswählen des Audiosignals für jeden empfangenden Konferenzstandort aus den empfangenen Audiosignalen insbesondere Folgendes ist:

jeweils Berechnen der Energie des audiokanalbasierten Audiosignals und/oder der Energie des audioobjektbasierten Audiosignals; und

Auswählen eines Audiosignals gemäß der Energie des audiokanalbasierten Audiosignals und/oder der Energie des audioobjektbasierten Audiosignals.

3. Verfahren zum Implementieren einer Audiomischung nach Anspruch 2, wobei das Berechnen der Energie des audiokanalbasierten Audiosignals Folgendes umfasst:

wenn der Audiokanal ein Monokanal ist, Übernehmen der Energie des audioobjektbasierten Audiosignals als Energie des Monokanals; und

wenn der Audiokanal ein Multikanal ist, jeweils Berechnen der Energie jedes Audiokanals und Übernehmen der maximalen Audiokanalenergie als die Energie des audiokanalbasierten Audiosignals oder Übernehmen eines Mittelwerts der Energie jedes Audiokanals als die Energie des audiokanalbasierten Audiosignals.

4. Verfahren zum Implementieren einer Audiomischung nach Anspruch 2, wobei das Berechnen der Energie des audioobjektbasierten Audiosignals Folgendes umfasst:

Berechnen der absoluten Energie des Audioobjekts gemäß der maximalen absoluten Energie und eines Verhältnisses der Energie des Audioobjekts zu der maximalen absoluten Energie, wobei die maximale absolute Energie und das Verhältnis der Energie des Audioobjekts zu der maximalen absoluten Energie in Zusatzinformationen des Audioobjekts geführt werden.

5. Verfahren zum Implementieren einer Audiomischung nach Anspruch 1, wobei das Umsetzen der ausgewählten Audiosignale in die Audiosignale, wobei die Anzahl von Kanälen jedes umgesetzten Audiosignals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist, Folgendes umfasst:

Beurteilen eines Typs der ausgewählten Audiosignale;

falls die ausgewählten Audiosignale audiokanalbasierte Audiosignale sind, Umsetzen der audiokanalbasierten Audiosignale in Signale, wobei die Anzahl von Kanälen jedes umgesetzten Signals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist;

falls die ausgewählten Audiosignale audioobjektbasierte Audiosignale sind, Beurteilen, ob Zusatzinformationen des Audioobjekts einen Wiedergabemodus führen, der mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist;

falls die Zusatzinformationen des Audioobjekts einen Wiedergabemodus führen, der mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist, Umsetzen des Audioobjekts gemäß einem Parameter eines entsprechenden Wiedergabemodus in den Zusatzinformationen in ein Audiosignal, wobei die Anzahl von Kanälen des umgesetzten Audiosignals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist; und

falls die Zusatzinformationen des Audioobjekts keinen Wiedergabemodus führen, der mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist, Umsetzen des Audioobjekts gemäß Positionsinformationen des Audioobjekts in ein Signal, wobei die Anzahl von Kanälen des umgesetzten Signals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist, wobei die Positionsinformationen in den Zusatzinformationen des Audioobjekts geführt sind.

6. Verfahren zum Implementieren einer Audiomischung nach Anspruch 5, wobei das Umsetzen des Audioobjekts gemäß den Positionsinformationen des Audioobjekts in das Signal, wobei die Anzahl von Kanälen des umgesetzten Signals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist, Folgendes umfasst:

Bestimmen eines Lautsprechers, der dem Audioobjekt am nähesten ist, gemäß einer Position des Audioobjekts und eines Orts eines Lautsprechers, der jedem Audiokanal des empfangenden Konferenzstandorts entspricht; und

Kopieren des audioobjektbasierten Audiosignals in einen Audiokanal, der dem Lautsprecher, der dem Audioobjekt am nähesten ist, entspricht.

7. Verfahren zum Implementieren einer Audiomischung nach Anspruch 1, wobei das Umsetzen der ausgewählten Audiosignale in die Audioobjekte gemäß dem Präsentationsmodus des empfangenden Konferenzstandorts Folgendes umfasst:

Einstellen eines Präsentationsmodus der ausgewählten Audiosignale gemäß dem Präsentationsmodus des empfangenden Konferenzstandorts;

wenn die ausgewählten Audiosignale audiokanalbasierte Audiosignale sind, Umsetzen der ausgewählten Audiosignale in Audioobjekte gemäß dem eingestellten Präsentationsmodus; und

wenn die ausgewählten Audiosignale audioobjektbasierte Audiosignale sind, Modifizieren eines Parameters des Audioobjekts gemäß dem eingestellten Präsentationsmodus.

8. Vorrichtung zum Implementieren einer Audiomischung, insbesondere eine Mehrpunktsteuereinheit, wobei die Vorrichtung Folgendes umfasst:

ein Empfangsmodul (61), das konfiguriert ist, ein von jedem sendenden Konferenzstandort gesendetes Audiosignal zu empfangen, wobei die Audiosignale ein durch einen audiokanalbasierten sendenden Konferenzstandort gesendetes audiokanalbasiertes Audiosignal und ein durch einen audioobjektbasierten sendenden Konferenzstandort gesendetes audioobjektbasiertes Audiosignal umfassen;

ein Auswahlmodul (62), das konfiguriert ist, Audiosignale für jeden empfangenden Konferenzstandort aus den empfangenen Audiosignalen auszuwählen;

ein Verarbeitungsmodul (63), das konfiguriert ist, die ausgewählten Audiosignale gemäß Typen der empfangenden Konferenzstandorte zu verarbeiten, wobei die empfangenden Konferenzstandorte einen audiokanalbasierten empfangenden Konferenzstandort und einen audioobjcktbasierten empfangenden Konferenzstandort umfassen; und

ein Sendemodul (64), das konfiguriert ist, das verarbeitete Audiosignal jeweils zu jedem empfangenden Konferenzstandort gemäß dem Typ des empfangenden Konferenzstandorts zu senden;

wobei

das Verarbeitungsmodul (63) Folgendes umfasst: eine erste Umsetzungseinheit (631), die konfiguriert ist, die ausgewählten Audiosignale in Audiosignale umzusetzen, wobei die Anzahl von Kanälen jedes umgesetzten Audiosignals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist; und

eine Audiomischungseinheit (632), die konfiguriert ist, basierend auf dem Audiokanal des empfangenden Konferenzstandorts Audiomischung auf den umgesetzten Audiosignalen auszuführen;

oder

das Verarbeitungsmodul (63) Folgendes umfasst: eine zweite Umsetzungseinheit (633), die konfiguriert ist, die ausgewählten Audiosignale in Audioobjekte gemäß einem Präsentationsmodus des empfangenden Konferenzstandorts umzusetzen; und eine Kombinierungseinheit (634), die konfiguriert ist, die umgesetzten Audioobjekte in einen Audioobjektdatenstrom zu kombinieren.

9. Vorrichtung zum Implementieren einer Audiomischung nach Anspruch 8, wobei das Auswahlmodul (62) Folgendes umfasst:

eine Berechnungseinheit (621), die konfiguriert ist, die Energie des audiokanalbasierten Audiosignals bzw, die Energie des audioobjektbasierten Audiosignals zu berechnen; und

eine Auswahleinheit (622), die konfiguriert ist, Audiosignale gemäß der Energie des audiokanalbasierten Audiosignals und der Energie des audioobjektbasierten Audiosignals auszuwählen.

10. Vorrichtung zum Implementieren einer Audiomischung nach Anspruch 9, wobei dann, wenn das Verarbeitungsmodul (63) eine erste Umsetzungseinheit (631) umfasst, die erste Umsetzungseinbeit (631) Folgendes umfasst:

eine erste Beurteilungsuntereinheit (6311), die konfiguriert ist, einen Typ der ausgewählten Audiosignale zu beurteilen;

eine erste Umsetzungsuntereinheit (6312), die konfiguriert ist, dann, wenn die ausgewählten Audiosignale audiokanalbasierte Audiosignale sind, die audiokanalbasierten Audiosignale in Signale umzusetzen, wobei die Anzahl von Kanälen jedes umgesetzten Audiosignals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist;

eine zweite Beurteilungsuntereinheit (6313), die konfiguriert ist, dann, wenn die auswählten Audiosignale audioobjektbasierte Audiosignale sind, zu beurteilen, ob Zusatzinformationen des Audioobjekts einen Wiedergabemodus führen, der mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist;

eine zweite Umsetzungsuntereinheit (6314), die konfiguriert ist, dann, wenn die Zusatzinformationen des Audioobjekts einen Wiedergabemodus führen, der mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist, das Audioobjekt gemäß einem Parameter eines entsprechenden Wiedergabemodus in den Zusatzinformationen in ein Audiosignal umzusetzen, wobei die Anzahl von Kanälen des umgesetzten Audiosignals mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist; und

eine dritte Umsetzungsuntereinheit (6315), die konfiguriert ist, dann, wenn die Zusatzinformationen des Audioobjekts keinen Wiedergabemodus führen, der mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts konsistent ist, das Audioobjekt gemäß Positionsinformationen des Audioobjekts in ein Signal umzusetzen, wobei die Anzahl von Kanälen des Signals mit der Anzahl von Audiokanälan des empfangenden Konferenzstandorts konsistent ist, wobei die Positionsinformationen in den Zusatzinformationen des Audioobjekts geführt sind.

11. Vorrichtung zum Implementieren einer Audiomischung nach Anspruch 8, wobei dann, wenn das Verarbeitungsmodul (63) eine zweite Umsetzungseinheit (633) umfasst, die zweite Umsetzungseinheit (633) Folgendes umfasst:

eine Einstellungsuntereinheit (6331), die konfiguriert ist, einen Präsentationsmodus der ausgewählten Audiosignale gemäß dem Präsentationsmodus des empfangenden Konferenzstandorts einzustellen;

eine vierte Umsetzungsuntereinheit (6332), die konfiguriert ist, dann, wenn die ausgewählten Audiosignale audiokanalbasierte Audiosignale sind, die ausgewählten Audiosignale gemäß dem eingestellten Präsentationsmodus in Audioobjekte umzusetzen; und

eine Modifizierungsuntereinheit (6333), die konfiguriert ist, dann, wenn die ausgewählten Audiosignale audioobjektbasierte Audiosignale sind, einen Parameter des Audioobjekts gemäß dem eingestellten Präsentationsmodus zu modifizierten.

12. System zum Implementieren einer Audiomischung, das mehrere sendende Konferenzstandorte (91), mehrere empfangende Konferenzstandorte (93) und eine Mehrpunktsteuereinheit (92) umfasst, wobei:

die sendenden Konferenzstandorte (91) konfiguriert sind, Audiosignale zu der Mehrpunktsteuereinheit (92) zu senden, wobei die Audiosignale ein durch einen audiokanalbasierten sendenden Konferenzstandort gesendetes audiokanalbasiertes Audiosignal und ein durch einen audioobjektbasierten sendenden Konferenzstandort gesendetes audioobjektbasiertes Audiosignal umfassen;

die Mehrpunktsteuereinheit (92) konfiguriert ist, die von den sendenden Konferenzstandorten (91) gesendeten Audiosignale zu empfangen, Audiosignale für jeden empfangenden Konferenzstandort aus den empfangenen Audiosignalen auszuwählen, die ausgewählten Audiosignale gemäß Typen der empfangenden Konferenzstandorte (93) zu verarbeiten und die verarbeiteten Audiosignale zu jedem empfangenden Konferenzstandort jeweils entsprechend dem Typ des empfangenden Konferenzstandorts zu senden, wobei die empfangenden Konferenzstandorte (93) einen audiokanalbasierten empfangenden Konferenzstandort und einen audioobjektbasierten empfangenden Konferenzstandort umfassen; und

die empfangenden Konferenzstandorte (93) konfiguriert sind, die verarbeiteten Audiosignale von der Mehrpunktsteuercinheit (92) zu empfangen;

wobei dann, wenn der empfangende Konferenzstandort ein audiokanalbasierter empfangender Konferenzstandort ist, das Verarbeiten der ausgewählten Audiosignale gemäß den Typen der empfangenden Konferenzstandorte (93) Folgendes umfasst: Umsetzen der ausgewählten Audiosignale in Audiosignale, wobei die Anzahl von Kanälen jedes umgesetzten Audiosignals konsistent mit der Anzahl von Audiokanälen des empfangenden Konferenzstandorts ist; und Ausführen von Audiomischung auf den umgesetzten Audiosignalen basierend auf dem Audiokanal des empfangenden Konferenzstandorts; und

dann, wenn der empfangende Konferenzstandort ein audioobjektbasierter empfangender Konferenzstandort ist, das Verarbeiten der ausgewählten Audiosignale gemäß den Typen der empfangenden Konfetenzstandorte (93) Folgendes umfasst:

Umsetzen der ausgewählten Audiosignale in Audioobjekte gemäß einem Präsentationsmodus des empfangenden Konferenzstandorts; und Kombinieren der umgesetzten Audioobjekte in einen Audioobjektdatenstrom.

Revendications

1. Procédé pour implémenter un mixage audio, comprenant les étapes suivantes :

recevoir (101) un signal audio émis par chaque site de conférence émetteur, où les signaux audio reçus comprennent un signal audio basé sur des canaux audio émis par un site de conférence émetteur basé sur des canaux audio et un signal audio basé sur des objets audio émis par un site de conférence émetteur basé sur des objets audio ;

sélectionner (102) des signaux audio pour chaque site de conférence récepteur à partir des signaux audio reçus ;

traiter (103) les signaux audio sélectionnés selon les types des sites de conférence récepteurs, où les sites de conférence récepteurs comprennent un site de conférence récepteur basé sur des canaux audio et un site de conférence récepteur basé sur des objets audio ; et

envoyer (104) le signal audio traité à chaque site de conférence récepteur, respectivement, selon le type du site de conférence récepteur ;

où, lorsque le site de conférence récepteur est un site de conférence récepteur basé sur des canaux audio, le traitement des signaux audio sélectionnés selon les types des sites de conférence récepteurs comprend les étapes suivantes : convertir les signaux audio sélectionnés en signaux audio où le nombre de canaux de chaque signal audio converti est cohérent avec le nombre de canaux audio du site de conférence récepteur ; et effectuer, sur la base du canal audio du site de conférence récepteur, un mixage audio sur les signaux audio convertis ; et

lorsque le site de conférence récepteur est un site de conférence récepteur basé sur des objets audio, le traitement des signaux audio sélectionnés selon les types des sites de conférence récepteurs comprend les étapes suivantes : convertir les signaux audio sélectionnés en objets audio conformément à un mode de présentation du site de conférence récepteur ; et combiner les objets audio convertis en un flux d'objet audio.

2. Procédé pour implémenter un mixage audio selon la revendication 1, dans lequel la sélection du signal audio pour chaque site de conférence récepteur parmi les signaux audio reçus spécifiquement comprend les étapes suivantes :

calculer, respectivement, l'énergie du signal audio basé sur des canaux audio et/ou l'énergie du signal audio basé sur des objets audio ; et

sélectionner un signal audio en fonction de l'énergie du signal audio basé sur des canaux audio et/ou de l'énergie du signal audio basé sur des objets audio.

3. Procédé pour implémenter un mixage audio selon la revendication 2, dans lequel le calcul de l'énergie du signal audio basé sur des canaux audio comprend les étapes suivantes :

lorsque le canal audio est un monocanal, prendre l'énergie du signal audio basé sur des canaux audio comme énergie du monocanal ; et

lorsque le canal audio est un canal multiple, calculer l'énergie de chaque canal audio respectivement, et prendre l'énergie du canal audio maximum comme énergie du signal audio basé sur des canaux audio, ou prendre une valeur moyenne de l'énergie de chaque canal audio comme énergie du signal audio basé sur des canaux audio.

4. Procédé pour implémenter un mixage audio selon la revendication 2, dans lequel le calcul de l'énergie du signal audio basé sur des objets audio comprend les étapes suivantes :

calculer l'énergie absolue de l'objet audio selon une énergie absolue maximum et un rapport de l'énergie de l'objet audio sur l'énergie absolue maximum, où l'énergie absolue maximum et le rapport de l'énergie de l'objet audio sur l'énergie absolue maximum sont acheminés dans les informations auxiliaires de l'objet audio.

5. Procédé pour implémenter un mixage audio selon la revendication 1, dans lequel la conversion des signaux audio sélectionnés en signaux audio où le nombre de canaux de chaque signal audio converti est cohérent avec le nombre de canaux audio du site de conférence récepteur comprend les étapes suivantes :

juger un type des signaux audio sélectionnés ;

si les signaux audio sélectionnés sont des signaux audio basés sur des canaux audio, convertir les signaux audio basés sur des canaux audio en signaux, où le nombre de canaux de chaque signal converti est cohérent avec le nombre de canaux audio du site de conférence récepteur ;

si les signaux audio sélectionnés sont des signaux audio basés sur des objets audio, juger si les informations auxiliaires de l'objet audio acheminent un mode de lecture qui est cohérent avec le nombre de canaux audio du site de conférence récepteur ;

si les informations auxiliaires de l'objet audio acheminent un mode de lecture qui est cohérent avec le nombre de canaux audio du site de conférence récepteur, convertir, conformément à un paramètre d'un mode de lecture correspondant dans les informations auxiliaires, l'objet audio en un signal audio où le nombre de canaux du signal audio converti est cohérent avec le nombre de canaux audio du site de conférence récepteur : et

si les informations auxiliaires de l'objet audio n'acheminent pas un mode de lecture qui est cohérent avec le nombre de canaux audio du site de conférence récepteur, convertir, conformément à des informations de position de l'objet audio, l'objet audio en un signal où le nombre de canaux du signal converti est cohérent avec le nombre de canaux audio du site de conférence récepteur, où les informations de position sont acheminées dans les informations auxiliaires de l'objet audio.

6. Procédé pour implémenter un mixage audio selon la revendication 5, dans lequel la conversion, conformément aux informations de position de l'objet audio, de l'objet audio en le signal où le nombre de canaux du signal converti est cohérent avec le nombre de canaux audio du site de conférence récepteur comprend les étapes suivantes :

déterminer, conformément à une position de l'objet audio et à un emplacement d'un haut-parleur qui correspond à chaque canal audio du site de conférence récepteur, un haut-parleur qui est le plus proche de l'objet audio ; et

copier le signal audio basé sur des objets audio sur un canal audio correspondant au haut-parleur qui est le plus proche de l'objet audio.

7. Procédé pour implémenter un mixage audio selon la revendication 1, dans lequel la conversion des signaux audio sélectionnés en les objets audio conformément au mode de présentation du site de conférence récepteur comprend les étapes suivantes :

définir un mode de présentation des signaux audio sélectionnés conformément au mode de présentation du site de conférence récepteur ;

lorsque les signaux audio sélectionnés sont des signaux audio basés sur des canaux audio, convertir les signaux audio sélectionnés en objets audio conformément au mode de présentation défini ; et

lorsque les signaux audio sélectionnés sont des signaux audio basés sur des objets audio, modifier un paramètre de l'objet audio conformément au mode de présentation définir.

8. Appareil pour implémenter un mixage audio, en particulier une unité de contrôle à points multiples, dans lequel l'appareil comprend :

un module de réception (61), configuré pour recevoir un signal audio émis par chaque site de conférence émetteur, ou les signaux audio comprennent un signal audio basé sur des canaux audio émis par un site de conférence émetteur basé sur des canaux audio et un signal audio basé sur des objets audio émis par un site de conférence émetteur basé sur des objets audio ;

un module de sélection (62), configuré pour sélectionner des signaux audio pour chaque site de conférence récepteur à partir des signaux audio reçus ;

un module de traitement (63), configuré pour traiter les signaux audio sélectionnés selon les types des sites de conférence récepteurs, où les sites de conférence récepteurs comprennent un site de conférence récepteur basé sur des canaux audio et un site de conférence récepteur basé sur des objets audio ; et

un module d'envoi (64), configuré pour envoyer le signal audio traité à chaque site de conférence récepteur, respectivement, selon le type du site de conférence récepteur ; où

le module de traitement (63) comprend : une première unité de conversion (631), configurée pour convertir les signaux audio sélectionnés en signaux audio, où le nombre de canaux de chaque signal audio converti est cohérent avec le nombre de canaux audio du site de conférence récepteur ; et une unité de mixage audio (632), configurée pour effectuer, sur la base du canal audio du site de conférence récepteur, un mixage audio sur les signaux audio convertis ;

ou

le module de traitement (63) comprend : une seconde unité de conversion (633), configurée pour convertir les signaux audio sélectionnés en objets audio conformément à un mode de présentation du site de conférence récepteur ; et une unité de combinaison (634), configurée pour combiner les objets audio convertis en un flux d'objet audio.

9. Appareil pour implémenter un mixage audio selon la revendication 8, dans lequel le module de sélection (62) comprend:

une unité de calcul (621), configurée pour calculer, respectivement, l'énergie du signal audio basé sur des canaux audio et l'énergie du signal audio basé sur des objets audio ; et

une unité de sélection (622), configurée pour sélectionner des signaux audio en fonction de l'énergie du signal audio basé sur des canaux audio et de l'énergie du signal audio basé sur des objets audio.

10. Appareil pour implémenter un mixage audio selon la revendication 9, dans lequel, lorsque le module de traitement (63) comprend une première unité de conversion (631), la première unité de conversion (631) comprend :

une première sous-unité de jugement (6311), configurée pour juger un type des signaux audio sélectionnés ;

une première sous-unité de conversion (6312), configurée pour, lorsque les signaux audio sélectionnés sont des signaux audio basés sur des canaux audio, convertir les signaux audio basés sur des canaux audio en signaux où le nombre de canaux de chaque signal audio converti est cohérent avec le nombre de canaux audio du site de conférence récepteur ;

une seconde sous-unité de jugement (6313), configurée pour, lorsque les signaux audio sélectionnés sont des signaux audio basés sur des objets audio, juger si des informations auxiliaires de l'objet audio acheminent un mode de lecture qui est cohérent avec le nombre de canaux audio du site de conférence récepteur ;

une seconde sous-unité de conversion (6314), configurée pour, lorsque les informations auxiliaires de l'objet audio acheminent un mode de lecture qui est cohérent avec le nombre de canaux audio du site de conférence récepteur, convertir, conformément à un paramètre d'un mode de lecture correspondant dans les informations auxiliaires, l'objet audio en un signal audio où le nombre de canaux du signal audio converti est cohérent avec le nombre de canaux audio du site de conférence récepteur ; et

une troisième sous-unité de conversion (6315), configurée pour, lorsque les informations auxiliaires de l'objet audio n'acheminent pas un mode de lecture qui est cohérent avec le nombre de canaux audio) du site de conférence récepteur, convertir,

conformément à des informations de position de l'objet audio, l'objet audio en un signal où le nombre de canaux du signal est cohérent avec le nombre de canaux audio du site de conférence récepteur, où les informations de position sont acheminées dans les informations auxiliaires de l'objet audio.

11. Appareil pour implémenter un mixage audio selon la revendication 8, dans lequel, lorsque le module de traitement (63) comprend une seconde unité de conversion (633), la seconde unité de conversion (633) comprend :

une sous-unité de définition (6331), configurée pour définir un mode de présentation des signaux audio sélectionnés conformément au mode de présentation du site de conférence récepteur ;

une quatrième sous-unité de conversion (6332), configurée pour, lorsque les signaux audio sélectionnés sont des signaux audio basés sur des canaux audio, convertir les signaux audio sélectionnés en objets audio conformément au mode de présentation défini ; et

une sous-unité de modification (6333), configurée pour, lorsque les signaux audio sélectionnés sont des signaux audio basés sur des objets audio, modifier un paramètre de l'objet audio conformément au mode de présentation défini.

12. Système pour implémenter un mixage audio, comprenant plusieurs sites de conférence émetteurs (91), plusieurs sites de conférence récepteurs (93), et une unité de contrôle de points multiples (92), où :

les sites de conférence émetteurs (91) sont configurés pour envoyer des signaux audio à l'unité de contrôle de points multiples (92), où les signaux audio comprennent un signal audio basé sur des canaux audio émis par un site de conférence émetteur basé sur des canaux audio et un signal audio basé sur des objets audio émis par un site de conférence émetteur basé sur des objets audio ;

l'unité de contrôle de points multiples (92) est configurée pour recevoir les signaux audio émis par les sites de conférence émetteurs (91), sélectionner des signaux audio pour chaque site de conférence récepteur à partir des signaux audio reçus, traiter les signaux audio sélectionnés conformément aux types des sites de conférence récepteurs (93), et envoyer le signal audio traité à chaque site de conférence récepteur, respectivement, selon le type du site de conférence récepteur, où les sites de conférence récepteurs (93) comprennent un site de conférence récepteur basé sur des canaux audio et un site de conférence récepteur basé sur des objets audio ; et

les sites de conférence récepteurs (93) sont configurés pour recevoir les signaux audio traités depuis l'unité de contrôle de points multiples (92) ;

où, lorsque le site de conférence récepteur est un site de conférence récepteur basé sur des canaux audio, le traitement des signaux audio sélectionnés selon les types des sites de conférence récepteurs (93) comprend les étapes suivantes : convertir les signaux audio sélectionnés en signaux audio où le nombre de canaux de chaque signal audio converti est cohérent avec le nombre de canaux audio du site de conférence récepteur ; et effectuer, sur la base du canal audio du site de conférence récepteur, un mixage audio sur les signaux audio convertis , et

lorsque le site de conférence récepteur est un site de conférence récepteur basé sur des objets audio, le traitement des signaux audio sélectionnés selon les types des sites de conférence récepteurs (93) comprend les étapes suivantes : convertir les signaux audio sélectionnés en objets audio conformément à un mode de présentation du site de conférence récepteur ; et combiner les objets audio convertis en un flux d'objet audio.

Drawing