TECHNICAL FIELD
[0001] The present invention relates to an acoustic space creation apparatus.
BACKGROUND ART
[0002] There is known a musical sound generating apparatus that can automatically create
musical sound data according to a motion of a person when the person is moving in
front of a camera, and play the musical sound data. For example,
Japanese Patent No. 3643829 discloses a configuration of a musical sound generating apparatus that reacts to
motion of a person as a subject of a camera, creates musical sound data based on a
position and a variation amount of the motion, and output it as a sound.
According to the musical sound generating apparatus, a performer does not need to
acquire knowledge and skills to operate an apparatus that automatically plays music
and play a musical instrument, and the performer can easily perform an improvisational
performance simply by performing the motion in front of a camera.
CITATION LIST
PATENT LITERATURE
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED
[0004] As described above, there is known an apparatus outputting a sound that reacts to
a motion of a subject. If there is an apparatus that can recognize the motion of the
subject from a viewpoint different from that of the prior art and play a new musical
sound generated by this, the musical sound generated corresponding to the motion of
the subject can be further enjoyed. In view of such a point, it is a main object of
the present invention to provide an apparatus that recognizes a state of a motion
of an object from a viewpoint different from that of the prior art and plays a new
musical sound generated from this such that the musical sound generated corresponding
to the state of the motion of the object (for example, a motion of a performer) can
be further enjoyed.
[0005] In the musical sound generating apparatus described in
Japanese Patent No. 3643829 described above, it is necessary both to specify a position of the motion on the
subject of the camera and to calculate the variation amount of the motion of the subject.
Accordingly, the above-described musical sound generating apparatus has a possibility
that a burden of data processing in performance becomes large. In view of such a point,
it is also an object of the present invention to provide an apparatus that can more
easily perform an improvisational performance based on a motion state of an object.
SOLUTIONS TO THE PROBLEMS
[0006] The present invention focuses on simple motions of "moving" and "stopping," and is
conceived for a purpose of converting temporal senses/sensations (intervals) of various
kinds of motions such as physical expression, breathing method, "pause," "pause,"
and ON/OFF of things into a sound or a syllable. Then, the present invention provides
an apparatus that mainly focuses on repetition of "MOTION (movement)" and "STOP" of
an object and generates and organizes a melody (melody) and a syllable according to
a specific musical scale, and an acoustic space based on the repetition.
[0007] Thus, the present invention provides an acoustic space creation apparatus that includes
a camera, a light source, an input unit, a processing unit, and a plurality of speakers.
The camera captures an object. The light source irradiates the object. The input unit
acquires images from the camera. The processing unit selects a sound to be generated
corresponding to a frame of the video. The plurality of speakers output a sound selected
by the processing unit. The processing unit compares a light amount in the frame with
a light amount in a frame immediately before the frame and determines whether the
object is in a motion state or a stop state based on a predetermined threshold for
light amount. When the object is determined to be in a motion state, the processing
unit generates a motion detection signal, selects a musical pitch to be output in
accordance with a pre-stored musical structure program, allocates one of the plurality
of speakers to each sound of the selected musical pitch in accordance with a pre-stored
spatial structure program, and causes the allocated speaker to output the sound. When
the motion state consecutively occurs a first predetermined number of times during
a first unit time, the number of times of the consecutive occurrences for the first
predetermined number of times reaches a second predetermined number of times, the
object is determined to be in a motion state after having been in a stop state during
a second unit time or greater, and a specific speaker among the plurality of speakers
is allocated to a predetermined sound, the processing unit causes the specific speaker
to generate a sine wave in accordance with the musical structure program. When the
motion state consecutively occurs the first predetermined number of times during the
first unit time, the number of times of the consecutive occurrences for the first
predetermined number of times reaches a third predetermined number of times, the object
is determined to be in a motion state after having been in a stop state during the
second unit time or greater, and the specific speaker is allocated to the selected
sound, the processing unit causes the specific speaker to output a first reverberation
sound in accordance with the musical structure program. When the motion state consecutively
occurs a fourth predetermined number of times during the first unit time, the processing
unit causes the speaker to output a second reverberation sound in a predetermined
order in accordance with the musical structure program.
[0008] The present invention also provides an acoustic space creation apparatus that includes
a sensor, an input unit, a processing unit, and a plurality of speakers. The sensor
is capable of detecting a state of an object. The input unit acquires signals from
the sensor. The processing unit selects a sound to be generated corresponding to the
signal. The plurality of speakers output a sound selected by the processing unit.
The processing unit compares the signal with a signal immediately before the signal
and determines whether the object is in a motion state or a stop state based on a
predetermined threshold for signal. When the object is determined to be in a motion
state, the processing unit generates a motion detection signal, selects a musical
pitch to be output in accordance with a pre-stored musical structure program, allocates
one of the plurality of speakers to each sound of the selected musical pitch in accordance
with a pre-stored spatial structure program, and causes the allocated speaker to output
the sound. When the motion state consecutively occurs a first predetermined number
of times during a first unit time, the number of times of the consecutive occurrences
for the first predetermined number of times reaches a second predetermined number
of times, the object is determined to be in a motion state after having been in a
stop state during a second unit time or greater, and a specific speaker among the
plurality of speakers is allocated to a predetermined sound, the processing unit causes
the specific speaker to generate a sine wave in accordance with the musical structure
program. When the motion state consecutively occurs the first predetermined number
of times during the first unit time, the number of times of the consecutive occurrences
for the first predetermined number of times reaches a third predetermined number of
times, the object is determined to be in a motion state after having been in a stop
state during the second unit time or greater, and the specific speaker is allocated
to the selected sound, the processing unit causes the specific speaker to output a
first reverberation sound in accordance with the musical structure program. When the
motion state consecutively occurs a fourth predetermined number of times during the
first unit time, the processing unit causes the speaker to output a second reverberation
sound in a predetermined order in accordance with the musical structure program.
EFFECTS OF THE INVENTION
[0009] The acoustic space creation apparatus of the present invention determines each of
motion state and stop state based on the motion of the object (for example, subject),
selects a sound in accordance with, for example, a frequency, a cycle, and/or the
number of times of each state, and outputs the selected sound as the musical sound.
Consequently, with the present invention, it is possible to easily perform a musical
sound corresponding to an aspect of the motion of the object and easily provide an
acoustic space that performs such a musical sound. Furthermore, the present invention
allows performing the musical sound and/or the acoustics of the new syllable generated
based on the repetition of motion and stop of the object.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010]
FIGS. 1A and 1B are schematic views illustrating one example of an acoustic space
creation apparatus according to an embodiment, FIG. 1A is a drawing viewed from a
front side, and FIG. 1B is a drawing viewed from above.
FIG. 2 is a block diagram illustrating a main part of the acoustic space creation
apparatus in FIGS. 1A and 1B.
FIG. 3 is a drawing illustrating one example of a use state of the acoustic space
creation apparatus in FIGS. 1A and 1B.
FIG. 4 is a flowchart illustrating one example of an operation of the acoustic space
creation apparatus in FIGS. 1A and 1B.
FIG. 5 is a flowchart illustrating processing of a processing unit when a sound of
a predetermined musical pitch is output.
FIG. 6 is a drawing illustrating an audio file of musical pitches.
FIG. 7 is a drawing for describing one example of the operation of the processing
unit.
FIG. 8 is a flowchart illustrating generation processing of a sine wave.
FIG. 9 is a drawing for describing one example of the operation of the processing
unit.
FIG. 10 is a flowchart illustrating output processing of a first reverberation sound.
FIG. 11 is a drawing for describing one example of the operation of the processing
unit.
FIG. 12 is a flowchart illustrating the output processing of a second reverberation
sound.
FIG. 13 is a drawing for describing one example of the operation of the processing
unit.
DESCRIPTION OF THE EMBODIMENTS
[0011] The following describes an embodiment of the present invention by referring to accompanied
drawings. However, the present invention is not limited to this. In the drawings,
to describe the embodiment, the scale is changed and expressed as appropriate, such
as describing by enlarging or emphasizing a portion. FIGS. 1A and 1B are a drawing
illustrating an outline on one example of a configuration of an acoustic space creation
apparatus 100 according to the embodiment, FIG. 1A is a drawing viewed from a front
side, and FIG. 1B is a drawing viewed from above. FIG. 1A illustrates a state where
a wall portion 11b on a front side is transmitted, and FIG. 1B illustrates a state
where a ceiling portion 11c is transmitted.
[0012] The acoustic space creation apparatus 100 according to the embodiment is an apparatus
that generates a melody and a syllable by a specific musical scale and creates an
acoustic space 10 performing the melody and the syllable. In the embodiment, the acoustic
space 10 is a space for performing music.
[0013] In the embodiment, the acoustic space 10 is an internal space of a structure 11.
The structure 11 is a structural body in a shape of a container and is formed to be
hollow and movable. For example, as illustrated in FIGS. 1A and 1B, the structure
11 is constituted by including a floor portion 11a that is approximately square in
plan view, wall portions 11b that stand up from each side of the floor portion 11a,
and a ceiling portion 11c arranged above the floor portion 11a so as to cover the
above-described internal space. The acoustic space 10 is a space defined by the floor
portion 11a, the wall portions 11b, and the ceiling portion 11c of the structure 11.
The acoustic space 10 is not limited to the internal space of the structural body
in a shape of a container like the above-described structure 11, and all the space
where the sound can propagate is applicable. That is, the acoustic space 10 may be,
for example, an internal space and an underground space of a building such as a concert
hall and an event hall. The acoustic space 10 is not limited to a closed space, and
may be, for example, a space on an outdoor stage and a space on a ground surface.
In this case, a camera 20, a light source 30, a table 40, a plurality of speakers
50, and the like, which will be described later, are installed, for example, on the
outdoor stage or on the ground surface.
[0014] The acoustic space creation apparatus 100 includes the camera 20, the light source
30, the table 40, the plurality of speakers 50, and an information processing apparatus
60 (see FIG. 2). While the camera 20, the light source 30, and the table 40 are installed
inside the acoustic space 10, they may be installed outside the acoustic space 10.
For example, the camera 20 and the like may be installed in a place away from the
acoustic space 10.
[0015] The camera 20 is a device that can capture a motion of a subject (an object) (namely,
hands of a user U) H (see FIG. 3). The camera 20 is installed in the proximity of
the ceiling portion 11c in a state so as to face downward. The camera 20 captures
the subject H from above and images a moving image of the subject H. The camera 20
is arranged in a central portion in the acoustic space 10 in plan view. The camera
20 is supported in the air, for example, is supported by a support metal fitting (not
illustrated) extending in a horizontal direction from the wall portions 11b. The camera
20 is not limited to the above-described configuration, and a direction of the camera
20, an installation place, an installation method, and the like can be conveniently
set, for example, the camera 20 may be installed to face upward or in the horizontal
direction or may be installed in a state being fixed to the ceiling portion 11c or
may be installed in a state of being suspended from the ceiling portion 11c.
[0016] The camera 20 consecutively images the subject H and transmits the captured images
to the information processing apparatus 60. Specifically, the camera 20 captures the
subject H by a predetermined frame rate and inputs it to an input unit 61 (see FIG.
2) of the information processing apparatus 60 as image data for each frame. The camera
20 is connected to the information processing apparatus 60 so as to be capable of
data communication, and the image data is transmitted to the input unit 61 from the
camera 20 via wired communication (for example, USB or Ethernet (registered trademark))
or wireless communication (for example, various kinds of radio wave communication
or Internet). The frame rate of the camera 20 is set corresponding to a speed of the
motion of the subject H and performance of the information processing apparatus 60.
In the embodiment, the frame rate of the camera 20 is set to 40 fps (frames per second).
In this case, the camera 20 captures the subject H at a frequency of 40 times per
second and transmits frame images acquired at a frequency of 40 sheets per second
to the information processing apparatus 60. The frame rate of the camera 20 is not
limited to 40 fps and may be set to, for example, 25 fps, 30 fps, 50 fps, 60 fps,
or the like.
[0017] The light source 30 is an instrument, device, or the like that emits light for irradiating
the subject H. The light source 30 is arranged in the proximity of the ceiling portion
11c in the central portion in plan view, inside the acoustic space 10. The light source
30 is, for example, an LED spotlight. The spotlight is illumination that concentratedly
illuminates a part of the acoustic space 10. The light source 30 is installed in a
state of facing downward and emits a light L directly downward. Thus, the light L
is projected to a region R (see FIG. 3) in a part of an upper surface 41 of the table
40. The light source 30 is installed integrally with the camera 20. The light source
30 is not limited to the above-described configuration and may be, for example, an
arc lamp, an incandescent bulb, a fluorescent lamp, sunlight, or the like, or may
be one that uniformly illuminates a wide area including the periphery of the table
40. The light source 30 can be conveniently set with respect to a direction of the
emitted light L, an installation place, an installation method, and the like and,
for example, may be installed so as to emit the light L upward or horizontally, or
may be installed separately from the camera 20.
[0018] The table 40 is installed on a floor portion 11a below the light source 30. A height
of the table 40 is set, for example, to a height corresponding to a position of the
waist of the user (the performer) U in an upright posture ((see FIG. 3)). The table
40 is installed approximately in the center of the surface of the floor portion 11a,
and has a rectangular parallelepiped shape with a height of 90 cm (cm) and respective
longitudinal and horizontal lengths of 45 cm. The upper surface 41 of the table 40
has a planar shape and is arranged so as to include the entire region R irradiated
by the emission light L of the light source 30. The table 40 is not limited to the
above-described configuration, and the shape, the size, the arrangement, and the like
can be changed as necessary. In the acoustic space creation apparatus 100, it is optional
whether or not to locate such a table 40.
[0019] As described above, the light L is emitted directly downward from the light source
30. The light L emitted from the light source 30 irradiates, for example, the circular
region R on the upper surface 41 of the table 40 when it proceeds downward. At this
time, an approximately conical space having the light source 30 as the apex and the
upper surface 41 of the table 40 as the bottom surface becomes a state of being partially
illuminated by the light L, in the acoustic space 10 set to be dark as a whole. Such
an approximately conical space illuminated by the light L of the light source 30 is
referred to as a light irradiation space S (see FIG. 3).
[0020] Five speakers 50 are installed in the acoustic space 10. These five speakers 50 output
the sound based on music data generated by the information processing apparatus 60.
These five speakers 50, which includes a first speaker 51, a second speaker 52, a
third speaker 53, a fourth speaker 54, and a fifth speaker 55, respectively have sound
emitting portions 51a, 52a, 55a that emit a sound in a predetermined direction.
[0021] The fifth speaker 55 is arranged in the central portion in plan view and on the upper
side in the acoustic space 10. The fifth speaker 55 is installed on an upper surface
of the camera 20 and is installed with the emitting portion 55a facing upward.
[0022] On the other hand, the first speaker 51, the second speaker 52, the third speaker
53, and the fourth speaker 54 are arranged in the bottom portion and arranged so as
to be equally spaced on an identical circumference centered on the fifth speaker 55,
in the acoustic space 10. The first to the fourth speakers 51 to 54 are each installed
at four corners of the approximately square floor portion 11a. The first to the fourth
speakers 51 to 54 are installed having the sound emitting portions 51a, 52a, 55a,
respectively directed toward the center side of the acoustic space 10 while being
tilted upward by about 5 degrees to 25 degrees with respect to the horizontal direction
so as to emit sounds slightly upward.
[0023] The plurality of speakers 50 included in the acoustic space creation apparatus 100
are not limited to the configuration of the above-described first to fifth speakers
51 to 55. That is, an installation count of the speaker 51 and the like, respective
arrangements of the speaker 51 and the like, and directions of the sound emitting
portion 51a and the like inside the acoustic space 10 can be changed as necessary.
Specifically, for example, a count of speakers included in the acoustic space creation
apparatus 100 may be two or more and four or less or may be six or more. All the speakers
51 to 55 constituting the five speakers 50 may be arranged, for example, in the upper
portion or may be arranged in the bottom portion or may be arranged so as to surround
the central portion, in the acoustic space 10. For example, the fifth speaker 55 may
be installed to be separated from the camera 20 in a state where the sound emitting
portion 55a is directed downward so as to emit the sound downward and the first to
fourth speakers 51 and the like may be installed so as to emit the sound in the horizontal
direction. While all the first to fifth speakers 51 and the like are speakers having
the identical configuration, some or all of them may be speakers having different
configurations.
[0024] The information processing apparatus 60 is configured by, for example, a computer.
The information processing apparatus 60 is communicatively connected with each of
the first to fifth speakers 51 and the like and the camera 20 via wired or wireless
communication. The information processing apparatus 60 acquires video of the camera
20 and causes the first to fifth speakers 51 and the like to emit a predetermined
sound. While the information processing apparatus 60 is installed outside the acoustic
space 10, it may be installed inside the acoustic space 10. FIG. 2 is a block diagram
illustrating a main part of the acoustic space creation apparatus 100. As illustrated
in FIG. 2, the information processing apparatus 60 has the input unit 61, a processing
unit 62, and a storage unit 63.
[0025] The input unit 61 acquires the video of the camera 20. The input unit 61 acquires
a plurality of frames imaged by the camera 20 as the image data. The processing unit
62 performs predetermined processing based on the input image data, selects a musical
pitch corresponding to the motion state of the subject H, and causes the speakers
50 to output each sound of the selected musical pitch. The data processing and the
like in the processing unit 62 will be described later. The processing unit 62 is
achieved by the configuration including, for example, a CPU. The storage unit 63 stores
the image data input from the camera 20, data generated by the processing unit 62
and the like. The storage unit 63 also stores a program for executing the processing
of the processing unit 62, a musical structure program and a spatial structure program,
which will be described later. The storage unit 63 is achieved by, for example, a
memory, a hard disk, and the like.
[0026] Next, a use method of the acoustic space creation apparatus 100 will be described.
FIG. 3 is a drawing illustrating one example of a use state of the acoustic space
creation apparatus 100. As illustrated in FIG. 3, the user U stands on a back side
of the table 40 and performs the motion of moving and stopping the hands H in a state
of entering the hands H in the light irradiation space S. The acoustic space creation
apparatus 100 is used in this way. At this time, the user U puts both the left hand
and the right hand H or one hand H in the light irradiation space S, performs the
motion of, for example, moving the hands H vertically or horizontally, rotating the
hands H, moving the fingers, or expanding and closing the palms of the hands H, and
temporarily stops the motion of the hands H. Such a series of motions of the hands
H is captured by the camera 20. Then, the sounds generated corresponding to the motions
of the hands H are output from the speakers 50.
[0027] Subsequently, an operation of the acoustic space creation apparatus 100 will be described.
FIG. 4 is a flowchart illustrating one example of the operation of the acoustic space
creation apparatus 100. In the following, the operation of the acoustic space creation
apparatus 100 will be described according to the flowchart in FIG. 4.
[0028] First, the moving image of the subject (the object) H is imaged by the camera 20
(Step S01). The camera 20 transmits the captured image data to the information processing
apparatus 60 (Step S02). For example, in the use state indicated in FIG. 3, the camera
20 consecutively captures the hands H inside the light irradiation space S at the
predetermined frame rate and inputs each frame as the image data to the input unit
61 of the information processing apparatus 60.
[0029] When the image data for each frame (frame image) is input into the input unit 61,
the processing unit 62 executes the predetermined processing and a predetermined operation
based on the image data (Step S03). Then, musical sounds or the like are output from
the speakers 50 (Step S04). The sound output from the speakers 50 has a melody (melody)
and a syllable composed of the sounds of a predetermined musical pitch selected by
the processing unit 62.
[0030] By satisfying a predetermined condition, the sound output from the speakers 50 includes
a sine wave (sine wave) sound and a reverberation sound in addition to the sound of
such a melody or a syllable. The conditions where the sine wave sound is output from
the speakers 50 and the conditions where the reverberation sound is output from the
speakers 50 and the like will be described later.
[0031] By the operations of Step S01 to Step S04 of the acoustic space creation apparatus
100, a musical sound automatically composed corresponding to the motion of the subject
H is performed from the speakers 50, and thus, the acoustic space 10 is created.
[0032] Subsequently, the contents of the processing of the processing unit 62 at Step S03
will be specifically described. FIG. 5 is a flowchart illustrating the processing
of the processing unit 62 when the sound of a predetermined musical pitch is output.
The processing of the processing unit 62 is, for example, automatically executed based
on the program stored in the storage unit 63.
[0033] As illustrated in FIG. 5, first, the processing unit 62 acquires the frame image
(Step S11). Next, the processing unit 62 compares a light amount of the image data
in the frame with the light amount of the image data in a frame acquired immediately
before the frame (Step S12). While, at Step S12, the light amount of the whole region
in the frame image is compared, instead of this, the light amount of a part of a predetermined
region in the frame image may be compared.
[0034] Subsequently, it is determined whether or not the difference in the light amount
is equal to or more than a threshold (Step S13). At Step S13, the processing unit
62 determines whether the difference between the light amount of the image data in
the frame and the light amount of the image data in the frame acquired immediately
before the frame is equal to or more than the threshold or less than the threshold.
The threshold is a value of a predetermined light amount, is set preliminarily, and
is stored in the storage unit 63.
[0035] When the above-described difference of the light amount is equal to or more than
the threshold (when "YES" at Step S13), the processing unit 62 determines that the
subject H is in a state of "motion" (movement) (Step S14). Then, when the subject
H is determined to be in a "motion" state, the processing unit 62 generates one motion
detection signal (Step S15).
[0036] On the other hand, when the above-described difference of the light amount is determined
to be less than the threshold (when "NO" at Step S13), the processing unit 62 determines
that the subject H is in a "STOP" (stop) state (Step S24). When the subject H is determined
to be in a stop state, the processing unit 62 may generate one rest/reverberation
signal (Step S25).
[0037] When one motion detection signal is generated at Step S15, the processing unit 62
selects one musical pitch of an output sound based on the preliminarily set and stored
musical structure program (Step S16). At Step S16, based on the musical structure
program, for example, the musical pitch of the sound to be output is selected from
a file having data indicated in FIG. 6. In this case, one musical pitch is selected
from data (A1 to A42) of the 42 musical pitches described in FIG. 6. The musical pitch
selected at Step S16 may be selected from, for example, a recording material such
as an acoustic noise, instead of the data (A1 to A42) of the 42 musical pitches described
in FIG. 6. At Step S16, a plurality of musical pitches among the data (A1 to A42)
of the 42 musical pitches described in FIG. 6 may be selected at a time.
[0038] Following Step S16, the processing unit 62 allocates a speaker to the sound of the
selected musical pitch by the preliminarily set and stored spatial structure program
(Step S17). At Step S17, based on the spatial structure program, any one of the five
speakers 51 to 55 is allocated to each sound of the selected musical pitch. When a
plurality of musical pitches are selected at a time at Step S16, at Step S17, any
one of the five speakers 51 to 55 is allocated to each of the selected plurality of
musical pitches.
[0039] FIG. 6 is a drawing illustrating the audio file of the musical pitch. In FIG. 6,
data, for example, regarding the 42 musical pitches are illustrated. In FIG. 6, A1
to A42 are serial numbers of the 42 musical pitches. These 42 musical pitches are
selectable musical pitches at Step S16. Musical Instrument Digital Interface (MIDI)
number is also called a note number or a note number and is a numerical value representing
a sound pitch and a sound range in MIDI. A simultaneous sound number is a count of
sounds that can be output at a time and is also a value corresponding to a count of
layers. An appearance frequency of each musical pitch when the motion detection signal
occurs 127 times indicates how many times each pitch is selected when the motion detection
signal occurs 127 times. An output frequency of SP1/SP2/SP3/SP4/SP5 when the motion
detection signal occurs 20 times indicates how many times each of a first speaker
51 (SP1), a second speaker 52 (SP2), a third speaker 53 (SP3), a fourth speaker 54
(SP4), and a fifth speaker 55 (SP5) is allocated when the motion detection signal
occurs 20 times and the sound of the predetermined musical pitch is selected 20 times.
[0040] Numerical values regarding the appearance frequency of each musical pitch and an
allocation frequency to each speaker of the sound of each musical pitch are preliminarily
set. At Step S16 and Step S17, selection of the musical pitch and allocation to the
speakers 50 are automatically performed according to the numerical values related
to these appearance frequency and allocation frequency.
[0041] Referring again to FIG. 5, the processing unit 62 causes the allocated speakers 50
to output the sound of the predetermined musical pitch (Step S18). At Step S18, the
sound of the selected musical pitch is output from the allocated speakers 50. When
the speakers 50 output a continuous sound, the sound may be output from the speakers
50 so as to move up, down, left, and right inside the acoustic space 10, by, for example,
changing the speaker 51 and the like allocated to each sound constituting the continuous
sound (for example, allocating speaker 51 and the like different for each sound constituting
the above-described continuous sound).
[0042] FIG. 7 is a drawing for describing one example of an operation of the processing
unit 62. As illustrated also in FIG. 7, by the above-described processing of the processing
unit 62, when the subject H is in a motion state, the sound of the selected musical
pitch is output from the speakers 50 based on the musical structure program. On the
other hand, when the subject H is in a stop state, the speakers 50 become a rest state
where the speakers 50 do not output the sound temporarily, and as a result, the acoustic
space 10 becomes in a state of being silent or emitting the reverberation. Accordingly,
when the subject H continuously keeps a motion state, the sound of the predetermined
musical pitch is continuously output, and thus, the sound as a texture is generated.
On the other hand, when the subject H appropriately sandwiches a stop state during
a motion state, a pause between the motion and the motion is expressed by the sound
and the reverberation of the sound can be heard. As a result, the acoustic space 10
where the melody (melody) and the syllable are performed is created.
[0043] In the processing at Step S03 described above, the processing unit 62 causes the
speakers 50 to generate a sine wave under a predetermined condition. This sine wave
is output in a synthesized state with the waveform of the sound of the above-described
predetermined musical pitch. Thus, the processing of the processing unit 62 related
to generation of such sine wave will be described.
[0044] FIG. 8 is a flowchart illustrating generation processing of the sine wave in the
processing unit 62. The processing unit 62 executes the following processing. As illustrated
in FIG. 8, the processing unit 62 determines whether or not the motion detection signal
has occurred consecutively a first predetermined number of times during a first unit
time (Step S31). That is, the processing unit 62 determines whether or not a motion
state has occurred consecutively the first predetermined number of times during the
first unit time. When it is determined to have occurred (when "YES" at Step S31),
the processing unit 62 determines whether or not the number of times of the consecutive
occurrences for the first predetermined number of times at Step S31 has reached a
second predetermined number of times (Step S32). When it is determined to have reached
(when "YES" at Step S32), the processing unit 62 determines whether or not the motion
detection signal has occurred after having been in a stop state during a second unit
time or greater (Step S33). Then, when it is determined to be in a motion state after
having been in a stop state during a second unit time or greater (when "YES" at Step
S33), furthermore, when the specific speaker 51 or the like among the plurality of
speakers 50 is allocated to the predetermined sound (when "YES" at Step S34), the
processing unit 62 causes the specific speaker 51 or the like to generate the sine
wave (Step S35). In the above-described generation processing of the sine wave, the
determination of the stop state may be recognized by non-occurrence of the motion
detection signal or may be recognized by detecting the rest/reverberation signal.
[0045] Specifically, for example, it is as follows. FIG. 9 is a drawing for describing one
example of the operation of the processing unit 62. The first unit time is preliminarily
set to, for example, 180 ms, the first predetermined number of times is preliminarily
set to, for example, 1 time to 30 times, the second unit time is preliminarily set
to, for example, 500 ms, and the second predetermined number of times is preliminarily
set to, for example, 10 times, respectively. These setting values are stored in the
storage unit 63. As illustrated in FIG. 9, when detecting the motion detection signal
after the motion detection signal consecutively occurred 1 to 30 times during 180
ms or less, and a sum of the cycles reached 10 times, furthermore, the stop state
continued for 500 ms or more, the processing unit 62 allocates, for example, the fifth
speaker 55 and causes the fifth speaker 55 to generate a first sine wave for 60 seconds
to 90 seconds. When the sound of the musical pitch selected by the occurrence of the
motion detection signal at Step S33 has been the sound of any musical pitch of the
data of 42 musical pitches in FIG. 6, for example, A19 to A25, the processing unit
62 causes a second sine wave to be generated, in addition to the first sine wave.
The second sine wave is a sine wave where a third sine wave and a fourth sine wave,
which will be described later, are synthesized by a sound volume curve of 10 seconds.
Here, the third sine wave is a sine wave with a frequency two octaves higher than
the musical pitch selected by the occurrence of the motion detection signal at Step
S33. The fourth sine wave is a sine wave where a frequency of 0.5 Hz to 11 Hz is added
to the third sine wave. The second sine wave is output from, for example, the fifth
speaker 55 for 10 seconds. In the fourth sine wave, when the numerical value is changed,
it reaches a reach value by using 1500 ms to 4000 ms time complementation.
[0046] Furthermore, in the processing at Step S03 described above, the processing unit 62
causes the speakers 50 to output a first reverberation sound, under a predetermined
condition. Thus, subsequently, the processing of the processing unit 62 related to
the output of the first reverberation sound will be described.
[0047] FIG.10 is a flowchart illustrating output processing of the first reverberation sound.
The processing unit 62 executes the following processing. As illustrated in FIG. 10,
the processing unit 62 determines whether or not the motion detection signa has consecutively
occurred the first predetermined number of times during the first unit time (Step
S41). When it is determined to have occurred (when "YES" at Step S41), the processing
unit 62 determines whether or not the number of times of the consecutive occurrences
for the first predetermined number of times at Step S41 has reached a third predetermined
number of times (Step S42). When it is determined to have reached (when "YES" at Step
S42), subsequently, the processing unit 62 determines whether or not the motion detection
signal has occurred after having been in a stop state during the second unit time
or greater (Step S43). Then, when it is determined to be in a motion state after having
been in a stop state during the second unit time or greater (when "YES" at Step S43),
furthermore, when the specific speaker among the plurality of speakers 50 is allocated
to the selected sound (when "YES" at Step S44), the processing unit 62 causes the
specific speaker 50 to output the first reverberation sound (Step S45).
[0048] A specific description is given, for example, as follows. FIG. 11 is a drawing for
describing one example of the operation of the processing unit 62. The first unit
time, the first predetermined number of times, and the second unit time are preliminarily
set to, for example, the above-described values. The third predetermined number of
times is set to, for example, 3 times to 5 times. These setting values are stored
in the storage unit 63. As illustrated in FIG. 11, the processing unit 62 counts the
cycle where the motion detection signal consecutively occurs 1 time to 30 times during
180 processing unit 62 detects the motion detection signal after having been in a
stop state for 500 ms or greater, the processing unit 62 allocates, for example, the
fifth speaker 55 and causes the fifth speaker 55 to output the first reverberation
sound for 60 seconds to 90 seconds. At this time, when the speaker allocated with
respect to the sound of the selected musical pitch based on the motion detection signal
after having been in a stop state for 500 ms or greater, which is described above,
is the fifth speaker 55, the first reverberation sound is output from the fifth speaker
55 by being synthesized with the sound of the above-described musical pitch.
[0049] Further, in the processing at Step S03 described above, the processing unit 62 causes
the speakers 50 to output the second reverberation sound under a predetermined condition.
Thus, subsequently, the processing of the processing unit 62 related to the output
of the second reverberation sound will be described.
[0050] FIG. 12 is a flowchart illustrating the output processing of the second reverberation
sound. The processing unit 62 executes the following processing. As illustrated in
FIG. 12, the processing unit 62 determines whether or not the motion detection signal
has consecutively occurred a fourth predetermined number of times during the first
unit time (Step S51). When it is determined to have occurred (when "YES" at Step S51),
and, furthermore, the specific speakers among the plurality of speakers 50 are allocated
with respect to the selected sound (when "YES" at Step S52), the processing unit 62
causes the specific speakers 50 to output the second reverberation sound (Step S53).
[0051] A specific description is given, for example, as follows. FIG. 13 is a drawing fir
describing one example of the operation of the processing unit 62. The first unit
time is preliminarily set to, for example, the above-described values. The fourth
predetermined number of times is set to, for example, 30 times. These setting values
are stored in the storage unit 63. As illustrated in FIG. 13, when the motion detection
signal consecutively occurs 30 times during 180 ms or less, the processing unit 62
causes the speakers 50 to output the second reverberation sound. After synthesizing
the second reverberation sound, for example, with the sounds of the musical pitches
of A1 to A18, A70 to A79 among the data of 42 musical pitches described in FIG. 6,
the processing unit 62 causes the second reverberation sound to be output for 3 seconds
to 10 seconds. The processing unit 62 causes the second reverberation sound to be
output from, for example, the first speaker 51 and the second speaker 52 and also
causes the second reverberation sound to be output from the third speaker 53 and the
fourth speaker 54 after one second of acoustic movement.
[0052] As described above, the processing unit 62 causes the speakers 50 to output the sound
of the predetermined musical pitch by the processing of Step S11 to Step S18. The
processing unit 62 causes the speakers 50 to generate the sine wave by the processing
of Step S31 to Step S35. Further, the processing unit 62 causes the speakers 50 to
output the first reverberation sound by the processing of Step S41 to Step S45. Furthermore,
the processing unit 62 causes the speakers 50 to output the second reverberation sound
by the processing of Step S51 to Step S53.
[0053] As described above, by the motion detection signal generated by the processing unit
62 that determines the repetition of "MOTION/STOP" (variations of motion/stop time)
through the input unit 61 (the sensor input), the acoustic space creation apparatus
100 creates the acoustic space 10 via the musical structure program and the spatial
structure program implemented in the storage unit 63. Then, the acoustic space creation
apparatus 100 determines each of motion state and stop state based on the motion of
the subject H, selects the sound corresponding to the frequency, the cycle, the number
of times, and the like of each state, and outputs the selected sound as the musical
sound. Consequently, the acoustic space creation apparatus 100 can easily perform
the musical sound corresponding to the aspect of the motion of the object, and thus,
can easily provide the acoustic space 10 performing such musical sound. The acoustic
space creation apparatus 100 can perform the musical sound and the acoustics of the
new syllable generated based on the repetitive motion between the motion and the stop
of the object. Under the predetermined conditions, the acoustic space 10 that performs
the unique musical sound including the sound generated by synthesizing the sine waves,
the first reverberation sound, the second reverberation sound, and the like can be
provided.
[0054] In the acoustic space creation apparatus 100, since the light L is irradiated with
respect to the subject H in motion, a user U can perform or appreciate the improvisational
music while watching the motion of the subject H irradiated by the light L. This allows
the acoustic space creation apparatus 100 to provide the acoustic space 10 including
a new entertainment feature where the user U can enjoy the improvisational music together
with a visual change. Then, since the acoustic space creation apparatus 100 allows
the user U to feel or listen to the music related to body breathing such as a physical
expression, yoga, and welfare generated from the motion state and the stop state,
the acoustic space creation apparatus 100 not only allows the user U to simply feel
or listen to the music, but also can contribute to improvement of spirit of the user
U.
[0055] Subsequently, a modification of the acoustic space creation apparatus 100 according
to the above-described embodiment will be described. While the above-described acoustic
space creation apparatus 100 has a configuration including the camera 20 capturing
the subject and the light source 30 irradiating the subject H, it may have a configuration
including a sensor capable of detecting the state of the object instead of both the
camera 20 and the light source 30. Thus, in the following, a configuration of an acoustic
space creation apparatus according to the modification will be specifically described.
[0056] In the configuration of the acoustic space creation apparatus according to the above-described
modification, as the sensor capable of detecting the state of the object, for example,
a distance sensor capable of detecting a distance from the object or the like, is
applicable. The camera 20 is also one of the sensor capable of detecting the motion
of the object.
[0057] Operations of an input unit and a processing unit of the acoustic space creation
apparatus according to such modification is similar to the operation of the input
unit 61 and the processing unit 62, which are described above. However, the input
unit according to the modification consecutively acquires, for example, a signal corresponding
to a distance from the sensor to the object from the sensor. The processing unit of
the modification compares the acquired signal with the signal acquired immediately
before and determines that the object is in a motion state when the difference is
equal to or more than a threshold. The threshold is a value related to the signal
and is preliminarily set. The subsequent processing of the processing unit is similar
to that of the above-described processing unit 62. That is, the modification includes
the sensor capable of detecting the state of the object, the input unit acquiring
the signal from the sensor, the processing unit selecting a sound to be generated
corresponding to the above-described signal, and the plurality of speakers 50 outputting
the sound selected by the processing uni. The processing unit compares the signal
with a signal immediately before the signal, determines whether the object is in a
motion state or a stop state based on the predetermined threshold, which is described
above, for the signal, generates the motion detection signal when it is determined
to be in a motion state, selects a musical pitch to be output in accordance with the
pre-stored musical structure program, allocates one of the plurality of speakers 50
to each sound of the selected musical pitch in accordance with the pre-stored spatial
structure program, and causes the allocated speaker to output the sound. The other
configurations of the acoustic space creation apparatus according to the modification
is similar to those of the acoustic space creation apparatus 100 according to the
above-described embodiment. The processing unit of the acoustic space creation apparatus
according to the modification causes the plurality of speakers 50 to output the sine
wave, the first reverberation sound, and the second reverberation sound corresponding
to the frequency, the cycle, the number of times, and the like of each state, in addition
to the sound of the above-described musical pitch.
[0058] According to such modification, the acoustic space creation apparatus can be achieved
with a simpler configuration. Consequently, for example, it is also possible to constitute
the acoustic space creation apparatus with a compact PC with a video camera function.
As a result, it becomes easy to apply the acoustic space creation apparatus to a music
teaching material for children and the like.
[0059] In addition to the above-described modification, the acoustic space creation apparatus
100 can also have the following configuration. Specifically, for example, by determining
a state of stepping a grid where a sensor is mounted and a state of taking a foot
off the grid or determining ON/OFF of a switch of a buzzer, the processing unit may
determine whether the object is in a motion state or a stop state based on a predetermined
threshold for the signal. That is, for example, by using a sensor capable of detecting
contact with the foot, signals different between when the foot is in contact with
the grid and when the foot is away from the grid (for example, 1 for being in contact
with the grid and 0 for being away from the grid) are output from the sensor to the
processing unit, or by using a sensor capable of detecting an ON state of the buzzer,
signals different between when the switch of the buzzer is ON and when the switch
of the buzzer is OFF (for example, 1 for ON and 0 for OFF) are output from the sensor
to the processing unit. Then, the processing unit compares the signal acquired from
the sensor with the immediately preceding signal and determines whether or not the
difference is equal to or more than a threshold (for example, whether or not it is
equal to or more than 1). Thus, the processing unit determines whether the object
is in a motion state (a state with change in motion) or a stop state (a state with
no change in motion) (for example, when the above difference is 1, it is determined
to be in motion state, and when the above difference is 0, it is determined to be
a stop state). Then, when the object is determined to be in a motion state, the processing
unit generates the motion detection signal, selects the musical pitch to be output
in accordance with the pre-stored musical structure program, allocates one of a plurality
of speakers to each sound of the selected musical pitch in accordance with the pre-stored
spatial structure program, and causes the allocated speaker to output the sound. In
addition to the sound of the above-described musical pitch, the processing unit causes
the plurality of speakers 50 to output the sine wave, the first reverberation sound,
and the second reverberation sound corresponding to the frequency, the cycle, the
number of times, and the like of each state.
[0060] In the acoustic space creation apparatus 100, in input voice information (for example,
voice input into a microphone), the processing unit may determine whether or not the
object is in a motion state or a stop state by determining presence /absence of an
attack (namely, a resolutely strong sound, a start of a sound, and a rise of a sound)
based on a predetermined threshold for signal. That is, for example, the acoustic
space creation apparatus 100 may include a microphone as a voice recognition device
for picking up the voice of the object, and the processing unit may compare the voice
signal acquired from the microphone as a sensor with the voice signal acquired immediately
before, determine whether or not the difference (for example, the difference of a
sound volume) is equal to or more than a predetermined threshold, and determine whether
the object is in a motion state or a stop state.
[0061] While the embodiment and the modification of the present invention have been described
above, the technical scope of the present invention is not limited to the above-described
embodiments. For example, the object the motion of which is detected by a sensor such
as a camera is not limited to the hands H and, for example, may be feet or a whole
body of a person, may be a thing and the like that the person holds (for example,
a bar), and may be a shadow of a leaf, a wave, and the like. The plurality of speakers
50 that the acoustic space creation apparatus 100 includes may be configured by a
plurality of stop speakers 51 or the like as the embodiment, and may be a headphone,
earphones for both ears, and the like including a pair of speakers that directly output
the sound to both ears of the user U. While the acoustic space 10 that the acoustic
space creation apparatus 100 creates is the actual space, it may be a virtual space.
That is, for example, when the headphone or the earphones for both ears are applied
as the speakers 50 of the acoustic space creation apparatus 100, the acoustic space
creation apparatus 100 provides a virtual acoustic space with respect to the user
U mounting the speakers 50. In the embodiment and the modification described above,
while the sound is generated corresponding to the result of the processing such as
determining a motion state (motion) or a stop state (stop) for the state of the object,
which is performed by using the sensor capable of detecting the motion of the object
(the subject), here, the motion of the object (the subject) is not limited to the
motion of the real object. That is, the acoustic space creation apparatus of the present
invention may generate the sound corresponding to, for example, the processing such
as determining a motion state (motion) and a stop state (stop) for the motion of the
object at a specific fictitious place in virtual reality (VR) or in a game (on an
OpenGL (registered mark) drawing and the like). The acoustic space creation apparatus
100 may include the storage unit 63 storing a plurality of musical structure programs
having algorithms where respective different conditions (a threshold and the like)
are set and may execute the processing at Step S16 described above by appropriately
selecting from the plurality of musical structure programs or simultaneously executing
the plurality of musical structure programs. In this case, the plurality of musical
structure programs are stored in the storage unit 63, for example, in a state of being
divided into respective different layers. This ensures appropriate selection of the
musical structure program corresponding to the usage environment and a count of implementation
locations of the acoustic space creation apparatus 100 and facilitated improvement
and addition of the musical structure program.
[0062] The acoustic space creation apparatus of the present invention may be achieved by,
for example, a configuration of a system where the multiple speakers 51 and the like
are randomly installed at branches of a tree in the natural environment, or a configuration
of a product where a plurality of ultra-compact speaker 51 and the like, and sensors
are set on a flat surface (for example, a picture, a book, or a wall portion (for
example, a square wall portion with a length and a width of 10 cm, respectively, or
5 m, respectively)), or a configuration of an environmental system where sensors,
and the speaker 51 and the like are implemented in a plurality of street lights that
are also the light source 30. The acoustic space creation apparatus of the present
invention may be one that constitutes a complex improvisational ensemble device/musical
instrument. The acoustic space creation apparatus in this case may have a configuration,
for example, that performs the processing such as, for the 42 numbers (A1 to A42)
to which the respective musical pitch files described in FIG. 6 are allocated, changing/allocating
to other recording material, or associating the operation of the other musical structure
program (layer) implemented in the storage unit 63. That is, for example, at Step
S16 described above, the acoustic space creation apparatus may have a configuration
that performs the processing such as, when the musical pitch number "A2" is selected,
playing a noise recording material and a sound collection/output of a microphone in
real time inside the acoustic space, and, after that, when the musical pitch number
"A2" or other number is selected once or multiple times, turning off the sound collection/output
of the microphone, and, simultaneously, causing the other musical structure program/layer
to operate. While the acoustic space creation apparatus 100 of the above-described
embodiment includes the plurality of speakers 50, a count of the speaker 51 and the
like may be one.