The audio/Midi/Score window |
|
|
The purpose of the audio/midi/score window [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
The audio/midi/score window helps you to convert an audio file
into a score and also helps you to transcribe a MIDI file or a
free MIDI record (without metronome) to get a more precise result
in music notation.
You can reach this window in multiple ways, according to the
type of operation you want to perform.
To convert an existing audio WAV file into a score, you can:
- Call the File, Convert an audio file into a
score... menu item.
- Open the audio file with the i, Open an audio file...
menu item or double-click on the audio file in the
document manager, and then right-click in the content of
the audio window and select the Convert into a
score... item.
- Right-click on the audio file in the document manager and
select the Convert into a score... item.
To record an audio file from a microphone and convert it into
a score, or to play the MIDI keyboard freely without metronome
and work out the score of it:
- While holding down the CTRL key, click on the red record
button in the score
window.
To convert a free MIDI file recorded without metronome, an
additional option is available in the import MIDI file dialog
box, entitled Manual adjustment. By checking this
option, the MIDI file is open in the audio/midi/score window (but
without the audio panel), which provides a more advanced tool to
adjust the conversion into a score. The ways to open a MIDI file
are multiple:
- With the File, Import MIDI file... menu
item.
- By double-clicking on a MIDI file in the document
manager.
- By right-clicking on a file in the document manager and
then selecting the Create the score option.
This window offers you two different features that must be
well understood and distinguished.
When you start from an audio WAV file (whether it already
exists or you record it from a microphone), the first part of the
work is to locate the notes played in that file.
Locating notes played in an audio file is a quite delicate
task for a computer program. You must often help the software to
make certain decisions and adjust the parameters of the
conversion. It is sometimes needed to correct one note or adjust
its exact duration. You can for instance specify where the notes
start, which increases the general precision of the conversion.
At this time, Pizzicato can handle a monodic melody (one note at
a time) correctly. Even if Pizzicato has a polyphonic recognition
tool, this tool is only helping you to transcribe, as an
important part of the work must be done by you. We will see one
example of it in this lesson.
The second part of the work is to transcribe the notes found
into a score, by taking into account the duration, rhythm and
measures used, so that the score looks readable and logical. Here
also, this window offers you a set of tools to define the
measures and beats, but the splitting into several voices and
staves as well.
To explain the features of this window, we will take several
practical examples.
Audio conversion - Example 1 - Guitar [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
Let us start with a simple melody on the guitar.
- In configuration 2 of the document manager (blue buttons
in the upper left part of Pizzicato), open the examples
and right-click on the example entitled Ex094.wav,
then select the Convert into a score menu item.
The audio/midi/score window appears as follows:

This window has the following areas (from top to bottom):
- The tool bar gives control over the main features of the
window.
- The display area of the audio file. Here you see the
audio file that we just opened, which contains a short
melody played on a real guitar. This area can be resized
by moving up or down the gray bar just below it.
- Just below this area is the audio transition area. It
will display and/or help you to modify the moments when
notes start and stop.
- The MIDI area, where the notes found in the audio file
will be displayed. You can also modify, add or delete
notes in that area. The note names are displayed in the
left part and this left part can be dragged up or down to
display a different range of pitches. It can also be
resized by moving up or down the gray bar just below it.
- The area of measures and beats, where you can adjust the
starting points of measures and the intermediate beats as
well.
- The last area displays the score in music notation.
- Below the score, you can use a scroll bar to move to
another part of the file, in case that the file needs
more space to be displayed.
Here is a description of the various controls of the tool bar:
- A menu and two buttons are used to change the horizontal
zoom. By zooming, you increase the precision of work.
- Two check boxes enable/disable the playback of MIDI (the
result of the conversion) and audio (original file).
- The icon with two small notes may be dragged and dropped
in a document of the document manager, at least for the
Pizzicato versions that enable multiple scores in a
document. The score dragged is the result of the
conversion.
- The two next buttons are Stop and Start playback (audio
and/or MIDI) used to hear the content of the working
areas.
- The original staff only makes sense when you
import a MIDI file and is used to specify which staff of
this file you are working on in the MIDI area. For an
imported audio file (as it is the case here), there is
only one choice, Audio/Midi.
- The Assign to menu is by default on Staff 1
- Voice 1, which means that the notes found in audio
and the edited notes in the MIDI area, will be
transcribed as the first rhythmic voice of the first
staff. The last choice of this menu is Other...
is used to add a new assignment:

You can define either a second voice in the same staff or
another staff. You can have up to 8 combinations on up to 3
staves to transcribe the notes from an audio file or from a
track of a MIDI file. A color can be used for each, so as to
make the work more easy in the MIDI area.
- The color of the assignment can be changed also in the
next control box.
- The Display all staves check box is used to
display all staves of the document in the lower part of
the window. When it is not checked, only the staves
resulting from the original staff of the MIDI file are
displayed. In the case of an audio file, it is unused
because there is only one audio track.
- The Options... button displays the dialog with
the audio/MIDI recognition parameters. We will go into
more details on this later in this lesson.
- The Direct Audio/Midi button starts a
conversion. The conversion is done and the transition
points are computed.
- The Detect zones button starts an analysis of
the transitions between notes. You can then modify them
before proceeding with the conversion itself.
- The Audio/Midi by zone starts a conversion that
is based on the zones existing between the transitions of
notes.
- The Quantize... button displays the quantize
dialog box of Pizzicato. When you validate, the MIDI
result will be quantized, which means adjusted on beats
and divisions of beats, so as to remove the imprecision
of the audio or of the MIDI recording. This options also
deletes the human interpretation of the music. But if the
recording is not precise, this option improves the
rhythmic aspect.
- The Keep exact timing check box is more subtle.
When you adapt the measures and beats so that they
generate a more logic and readable transcription in music
notation, if the result is played with the same tempo
value, the original performance will be transformed
according to the new measures/beats setup. To compensate
for this, by enabling this check box, Pizzicato will add
tempo variations in such a way that the original
performance will be restored.
- The Validate... button saves the resulting score
under a name you provide. The audio/midi/score window is
then closed and the resulting score is opened in the
score view of Pizzicato.
We will examine an example of the two ways to handle an
audio/Midi conversion.
- Listen to the melody, played somehow irregularly by a
beginning guitarist (me..!).
- Click the Direct Audio/Midi button. The
following result appears:

- Listen to the MIDI playback, by disabling the audio
playback.
Three notes have a wrong section in them (up one octave),
at the time references 3, 7 and 11 (in seconds). Notice that
the program displays the transition found, by red circles in
the transition area and by corresponding vertical lines in
the audio area. To improve the result, you can edit the
transitions. Most of the time, the program will find more
transitions than it should, so the most obvious correction is
to delete the transitions that do not correspond to a note.
To do that, simply double-click inside the circle of a
transition.
- For instance, double-click on the transition located in
the middle of the note of time reference 3. Pizzicato
adapts the conversion of the resulting note and displays:

- The final transition of this note as well as the starting
transition of the next note are wrongly estimated and the
result is that instead of displaying a rest, Pizzicato
finds a lower harmonic of the previous notes. Move the
two transitions (by clicking and dragging the circles) so
as to get this (you can zoom in to increase the
precision):

which removes the unwanted note in the MIDI area.
- For the note located between 7 and 8, delete the
transition in the middle of these two notes.
- Similarly for the final note, delete the two unnecessary
notes to display:

- In some cases, to have a smoother melody, you can clean
up the unnecessary transitions. The important transitions
are the ones that correspond to the real beginning of the
note. For this example, you could get this:

To edit transitions, you can:
- Move a transition by clicking and dragging it left or
right.
- Add a new transition by double-clicking where no
transition exists.
The next phase is to transcribe the notes into music notation
in a way that is readable and logic. For the moment, we did not
pay any attention to it and the staff displays the following in
the score area:

The music seems confused, even if all notes and durations are
present. This is because the notes are not synchronized (aligned)
with measures and beats. This is where the measures and beats
area takes its interest. It is located between the MIDI area and
the score area. The starting points of the measures are displayed
as blue rectangles (that also display the time signature) and the
beats are displayed as blue circles. Here are the operation you
can do in this area:
- Moving a measure by dragging its rectangle: the current
measure and all the beats and starting points of the
following measures moves accordingly. It is useful as you
will often work from left to right to adjust the measures
and beats. The beats of the previous measure are expanded
or reduced accordingly. You can avoid these behavior as
follows: by holding the SHIFT key, the previous measure
is not modified and by holding the CTRL key, only that
measure rectangle is moved.
- To delete a measure or a beat, double-click it.
- By double-clicking in a free area, a new beat is added.
- By double-clicking in a free area while holding down the
CTRL key, a new measure is added. The measure is
determined by the number of beats defined in it.
- By right-clicking a measure, you can change the type of
measure between 1, 2, 4, 8, 16 or 32. To work in 6/8,
select 8 and add 5 beats after it. If you hold down the
CTRL key during this operation, all measures are affected
by the change and if you hold down the SHIFT key, the
current measure and the following measures are all
affected.
Here is how to adjust the score in a few clicks, based on the
quarter note:
- Drag the first measure just in front of the first note.
Be careful to place it just a bit before, otherwise the
note is not taken into account and is replaced by a rest.
The score displays the following:

- Drag the second measure to the left, so that it contains
the first four notes. The beats also move to display:

The first measure is now readable as four quarter notes,
even if the intermediate beats are not exactly positioned
(this is caused by the imprecision of the recording or
conversion). If the notation was wrong because of that, you
could simply move the beats to align to the MIDI notes.
- Do the same with the next two measures so as to display:

- For the last measure, there are no more beats and
measures, so add three beats and one measure and adjust
them to get:
Once the result is fine, you can click on the Validate...
button and save it by giving it a name. Pizzicato opens it in the
score editor and the audio/Midi/score conversion is finished.
Audio conversion - Example 2 - Guitar [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
The second method to convert an audio file is to start working
on the transitions and only then convert to MIDI. The fact of
specifying the transition helps the program as it can then
consider that there is one one between each transition. Let us
take another small guitar melody to demonstrate this method.
- In configuration 2 of the document manager, open the
examples and right-click on the example entitled Ex095.wav
and select the Convert into a score... menu
item. The audio/Midi/score window opens.
- Listen to the melody and then click on Detect zones.
Pizzicato displays the following transitions that
determines the zones:

- Clean up the unnecessary transitions, then click on Audio/Midi
by zone. This button computes the notes located in
each zone. Listen to the MIDI playback. You get the
following:

- By adjusting the measures and beats as described in the
previous example, you can easily get the following:

According to the audio record and the instrument, one or the
other method will be better. Make some tests and use the more
efficient one.
Audio conversion - Example 3 - Flute [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
If you want to directly record an audio file and convert it,
you can do it by clicking on the record button
located in the score window of Pizzicato.
The audio/Midi/score window opens. Click then on the button
located in the audio area (which is empty
at the beginning). The audio record dialog opens to display:

Here is how it works:
- Click on Record audio to start recording. You
can adjust the record level with the two sliders. The
black bar graphs display the level of the signal in
green. If there is not reaction on these bar graphs,
check that the main Pizzicato audio input is well
connected to the microphone or other input where the
signal is available. Check this in the Options, Audio
setup... menu item.
- Once the audio record is done, click on Stop.
- You can record again if you need it, the previous
recording being automatically deleted if you start a new
one.
- The OK button validates the last record and
inserts it into the audio zone.
Here is an example of a flute audio record. It is example Ex096.wav
that you can open as explained in the previous examples:

Let us explain another method, more natural, that you can use
to specify the transitions of the above signal. The idea is that
you can record them in real time by clicking with the mouse.
Listen to the melody many times. In the case you record the
melody yourself it will be more easy, as you know the melody
because you played it yourself. Then the procedure is the follow
(you can do it several times until you are satisfied with it):
- Click on the
button that is located at the
beginning of the transition area (just below the audio
area).
- The playback starts and each time you click inside
the transition area, a new transition is added. Do this
for each note as well as for the end of the last note.
- Once this is done, transitions can be adjusted manually
as previously explained. Here is what it could look like
after a real time recording:

An important remark must be done. Windows very often has a latency
time for the audio sound card of a PC. This delay happens
between the moment when Pizzicato sends the audio samples to
Windows and the moment they are really played by the sound card.
Because of this, even if you click right when you hear the note,
the transition will necessarily be displayed a little bit after
the real note in the audio area. On Mac, the latency time is very
small and so does not really interfere. This delay also explains
why you hear the audio and MIDI with a slight delay when you play
them together. There are two methods to handle this problem.
The first is to shift all transitions slightly to the left so
that they are aligned with most of the audio notes. You can do
that by dragging a transition while holding down the CTRL key. If
you hold the SHIFT key, only the current and following
transitions will be moved.
The other method is to determine this delay by experience and
compensate for it in the Options, Midi play options... dialog
box, with the slider displaying MIDI delay in milliseconds
(correction for audio).
- You can also use the same principle to record the
measures and beats. Click on the
button in that area to start
recording in real time and then click inside
that area to record measures and beats.
- Click with the left button for a measure and with the
right button for the intermediate beats.
- Adjust them manually if needed and you will get for
instance:

In this case, the time signature is 2/4 (you could also
transcribe it in 4/4) and the melody starts on an eighth note
upbeat. So you must place a measure at the beginning and the next
measure must start on the second note.
Notice that you can also record the MIDI from a MIDI keyboard
directly in real time. You must use the
button located in the MIDI area to start
recording. Then you can use the tools to fit the recording into
nice music notation on the staff.
Audio conversion - Example 4 - Piano [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
Let us take another example, to illustrate the possible
correction in the MIDI area.
- Open the Ex097.wav example as explained before.
Listen to the melody, click Detect zones and
then audio/Midi by zone. You get the following:

The MIDI area provides tools for the following operations,
that will influence the resulting score in the score area (make
some tests in the above example):
- To delete a note (represented by a thick red line), click
it while holding down the CTRL key.
- To move a note, click on it and drag it horizontally in
time and/or vertically for transposition.
- By clicking on a note while holding down the SHIFT key,
you can increase or decrease its duration by shifting
horizontally.
- By clicking on a free location while holding down CTRL
and SHIFT, you can add a new note and adjust its duration
horizontally.
After removing the unnecessary transitions and adjusting the
measures and beats, the transcription of this melody will look
like this:

Audio conversion - Example 5 - Polyphony [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
Here is a more delicate example, where Pizzicato can help you,
but where an important part of the work must be done to
transcribe a score correctly. We hope to improve this function in
the future so that the manual part of the work will be reduced
progressively.
- Open the example entitled Ex098.wav and listen
to the audio track.
- Click on the Options... button, change the Maximal
polyphony value to 2 (by default it is 1, which
means that only one note will be detected at a time),
then validate by OK.
- Click on the Detect zones button. Another way
would be to record the transitions in real time.
- Adjust the position of the transitions as best as you
can, based on the peaks you see in the audio signal and
then click on the Audio/Midi by zone button. You
should get the following:

- Listen to the MIDI result and you will hear that most
notes are there but an important clearing work must still
be done. Let us take this example to illustrate how to
assign some of the notes to another rhythmic voice or
staff.
- In the Assign to menu, select the Other...
item. In the following dialog box, select staff 2,
voice 1. We need to separate the two hands of the
piano recording into two staves.
- To assign a note in the MIDI area to the selection of the
Assign to menu, you can simply click the note
with the right button. The note then takes the color of
the assignment (here in blue). For instance, click the
first four lower notes to the right, then click the Midi/Score
button and you get:

- Another method is to click with the right mouse button
and drag a rectangle to cover the notes you want to
assign to another staff or voice. Do it for the next four
notes, to the right of the blue notes.
- A third method is to click with the right mouse, then
press and hold the CTRL key and drag vertically, so as to
cover a specific range for the whole area. Do it to
assign all notes equal or lower than A3.
- If you use the tools to correct the notes in the MIDI
area, then adjust the measures and beats (you can record
them in real time, it is faster), and by comparing to the
audio version, you can reach the follow result (after
quite some work):

Importing a MIDI file [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
The basic principle to import and edit MIDI files is similar
to what has been explained here, except that there is no audio
area.
- Right-click the example entitled Ex099.mid in
the Examples folder of configuration 2 of the
document manager, then select the Create the score
menu item. In the following dialog box, check the Manual
adjustment box, which forces the display of the
audio/midi/score window, then click on Import.
- Listen to the music (beginning of a Chopin's Prelude).
- In the Original staff menu, there are two lines
for Piano, representing the two hands of the
piano. The melody is played with the right hand and the
left hand plays a series of chords made out of 3 notes
each. Select the second Piano line in the Original
staff menu and the left hand is displayed in the
MIDI area. Check Display all staves so that both
staves are displayed in the score area.
Let us use this example to show how you can separate voices.
We will split the three notes of the chords into three separate
staves.
- In the Assign to menu, select Other...
and choose staff 2.
- Select Other... again and choose staff 3.
- Click on the Midi/Score button and the two new
staves are displayed.
- If you reduce the zoom value, the beginning of the score
is displayed like this:

- To assign the central notes to the next staff, select Staff
2 - Voice 1 in the Assign to menu. Then
draw rectangles with the right mouse button, to cover the
central line of notes. They are then displayed in blue.
- Do the same for the lower notes, by selection Staff 3
- Voice 1 in the menu and transfer the notes in the
same way.
- Click on the Midi/Score button. The window must
then look like this:

Selection of an audio or MIDI section [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
In the MIDI and audio areas, you can also select an horizontal
section. This is done by clicking and dragging. The selection is
displayed with a light green background, in the audio area as
well as in the MIDI area. The conversion operations as well as
the playback and recording of audio, MIDI, transitions, measures
and beats, will only be done inside that selection. To cancel the
selection, simply click in the area.
When a selection is present, contextual menus are available
with a right click:
- In the audio area, Delete content will replace
the selected part of the signal with silence. This is
useful to remove some noise or accidental note that
should not be taken into account for the conversion.
- In the MIDI area, Delete notes will delete all
notes inside the selection.
- In the audio and MIDI areas, Remove zone will
remove the entire selected section. The section right to
the selection will be moved close to the section before
the selection.
- In the audio area, Delete transitions will
simply delete the transitions which are displayed in the
selected zone.
Conversion parameters [Beginner] [Professional] [Notation] [Composition Light] [Composition Pro] [Drums and Percussion] [Guitar] [Choir] [Keyboard] [Soloist]
Clicking the Options... button opens the following
dialog box:

You will find here the various parameters that define the way
an audio/Midi conversion is done. Some parameters may be quite
complex to explain and are related to signal processing and to
the harmonics of a note. We won't get too technical about it.
Here is an explanation of the main parameters.
The upper part of the dialog is used to save different
configurations of parameters. At the beginning, only one
configuration exists, named Default values. You can
duplicate it with the Duplicate button and then rename
it. You can create several configurations and go from one to the
other easily, at least for users who want to experiment with
these parameters. In most cases, the first configuration will
work properly. The Delete button deletes the current
configuration and the Default values button is used to
restore the default values of the parameters in the current
configuration. The Label text box is where you can
rename the configuration. Configurations are automatically saved
when you close the dialog. The current configuration is displayed
by the Setup menu and it will be used for the conversions.
- Frequency of reference A note (Hz): you can
specify the exact frequency of the A note, which is by
default 440 Hz.
- Minimal note duration (milliseconds): by
default, the notes shorter than 100 mSec (0.1 second) are
ignored, as they are in most cases produced by the
transitions of the signal. If you play a slow melody, you
can even increase this value to improve the process.
- Range: by default, notes are searched for in
three octaves (from C in second space in F clef to C two
ledger lines up in G clef). If your melody is contained
in a smaller range, you can adjust the limits, which may
improve the results.
- Maximal polyphony: by default, the analysis is
done for a monodic melody, that is, a melody with only
one note at a time. In this case, two methods are
possible and may give different results according to the
instrument and record quality. When we search for several
notes, only the method using spectral analysis may be
used (the search is made on possible frequencies found in
the signal). This method is consumes much processing
power. While searching for only one note at a time, it is
also possible to use the time analysis, where the period
of the signal is analyzed and the frequency and note are
derived. The value of the maximal polyphony is the
maximum number of notes played together, that the program
will search for.
- Analysis frame: for the spectral analysis, the
program starts with a section of 10 x 50 milliseconds
(0.5 seconds) and find the frequencies present. Then the
analysis is done again for each 50 milliseconds to refine
the real position of notes in that time frame. These
durations may be changed here for experimentation.
- Time smoothing: this is used to
"smooth" the starting points of notes so that
they are played more regularly. For instance, if you play
continuous eighth or sixteenth notes, the program may not
locate their starting points exactly each time, so that
the result is played irregularly. This option partly
solves this problem.
- Harmonics parameters: these are internal
parameters for the program, which influence the way notes
are extracted from the set of frequencies found in the
signal. As a note has several harmonics at multiples of
its frequency, a possible error is that the program
detects a note one octave too high or too low, because of
the presence of harmonics. Here is a short description of
each:
- Harmonics taken into account: the program
analyses the notes based on this number of harmonics.
- Minimal volume to force an harmonic into use:
below this level, the frequency is ignored.
- Frequency range to force an harmonic into use:
by reducing this value, the frequency range is
reduced around the note and if outside that range,
the frequency is ignored.
- Limit of H1/H2 acceptable ratio: H1 and H2
are the detected levels for harmonics 1 and 2. It may
happen that an instrument plays a note which has a
level of harmonic 2 greater than harmonic 1 (its
fundamental frequency). A value of 20 % means that
Pizzicato will take this phenomenon into account up
to the situation where the level of harmonic 2 will
be less than 5 times the level of harmonic 1.
- Volume to consider an harmonic as a possible
note: if this level is reached, the frequency
may be analyzed as a main note (harmonic 1).
- Set velocity to: set the velocity of each note
to the specified value (between 1 and 127). When this
option is disabled, Pizzicato tries to determine the
velocity of each note by computing the energy of the
note.
- Ignore rests under: a rest that is shorter than
this value is ignored, which means that the previous note
will be lengthened to fill this rest.
A portion of the analysis work is related to detecting the
presence of repeated notes or of the end of a note. The following
parameters are used for that:
- Volume under which a note repeat becomes possible:
if the level of a note decreases below this level,
(compared to its maximal peak in the attack of the note),
then a repeat (second attack of the note) may be searched
for.
- Increase of volume to trigger a note repetition:
if the volume of a note increases by this value, after it
dropped below the previous parameter, then the note is
repeated.
- Volume under which a note is terminated: if the
note volume decreases below this value (compared to its
first peak value), the note is terminated.
By clicking on OK, the parameters are saved and the
current configuration will be used for the next conversions.
The Convert button also saves the parameters and
starts a new conversion.
The Cancel button restores the parameter
configurations that were used before opening this dialog box.