Full extraction with ms3 v2.4.0, adding 'chords' facet and metronome marks

Added metronome marks

Each movement now comes with an invisible metronome mark which naturally indicates the beat unit. The tempi roughly reflect those played by Fazıl Say in his complete recordings. The beat units have been chosen to reflect how one might conduct the music rather than the denominator of the meter (the corresponding value for which can be automatically derived).

Added chords facet

Not to be confounded with labels or chord annotations, a chord is a notational unit in which all included notes are part of the same notational layer and have the same onset and duration. Every chord has a chord_id and every note is part of a chord. These tables are used to convey score information that is not attached to a particular note, such as lyrics, staff text, dynamics and other markup.

Changes to the TSV facets

Updated filenames

The TSV files have been renamed and now come with a suffix specifying the data facet that they represent. The three facets are

• measures
• notes
• harmonies

so, for example, the file measures/01-1.tsv has been renamed to measures/01-1.measures.tsv.

Added resource descriptors

Each TSV file is now accompanied with a JSON resource descriptor following the frictionless specification for Tabular Data Resources. They can and have be used to validate the tabular data.

These files have the same names as the TSV files they describe but replacing the .tsv extension with .resource.json. For example, the file measures/01-1.tsv has the corresponding descriptor measures/01-1.resource.json.

These metadata files replace the previous csv-metadata.json files (which followed the CSVW standard) that had described the data inadequately.

Added column quarterbeats_all_endings

The quarterbeats column (also known as qstamp) that had already been available does not provide values for first or third endings, expressing the dimensions of a singular playthrough without taking into account any repeats. The newly added column quarterbeats_all_endings does not have any empty values, providing continuous positions as if each measure followed the previous one.

Renamed index column

The first column of metadata.tsv was renamed from fname to piece.

Changes to the reviewed folder

The _reviewed.mscx files compare the current set of annotation labels against those from the previous version (v2.1). They are identical, so no differences are displayed.

Wherever the MuseScore parser throws one or several warnings (e.g. because an
annotation label does not match the score very well), these warnings appear in a .warnings file. For example, the
warnings for 01-1 can be seen
in reviewed/01-1.warnings.
One goal for future versions of this dataset would be to reduce the number of warnings to the point
of addressing all of them.

Frictionless datapackage

This release is the first one that has a frictionless datapackage attached (below). It consists of

• beethoven_sonatas.zip, a ZIP file containing one TSV file per facet, that corresponds to a concatenation of the TSV files in the respective folder, that is
  • beethoven_sonatas.expanded.tsv
  • beethoven_sonatas.measures.tsv
  • beethoven_sonatas.metadata.tsv (concatenation of a single file)
  • beethoven_sonatas.notes.tsv
• beethoven_sonatas.datapackage.json, the package descriptor.

If one has the frictionless framework installed, one can use the descriptor to validate the package using the command

frictionless validate beethoven_sonatas.datapackage.json

Annotated Mozart Sonatas v2.0

Changes to harmonize with other DCML corpora

• Renamed folder scores to MS3 (7549f6a)
• Extracted facets and metadata with ms3 1.0.1 (9eb9fe3)
  • TSV files now come with the column quarterbeats, which measures in quarter notes each event's position as its distance from the beginning
  • The extracted harmony labels in the folder harmonies are expanded into feature columns by default.
  • Extracted notes now come with the columns name and octave.
  • Column volta (containing first and second endings) removed from pieces that don't have any.
  • metadata.tsv has been enriched with further columns, in particular information about each movement's dimensions,including dimensions upon unfolding repeats (for instance, last_mn has the number of measures, last_mn_unfolded the number of measures when playing all repeats)
  • The folder reviewed contains two files per movement:
    • A copy of the score where all out-of-label notes have been colored in red; additionally, modified labels (w.r.t. v1.0) are shown in these files in a diff-like manner (removed in red, added in green).
    • A copy of the harmonies TSV with six added columns that reflect the coloring of out-of-label notes ("coloring reports")
  • As long as the ms3 review has any complaints, it stores them in the file warnings.log. Currently, it is showing those labels where over 60% of the notes in the segment have been colored in red and probably need revisiting (Pull Requests welcome)
• Score updated using ms3 update (13dfb6d)
  • Files updated to MuseScore 3.6.2
  • All labels moved from the chord layer of staff 1 to the Roman Numeral Analysis layer of staff 2. This changes how they are displayed and eliminates the requirement to prepend a full stop to labels starting with a note name.
• Cadence labels now integrated with harmony labels as per DCML harmony annotation standard 2.3.0 (1c290e8)
• TSV files are automatically kept up to date using the dcml_corpus_workflow (c203595)

Changes to the content

• Made phrase annotations consistent by adding missing curly brackets. (9f10fc0)
• Introduced first and second endings at the beginning of K311-2 in order to introduce an EC label on the repetition of bar 1.
• Fixed repeat structure in da capo movements K282-2 and K331-2 for correct unfolding (b7271da..0e9f060)
• updated labels of K283-3 (f1fe032)
• corrected scores in a few places (b6aa4f1, 438acb0)

Removed mozart_loader.py

The functionality of the loader has been superseded by the ms3 parsing library.
Once installed (pip install ms3), you'll have several commands on your hands, one of which is ms3 transform. For
example, head to the folder with the dataset and type ms3 transform -N to create the concatenated note list.
ms3 transform -h will show all options.

First release: The basis for the data report

v1.0

Referenced data report