Adding/Editing Tiers/Controlled Vocabularies

You may do this if you want to annotate a new type of information not already captured by the ACLEW annotation scheme.

ELAN terms

Controlled vocabulary (CV) - a set of values that a segment can take. It is a property of the tier. In ELAN, when you click on a segment, there is a dropdown menu with all the possible values. You can't directly create a new segment with a value that isn't in the vocabulary. If a tier didn't use a CV and it was added later, values not from the CV will be preserved. For example, tier "lex" uses a CV that is also called "lex" that has two values: "W" and "0" for 1+ words and 0 words respectively.

Creating a New Controlled Vocabulary

So that all EAFs use the same CV definitions, we store CVs in .ecv files in the bergelsonlab/public-files GitHub repo. When we need to create a new CV, we first create it in one file using ELAN, export it and upload it to GitHub, and then refer to that file in all EAFs.

First file

1. Create your working copy of the recording folder using VIHI Parallel Annotation.

2. Open the EAF in ELAN.

3. Create a new controlled vocabulary(ies)

   1. Go to "Edit Controlled Vocabularies" (Ctrl-Shift-C).

   2. Type new name and description. For consistency with other CVs, use three lower-case letters for the name.

   3. Click "Add".

   4. In the lower half, add possible values and their descriptions one by one.

4. Add tiers

   1. Create a new tier type that uses your new CV. Type -> New Tier Type (Ctrl-Shift-T). (note: the steps below are for annotation tiers that will be children of other tiers)

      1. Use the same name you used for the CV, optionally capitalize it.

      2. For "Stereotype", select "Symbolic Assocation".

      3. Select your CV in "Use Controlled Vocabulary".

      4. Make sure "Time alignable" is not checked.

   2. Create new tiers. (note: the steps below are for annotation tiers that will be children of other tiers)

      1. Go to Tier -> Add New Tier... (Ctrl-T).

      2. Select the parent tier. It doesn't have to be the participant tier, it is often one of its descendants.

      3. Select the tier type you created in the previous step.

      4. Fill in "Participant". This should be the same as for tha parent tier.

      5. Fill in the name in the following format: <lower-case-cv-name>@<PARTICIPANT>.

   3. Go back to the "Edit Controlled Vocabularies" if you need to add more entries or edit existing ones.

5. Once you are satisfied, commit you annotation-in-progress copy and push to the main repo. See VIHI Parallel Annotation

6. Move the CV from the EAF to GitHub.

   1. Export the CV to a file:

      1. Go to "Edit Controlled Vocabularies" (Ctrl-Shift-C).

         1. Click "Export .ecv".

         2. Choose only the CV(s) that you added.

         3. Save anywhere. Use a descriptive name.

   2. Upload the file to GitHub at bergelsonlab/public-files.

   3. Click on the file in the repo, then click "Raw". Once the page reloads and shows just the text content of the file (no GitHub UI), copy the URL from the address bar.

   4. Add as external CV:

      1. Go to "Edit Controlled Vocabularies" (Ctrl-Shift-C).

      2. Click "External CV".

      3. Paste the URL you copied.

      4. ELAN will ask what to do with the CVs that are already in the file. Choose "Replace" for each of the new CVs. If you see a CV that isn't one of yours, that means you exported some CVs you shouldn't have. git restore the EAF and start again from "Export the CV to a file:".

   5. Save, commit, push.

Second and further files

1. Add the .ecv file as "External CV". The steps are the same as above, except there won't be any CVs to ovewrite.

2. Add tiers. The steps are the same as above except you won't be able to edit the CV.

3. Edit the .ecv file on GitHub:

   1. The file is in XML format but it should be nevertheless simple to edit it keeping the formatting.

   2. Use https://jsonformatter.org/xml-validator to check that the result is still valid XML before committing.

   3. Commit and push. Or just commit if you used GitHub web UI to edit the file.

   4. Once committed, test that the changes have propagated to the EAF file. There are a few things that can create a delay.

      1. Go the raw.githubusercontent.com... URL of the .ecv file. Sometimes, it will take GitHub a minute to update the file.

      2. Completely close ELAN, then reopen the file.

(for the lab technician) Adding new tiers

• Add/check reference to the external CV file.

• Add/check linguistic type that uses that CV.

• For each participant than needs this descendant tier, add one.

• For each transcription, create an empty segment in the added tier.

See:

• README.md and add-tiers.py in one_time_scripts.

• This commit that added two new ters.

• Module blabpy.eaf, specifically add_* functions.