VIHI Parallel Annotation

Introduction

Two or more people should be able to update annotations at the same time without overwriting each other's work. To achieve that, everyone should be working using their individual copy of recording files that is also stored on the shared drive. Once they are done, that copy should be added to the main VIHI dataset. We have helper programs to do these operations and these instructions will teach you how to use them.

You will need to know how to use the Terminal - how to open it, change the working directory, run a command, and read its results. You won't need to write any commands from scratch. You'll have to change them just a little bit, e.g., substitute your name for the placeholder. Other than that, you'll just need to copy, paste, and hit enter. See Using Terminal/Shell/Command Linefor a brief introduction.

At the moment, not all programs have been finished so you will also need to run git commands in the Terminal when you are done with the recording - either for the day or completely. You won't need to know how to work with git or even what it is. The instructions will list the specific commands you need to use - you'll mostly need to just copy, paste, and hit return/enter. See Git Basics: Happy git with BLabif you want to learn more about git.

Prerequisites

You should have:

• (for PC users only) git and git-bash. Go to https://gitforwindows.org/ and install it if you don't.

• Python 3. See Getting Python on your computer for instruction to check and install.

• blabpy version 0.29.1 or newer installed (see Check blabpy version below),

• BlabShare drive mounted (see ) (private page)

Setting up a computer to do parallel annotation (only once! And if you're working on an RA machine it's probably already done)

Check blabpy version

Run the following command in the Terminal:

pip show blabpy | grep Version

It will either output three dot-separated numbers like 1.17.92 or it will say WARNING: Package(s) not found: blabpy

If the version is 0.29.1 or newer, e.g., 0.29.1, 0.29.7, 0.30.0, 1.9.7, you are all set.

If the version is older, e.g., 0.29.0, 0.14.7, etc., or if the package is not found then run the following command:

pip install --upgrade blabpy

If, instead of installing/updating blabpy, this command throws errors, check that you have Python installed and install it if you don't - see Getting Python on your computer

Create a copy of the recording folder

To create a copy for you to work on, run the following command in the Terminal substituting your name, the recording ID, and email (keep the double quotes around the name).

1. Copy the following command to a text editor.

   vihi annotation start --name "First Last" --email [email protected]

2. Change the name and email to your name and email.

3. Run the result in a Terminal window.

4. Type the recording ID when prompted.

5. Wait until the program finishes working.

   • If there are errors, (1) tell or Slack Zhenya, (2) follow instructions in Fallback

6. At the end, the program will print the path to the folder with your copy of the recording files. Something like this:

   Annotation files have been copied into the following folder:
   /Volumes/Fas-Phyc-PEB-Lab/VIHI/SubjectFiles/LENA/annotations-in-progress/XX_123_456_First-Last

   Open the .eaf file in that folder and start annotating. If you accidentally closed the window, follow instructions in Starting or Continuing Annotationinstead.

If any of the steps above results in error printed out, see if the fix is described in Troubleshooting. If not, follow the Fallbackinstructions.

Fallback

If vihi annotation start doesn't work:

1. Copy the output of the command and Slack it to Zhenya.

2. Go to <BLAB SHARE>/VIHI/SubjectFiles/LENA/annotations-in-progress.

3. Let's say your name is Mike Wazowski and you are annotating recording TD_123_456. Create a folder inside the annotations-in-progress folder called TD_123_456_Mike-Wazowski_fallback.

4. Go to <BLAB SHARE>/VIHI/SubjectFiles/LENA/annotations (not the in-progress one) and copy TD/TD_123/TD_123_456/TD_123_456.eaf and the coding issues .docx file to the fallback folder you created in the previous step.

5. Annotated the copied .eaf file in ELAN, edit the .docx in Word.

Switching from a fallback folder

Once the problem with vihi annotation start is fixed and your personal in-progress annotation folder at <BLAB SHARE>/VIHI/SubjectFiles/LENA/annotations-in-progress/TD_123_456_Mike_Wazowski is created (note that the name of the folder doesn't have _fallback at the end):

• copy the .eaf and the .docx file from the fallback folder to the newly created folder overwriting the files already present there,

• follow the Saving your work instructions,

• open .eaf and .docx in the new location to check that they have definitely been overwritten,

• delete the fallback folder.

Starting or Continuing Annotation

Once you have create your individual copy of the recording files:

1. Open .../VIHI/LENA/annotations-in-progress in a file manager (Finder/Explorer).

2. Find the folder with your name and the recording ID in its name.

3. Click through to the EAF file, open in ELAN.

Saving Your Work

You should do this every time you stop annotating a file - either because you've finished completely or just for the time being.

If at any step the instructions below doesn't make sense or outputs errors: 1. Don't worry! Your work is still safe, it's just not super-duper safe. 2. Slack Zhenya! Copy the contents of the Terminal window to the message. 3. Don't wait for Zhenya to respond - you can move on to your next task

1. Make sure you saved your changes in ELAN and/or Word.

2. Open a Terminal window at the folder with the EAF file. The last folder before the actual ELAN file. You can do this by typing cd followed by a space and then dragging in and dropping the folder that contains your EAF— it'l past the file address into the terminal line. Hit enter.

3. Check that git knows who you are by running

   git config user.name && git config user.email

4. Check which files have changed:

   git status

   See if the result corresponds to what you expected - are the files that you modified and only those files listed as modified? Only XX_123_456.eaf and/or VIHI_Coding_Issues_HI_433_721.docx should be potentially listed as modified. If more files are listed, please copy the result of git status and Slack it to Zhenya and then continue to the next step. It is fine if only the .eaf file is listed as modified.

5. For each file you modified, save it by running. Use your .eaf file. If you have finished the file, say "finished + what you were annotating for."

   git commit <XX_123_456.eaf> -m "I annotated until 14:05 point"

   or

   git commit <VIHI_Coding_Issues_XX_123_456.docx> -m "I listed seven people"

   The message (text between -m " and ") doesn't have to be very detailed but try to make it somewhat informative.

6. Send a copy of your work to our central annotations repository:

   git push

For Lilli the super-checker

There are no helper utilities as of yet.

• go to the annotator/recording's folder,

• super-check,

• change user.name and user.email to yours if necessary (shouldn't be necessary on your computer),

• commit and push,

• tell Zhenya to merge the results.

For the lab technician

Troubleshooting

fatal: detected dubious ownership in repository at

This error can look something like this:

fatal: detected dubious ownership in repository at '//sox4.university.harvard.edu/Fas-Phyc-PEB-Lab/VIHI/SubjectFiles/LENA/annotations-in-progress/TD_123_456_Mike-Wazowski'

(the path can be completely different, just make sure it is within .../Fas-Phyc-PEB-Lab/VIHI/SubjectFiles/LENA)

With this error, we need to tell git that the folder in question is safe to use with the following command:

git config --global --add safe.directory '<path-to-the-folder>'

The <path-to-the-folder> will differ depending on whether the path in the error message starts with //sox4.university.harvard.edu or not. If it does, prepend %(prefix)/ to the address:

git config --global --add safe.directory '%(prefix)///sox4.university.harvard.edu/Fas-Phyc-PEB-Lab/VIHI/SubjectFiles/LENA/annotations-in-progress/TD_123_456_Mike-Wazowski'

(note that there are three forward slashes after (prefix))

If the path doesn't start with //sox4.university.harvard.edu then use the path from the error message as-is.