Annotation Introduction | Bergelson Lab @ Harvard

File structure:

The SEEDLingS files live on the BLab share (see New RA Instructions for how to connect to the server).
Annotated files can be found in Subject Files.
- Subject files are organized by subject number
  - baby 01 through baby 46
  - note that babies 05 and 24 are dropped
  - also note that 17_06 does not have audio, and 22_09 does not have video
- With subfolders for each month of the study
  - month 6 through month 18
- Within those, subfolders for data from home or lab visits.
  - Home visits occurred every month from 6 months to 17 months.
  - Lab visits occurred every even month from 6 months to 18 months.
Naming conventions for files:
- XX_XX is always subject number_month
- So, 01_08 is baby 01, month 8.
Files in Subject Files should be the most up-to-date version.
- There shouldn't ever be extra files or old versions.
Files should be worked on in Working Files.
- See Checking Process for how to make a Working File.

Coding folder:

The Audio_Annotation and Video_Annotation folders are the only ones that matter.
- e.g. in Subject_Files/40/40_16/Home_Visit/Coding/Audio_Annotation/
Some relevant files
- Audio files have a .cha extension
- Video files have a .opf extension
- There should be a document called Audio or Video Coding Issues in each Annotation folder.
Other files that may be in this folder include subregion or personal information files, in .csv format, and possibly a folder called "error" or "old_chas"
- You should only need to use these items in specific circumstances where you've been told by lab staff to address something in one of these places.

Analysis folder:

This is where the Basic Levels live.
Basic Levels are .csv files with all of the words and codes from the file in spreadsheet format.
Basic Levels should have audio_sparse_code or video_sparse_code in the file name.

Processing folder:

Audio contains the .wav file (and also maybe a scrubbed.wav)
Video contains the .mp4 file (and also maybe a scrubbed.mp4)
These should be at the bottom of the list of files in the folder.
They should never be moved or altered.

codes look like: object &=utterance_presence_SPEAKER_annotationid e.g. ball &=d_y_MOT_0xbf4b1f e.g. book &=r_n_FAT_0xa4g3j5

Object: a concrete, imageable noun. Coded if infant-directed or if said in close proximity to the infant.

Utterance type: a code that tells us what type of sentence the object word occurs in.

declarative (d): a statement, ends in a period.
- "That's a flower."
question (q): asking something, ends in a question mark.
- "Where's the flower?"
imperative (i): a command, usually omits the subject.
- "Go get the flower."
reading (r): coded when someone is reading text. Note: sometimes caretakers look at books with infants but do not read the text, and sometimes they interject non-text sentences while reading!
- "Beep! Went the friendly little blue truck."
singing (s): coded when someone is singing a song or speaking/chanting tunefully.
- "Twinkle, twinkle, little star."
non-utterance (n): a word in isolation or a less-than-3-word noun phrase
- "A flower!"

Object Presence: is the object present at the time of the utterance (or within 5 seconds)?

yes (y)
no (n)

Speaker: a three letter code that tells us who is speaking. MUST be consistent with past codes used for that subject. In each subject's primary folder, there should be a Speaker Codes document that can help you figure out what a 3 letter code means, or decipher who's who. There should also be notes in Audio/Video Coding Issues about which speakers appear in the file. Speaker codes are different for anything that happens in unison, which could be mom and dad singing together, or a parent saying something at the same time as a toy. Check the Speaker Codes page to figure out what speaker code to use.

Annotation ID (do not change, not a code): a unique identifier to distinguish each instance

Videos are coded in DataVyu.

The video content varies by subject and month, but there will be some combination of a tripod view (right side in the example below), baby cams (on the sides of a hat that they wear--both left frames in the example below) and sometimes a parent cam.

In a separate window, there's the annotation spreadsheet with all of the coded words. Each word is in its own cell with a cell number (also called an ordinal), onset time, offset time, and the codes in parentheses.

If you were to add a new cell (Enter on the number pad), the template would look like this, ready to be filled in:

Watch 20 minutes of a video

Navigate to this file:

Fas-Phyc-PEB-Lab/Seedlings/New_RA_Training_Materials/Intro_files/03_08_sparse_code.opf

If you get a pop-up message when the file opens, it is probably because the video file is missing. Click "Add Data"

In the finder window that comes up, navigate through the following directories, starting with Fas-Phyc-PEB-Lab.

Seedlings
- Subject_Files
  - 03
    03_08
    Home_Visit
    Processing
    Video_Files
    03_08_video.mp4

Start at the beginning and watch 20 minutes, making notes about any questions that come up for you. Get used to using the number pad controls to start and stop the video. Make sure you can also make the video start at a certain word, jog, and jump back by 5 seconds.

Audios are coded in CLAN, which is a lot less pretty than DataVyu.

Ways to navigate in CLAN:

Esc + 4 allows you to hide tiers. If you want to clean up the CLAN scenery, you can hide tiers that start with %xdb. After you hit Esc + 4, a command box will open at the bottom of your file. Type “h” for hide, hit Enter, then type “%xdb” and hit Enter again.
F6 plays the audio from the beginning of whatever line your cursor is on.
F4 plays just the isolated line of audio for wherever your cursor is.
Esc + a allows you to see timestamp of the lines. Timestamps are irregularly spliced. Lines are not all the same length.
Cmd (or Ctrl for PC) + f allows you to search for character strings in the file. Automatically sets to wraparound search, which searches the file from the point where you start the search, goes to the end, and then wraps back to the beginning. Also automatically sets to being not case sensitive. Turn on case sensitive if, for example, you want to find MOT, the speaker, and not _mot_orcycle.

Audios are annotated in the following way:

6 & 7 months coded in their entirety
8 to 13 month files, 4 one-hour subregions coded (top 4 of 5)
14-17 month files, 3 one-hour subregions coded (top 3 of 5)

Files contain SKIPS and SILENCES

A skip is a 10+ minute section where there are no codeable words
- A car ride where nobody is talking to the baby
- Baby is left alone for a bit
- Vest is off
A silence was detected in the file during audio processing and is likely a nap
- Silences were added programmatically, while skips were added by hand
Both skips and silences should have been made up elsewhere in the file (in lowest ranked subregions as "make up time" or elsewhere as "extra time"
Any time that was made up (accidentally) over the amount of time that was supposed to be made up was labeled "surplus coding"

Open up this file: Fas-Phyc-PEB-Lab/Seedlings/New_RA_Training_Materials/Intro_files/audio/18_11_sparse_code.cha

Open Audio Coding Issues ("coding_issues.docx") and the Basic Level ("sparse_code.csv") from the same directory. Read through Audio Coding Issues.

In the .cha file, use Cmd or Ctrl + F (see the previous tab) to find "subregion". This should bring you to the beginning of Subregion 1.

Listen for approximately 20 minutes (of your time, don't worry about exactly how much time has elapsed in the file). Get used to using the keyboard shortcuts for viewing timestamps, hiding tiers, and playing the file continuously and line-by-line.

Spend a little bit of time getting to know how the .cha file compares to the Basic Level (the .csv). What information is retained in the export? How is it organized? What information is added? This will help you be able to detect if something is ever wrong with your basic level.