Seedlings Project Goals

Description of Seedlings and how they (barely) work

General notes:

• pay attention to the branches that things are on, it is a mess (sorry again)

• BergelsonLab can have private repos but not SeedlingsBabylab, so no PI (private information) on Seedlings, and make sure that the repo in BergelsonLab is private before putting PI things on it

Listened time

Goal of the project

Get the amount of listened time in each cha file so that we can figure out if all the files have been consistently annotated.

Retrieving the current listened time

Use the script in /Volumes/Fas-Phyc-PEB-Lab/Duke/Seedlings/Scripts_and_Apps/Github/seedlings/annot_distr called recap_regions_listen_time_compute.py (usage here -- README has not been updated since surplus was added, a lot of scripts that were there before I (GB) got here, sorry about the mess)

Checking the output

The basic processing pipeline is:

• running the script (lab tech)

• checking the output (lab manager)

  • if errors in output (pointed out by lab manager), modify script and re-run (lab tech)

  • if incoherences in cha files, fix (lab manager) an re-run script (lab tech)

Reliability

See here

Samples were taken from lena recordings from different corpora and annotated by the lab that recorded it and another lab. The idea is to measure how much they agree with each other.

Annotation format

We use the pyannote library to store the annotations. From there, we can use the built-in pyannote metrics to compute what we need.

Detection of speech

The first thing coders should agree on is when there is speech and when there is not. pyannote detection measures see code for detection accuracy and kappa computation

For detection, we computed:

• %true pos, %false pos, %true neg, %false neg

• precision, recall, f1

• accuracy (pe), kappa

Identification of speaker

Each segment of speech is labeled with a speaker by each coder. The second thing coders should agree on is the identity of this speaker -- at least, for now, whether it is a female, a male, the target child or some other child. See code for how speakers are concatenated into categories (CHI-target child, FA-female adult, MA-male adult, OC-other child)

For identification, we computed:

• identification error rate with the raw as referent, with the rely as referent

• %match, %false alarm, %miss

• accuracy, kappa

More detailed annotation

CHI tiers

VCM, LEX, MWU annotations per category: basic reliability, v similar to speaker id reliability; requires those annotations to be read in in the first place, which they are not right now

Other tiers

XDS annotations: multi-category multi-coder reliability, ask Middy for what, Stephan for how (?)

Tests

Build basic eaf files for which reliability values are known and on which to run the metrics and check that values are correct

TODO

• read XDS, VCM, LEX and MWU annotations in pyannote format

• change spkr id metrics so that they compute the same values but for VCM, LEX and MWU

• implement reliability for XDS -- ask Stephan

• create test files+CI

Babble corpus

See here

HV

Four repos, HV creation and HV Elika and HV Sarah and templates

Vital records

Birth records

See here

Death records

See here for issue filed about how to handle death records

Lab passwords

See

blabr

See here

R library containing functions useful to the lab. Missing functions recently identified were: renaming participants in a coherent way (leading 0 and so on), merge w/ CDI (Charlotte working on that)