Reliability coding Habit studies in ELAN
This page describes how to do reliability coding for Habit studies run over Zoom or in the lab
Last updated
This page describes how to do reliability coding for Habit studies run over Zoom or in the lab
Last updated
If you set up Zoom recordings according to the guidelines found here, you should have a set of 5 files for every participant. For reliability coding, you will want the ones labeled
'gvo', which should be the view with just the participant while they were doing the experiment,
and 'as', which is the shared screen and allows you to see when trials start and end
the sound file can also be useful. Zoom outputs a mp4, but ELAN needs the wav file. You can convert this in Audacity
If it gives you an error, this page may be helpful https://manual.audacityteam.org/man/importing_audio.html
Open ELAN
Select File > New, and a new window will pop up
Select "Add Media File...", navigate to the participant folder, and select the 'gvo' video, it will now appear in your selected files.
Select "Add Media File..." again and now choose the 'as' video
Select "Add Media File..." again and now choose the audio only file
Once they both appear in your selected files list, click OK, your window should look something like this
First, add two new Tier types
Go to Type > Add Type and add the Trial Type
You will see the default tier at the top, you can delete that if you want (we don't need it)
Then you should add a new Type called 'Trial', with the Stereotype "None", see below.
And then a new Type called 'Looking', which will have the Stereotype "Included In", see below.
Now you need to add two tiers:
Go to Tier > Add Tier
Add a Trial tier, which will be type Tier (see below)
Add a Looking tier, which will be type Looking, with parent Trial (see below)
You can delete the default tier if you want, we don't need it
Go to Options and Select Annotation Mode
Make sure that the Trial tier is selected, you can do this by double clicking on it. If it is selected, it should look like this, with the Trial tier at the top highlighted in red:
To the right, you have a couple of options for segmentation. I will outline two below.
If you select "One keystroke per annotation, press to mark start, release to mark end" you will be able to watch the whole video and essentially live code it. Below this is the option to "compensate for keystroke delay". About 200-300 ms is about how long it takes a human to press/release a key after making the decision to, so you can check this box to adjust accordingly.
Start at the beginning of the videos, and click the play button under the participant video
While watching the shared video on the right, hold the space key down when you see the attention getter end, and keep holding it until the attention getter starts again.
Don't worry if you make mistakes! You will check and make adjustments at the end.
Go through the whole video like this, holding down "return" when the trials are happening and releasing it when there is the attention getter.
If you select this option, you will press "return" once to start the trial, and then press it again at the end of the trial. This version can be easier if your VPN connection is a little slower, as it allows you to find the location of the start/end and just mark it that way, without watching the whole video.
This is where the audio file is particularly helpful, as you can scroll through the files by looking at the waveform of the audio. For example, I know the waveform of the attention getter is different from the waveform of the words played on trials, so I can find the general area where the trials starts and ends, and then use the advance frame by frame button (the one next to the play button) to find the exact spot where the video changes signaling the start/end of the trial
When you are done, you will have a number of bounded trials, like this:
Because we can't be perfectly accurate, you then should go back and adjust the edges accordingly.
When you select the beginning or the end, it will turn green
You can then drag it either way until it perfectly matches the beginning/end of the trial, looking at the shared screen to identify exactly when the trial starts and ends
If you select the entire trial (just by clicking somewhere on it, the whole thing will highlight first), you will also see the total length of the trial (where the arrow is located in the picture below). This number should match the "LookDisabled"-"LookEnabled" in the Habit output file for that trial.
You now want to name your trials so that they match the name of the trials in your Habit results file.
Under Options, return to "Annotation Mode"
When you double click on one of the bounded trials, you will now be able to type something into it to label it. You will make your life a lot easier by labeling things exactly as they are labeled in the Habit file.
If your trials are called "Trial" and in another column they are labeled by number, I would recommend labeling them as e.g. 'Trial_1'
After you have marked the starts and ends of every trial, you should save the file for annotation purposes. e.g. ParticipantNum_Reliability
Now, it's time for all coders who need to check their coding against the original to live code the file. Open the .eaf (created above) file for the participant whose video you will be checking and save a new version with your initials.
Go to "Options" in the menu bar and select "Segmentation mode." As outlined above, there are several options for segmentation.
For reliability coding, we should use One keystroke per annotation, press to mark start, release to mark end. This most closely resembles the coding mechanic used in Habit, where the coder presses the "5" key when the child is looking, and releases it when they look away.
If participant was run over Zoom: Go to "View>media player" and a window will display the titles of the videos that are linked to the file. You should keep the version with "gvo" in the title, and uncheck the version with "as" in the title. You will now see only the participant's video, and not the stimulus.
Both in-lab and Zoom studies: Click on the volume button on the far right of the row of controls in the top half of the window. Drag the volume all the way to zero so you will not be able to hear the stimulus.
Click into the timeline in the lower half of the screen, and use the arrow keys to toggle between tiers until the "Looking" tier is selected.
Zoom in using the scroll bar in the lower right corner until one trial takes up about half/three-quarters of the width of your timeline (more or less depending on your preference).
Drag the center double-bar down until the video window is big enough to get a good view of the baby's eyes (but make sure you can still see the timeline!)
Set the red vertical bar to the beginning of the experiment, with buffer time before the first trial. When you are ready to begin, start the video using the play button or ctrl+space.
You will know when the first trial begins when the red bar passes into the annotations on the trial tier.
During each trial, press enter/return when the participant is looking, and release it when they look away.
Annotations will appear within the bounds of each trial marked on the Trial tier, but not between trials. This means any looks you mark at the beginning or end of a trial will automatically line up with the trial boundary.
Pay attention to trial boundaries and make sure to release the key somewhere between trials, or ELAN won't create a new look on the next trial.
If annotations disappear as soon as you release the key, or they appear outside the boundaries of Trial annotations, it means the tiers are not yoked together properly.
Check that the annotations appear on the Trial tier (not the "default" tier), check their formatting with the instructions above, or check with the study leader.
When you have coded all trials in real time, it's time to clean up your code to fix any ELAN-related errors.
You can change the beginning or end of an annotation by hovering your cursor over the edge, and an arrow will appear. As you drag, the video will move as well. Drag to you desired point and release.
You can shift an entire annotation by clicking the middle and dragging.
Make sure the first look of a trial lines up exactly with the beginning of a trial annotation.
This is true most of the time, though sometimes a baby will fake out the experimenter and the trial starts when the baby is looking away.
Make sure the end of the looking annotation falls right as they look away.
Delete any extraneous looks caused by finger twitches, or merge annotations that don't have a look away in between.
Right click on the annotation in the top white bar, and options will appear
Go to "Options>Annotation mode."
Go to "Search> Find (and replace)." A window will pop up. Under "Find," click "an annotation."
An annotation on tier Looking that matches ^$, and click search at the bottom.
A list of annotations that are blank (should be all of them, though sometimes annotations that you have merged contain spaces) will appear.
Click "Replace" at the top of the box, and type "look" in the field. Click ok, and the list will disappear. Close the search window.
In the control panel, go to the Grid tab, select the Looking tier from the drop down, and check that all your annotations have been filled in with "look." If they are blank or have other values, double-click on that annotation in the grid listing and you will be able to manually change the annotation.
Save your file and notify the study leader that you have finished.
