Interqual Reliability

Coding Interqual reliability

We are going to calculate inter-coder reliability by re-coding 10% of the utterances for the annotations that have an inq and utt closed vocabulary. Do not code xds/any other tiers for this round. We will NOT be re-segmenting utterances or re-transcribing them.

We ran a script that left behind 10% of utterances that have xds, at random, throughout the file. Code the utterances by jumping between utterances or code nums on the grid based on speaker, make sure to hit every utterance so you don't miss any.

NEW! Listen to the context of the utterances - listen to the entire 2 minute code segment before coding.

Coding protocols for VI/TD annotation for the Input and Interaction Quality study:

Tier 1: Utterance type - the tier that tells us what type of sentence, or partial sentence, the speaker just said.

utt@Speaker

Symbolic Association

Closed vocabulary

• I - Imperatives a command, usually omits the subject. "Go get the flower."

• Q - interrogatives (Questions) asking something, ends in a question mark. "Where's the flower?"

• N - iNdirect imperative a question, ends with a question mark. The response to this question is an action “Can you touch the flower?” “Do you want to play with the flower?” (note that the normal response here is not Yes or No, it's touching the flower or playing with the flower)

• D - Declarative a statement, ends in a period. "That's a flower."

Code also incomplete sentence - single words or two word responses. Rely on prosody to distinguish between incomplete utterance types, and between declaratives and interrogatives, if no other information is available. If your syntax is declarative, and your intonation is that of a question, code an interrogative.

Do your best to decide on an utterance type. Imperatives trump everything else. If you can't decide between a question and an imperative, it's probably an indirect imperative.

Tier 2: Utterance content - the tier that tells us what kind of content (or discourse structure) is in the utterance, what kind of conversational response the speaker made.

Code conversations both between children and adults, and between several adults (both xds@C and xds@A)

inq@Speaker

Symbolic Association

Closed vocabulary

• B - Reading written material (Book) coded when someone is reading text, or singing a song. That includes lyrics humming, or singing "lalala". Note: sometimes caretakers look at books with infants but do not read the text, and sometimes they interject non-text sentences while reading! These utterances should be coded as normal, and not as reading.

  • "Beep! Went the friendly little blue truck."

  • "Billy Jean is not my lover..."

• N - Negations: code when people say "no" or "you can't". Beware, often, negations are extensions if they contain any kind of content. Negations on their own without additional content are rare, but not unheard of.

• E - Expansions: Expansions are when an adult takes a child's incorrect utterance, and "fixes" it by adding linguistic information, but no other kind of information. Expansions are the rarest type of utterance in content.

  • CHI: "Want flower!"

  • A: "You want this flower."

• X - Extensions: Extensions add other kinds of information to the conversation code responses and continuations on the same topic as Extensions code parents' elaborations on a (written) story as Extensions. Extensions are very common.

  • CHI: "Flower."

  • A: "A blue flower. Flowers grow in the garden."

• R - Repetitions: repeat what was previous said basically word for word. Repetitions do NOT include people repeating themselves, only other people. For self-repetitions, code the first instance as whatever function it has, and then use Other.

  • CHI: "Flower"

  • A: "That's a flower!"

• A - Affirmations and confirmations: used to indicate positive responses OR that the other speaker has been heard, acknowledge what the speakers are saying. Affirmations do not involve all instances of backchanneling. Code affirmations in instances of explicit approval of what the infant (or other conversation partner) said or did.

  • "Yeah." "Uh-huh." "That's right." "Good job!"

• I - Initiations: Conversational starters and topic changes. When getting someone's attention, addressing a new person, or starting a new topic.

  • "Hi! Benny!"

  • "Benny, stop!"

• O - Other: code Other for unrelated utterances, self-repetitions, and undefined utterances such as [!=laughs], interjections and backchannels.

Content hierarchy and decision tree - how to determine between category codes.

If the parent is reading a book or singing a song, simply use the B code, without further fine-grained coding, even if the song or book is providing information.

It can be difficult to differentiate between an Extensions (continuation of a conversation) and an Initiation (a topic-switch or a new conversation). If there is a sufficiently long break, unless very explicitly referring to the previous conversation, it's a new conversation and an initiation. Addressing a new conversation partner is an Initiation.

Deciding between categories:

Since what we care about is informativeness, the primary decision to make is whether the utterance adds information to the conversation on the same topic, or not. If an utterance contains both an affirmation and an extension (e.g. "Yeah, that's a grey duckie"), decide whether new information is added. if so, it is an extension.

If the utterance does not contain new information, but is conversation-supporting, and it contains some kind of verbal affirmation (e.g. "Yeah," "that's right", "good job!") code it as an affirmation.

NEW! Double code breakdown:

• if EA is informative call is E otherwise call it A.

• If ER is informative call it E otherwise call it R

• If XA is informative call it X otherwise call it A

• If XR is informative call it E otherwise call it R

• if EX adds linguistic information, code it as E, otherwise, code it as X

• if RA orivudes a verbal affirmation, code it as A, otherwise code it as R

Some examples and weird edge cases:

• "Thank you" is an affirmation. "I see" is a confirmation. Code them as A

• "Hey" is an initiation.

• Calling a name is an initiation.

• Questions that request new information additions to the conversation should be coded as X

• Many (though not all) imperatives are initiations.

Coding notes

3/7 - Held training session with RAs (WR/VW)

3/11 - Held training session with RAs (OWO/CW)

3/17 - Training session with lab manager (SS)

3/18 - Updated precise instructions for listening times