For Canadian practices outside Quebec
Interobserver agreement and treatment fidelity data sheets
Interobserver agreement (IOA) compares independent observers’ records. Treatment fidelity, also called procedural integrity, examines whether a defined procedure was implemented as planned. Record both questions separately: observers can agree on the same inaccurate observation, and a consistent observation does not prove the plan was implemented.
Fictional example only. Adapt this to your clinic’s own policy and professional judgment. It is not clinical, legal or regulatory advice and does not claim to meet any payer, regulator or certification-body requirement.
Last checked: 2026-10-10
Download a printable iOA and fidelity pack (PDF or Word)
Independent paired observations and approved-procedure implementation are separate questions. The blank PDF prints on US Letter paper (2 pages) and the Word document can be edited to match your clinic’s fields. Both are free to download with no sign-up.
Choose an agreement method that matches the source data
Count, trial, interval and duration IOA answer different questions. The Reed and Azulay methods paper distinguishes these algorithms [IOA1]. Before comparing records, agree on the method, operational definition, observation window, paired unit and treatment of incomplete records.
Total-count agreement compares aggregate counts. Exact count-per-interval agreement asks whether each paired interval has the same count. Trial-by-trial agreement asks whether both observers gave the same categorical score on each paired trial. Do not convert a prompt-category record into a binary score without stating what information is lost.
Swipe or scroll sideways to view all table columns.
| Method | Numerator and denominator | Limitation |
|---|---|---|
| Total-count IOA | Smaller total ÷ larger total × 100 | Matching totals can come from different events |
| Exact count-per-interval IOA | Paired intervals with equal counts ÷ valid paired intervals × 100 | The selected interval boundaries matter |
| Trial-by-trial IOA | Matching categorical scores ÷ valid paired trials × 100 | Agreement does not prove correct scoring |
Collect independent paired observations step by step
Two observers need the same definitions and synchronized windows, but independent records. Discuss disagreements after recording rather than editing both original sheets to match.
- Record the method, unit, definition version, observer references and planned observation window.
- Pair records by a shared trial or interval identifier, not by a convenient spreadsheet row order.
- Have each observer record independently, with their actual coverage and any interruption.
- Preserve both source records. Mark missing, unmatched and invalid pairs with reasons.
- Apply the selected formula to valid comparable pairs and state the number and coverage excluded.
- Review disagreement patterns and definition drift; record any resulting training or definition change separately.
Completed fictional count and trial agreement examples
Two invented observers record counts during four synchronized one-minute windows under definition C1. Both observe all four windows. The counts below demonstrate why aggregate agreement and interval agreement cannot be substituted for one another.
Swipe or scroll sideways to view all table columns.
| Window | Observer A | Observer B | Exact count match |
|---|---|---|---|
| 1 | 2 | 1 | No |
| 2 | 0 | 0 | Yes |
| 3 | 1 | 2 | No |
| 4 | 1 | 1 | Yes |
| Totals | 4 | 4 | Two of four intervals match |
Total-count IOA = 4 ÷ 4 × 100 = 100%; exact count-per-interval IOA = 2 ÷ 4 × 100 = 50%.
Missing pairs and zero totals need explicit reporting
For a separate fictional trial example, A records independent, assisted, no response and independent. B records independent, assisted, assisted and not observed. The first three are valid pairs: two match and one differs, giving 2 ÷ 3 × 100 = 66.7% trial-by-trial agreement. The fourth is excluded because B did not observe it; report three valid pairs out of four planned pairs.
If both total event counts are zero, smaller ÷ larger is 0 ÷ 0 and undefined. Do not display automatic 100% total-count agreement. Interval methods may include paired nonoccurrences under their stated formula; they can look high when almost every interval is empty. Scored-interval and unscored-interval methods use different subsets and can have a zero eligible denominator.
Compare the three-category trial example manually on the original paired record. The linked calculator’s trial mode accepts binary 0/1 scores, so it cannot reproduce that categorical comparison directly. Use it only when a justified binary outcome rule was defined in advance; do not recode categories just to fit the tool. For supported count and interval methods, select the matching method and check exclusions. Never compare counts observed for different lengths as if the windows match. A percentage is not a pass mark or evidence that the observer is certified.
The calculator provides a blank CSV, source CSV and report CSV for a spreadsheet workflow. When importing into Excel or practice-approved Google Sheets, retain pair identifiers, units and windows and keep the selected method with the report. Check excluded pairs and zero denominators after any formula change. These exports are separate from the original printable and editable document pack.
Completed fictional procedural-integrity checklist
Fidelity concerns implementation of the approved procedure [FID1]. Write observable checklist items from that procedure, record whether there was a genuine opportunity to implement each item, and avoid overlapping items that count one omission twice. The following invented checklist concerns a documentation practice exercise, not a treatment protocol.
Swipe or scroll sideways to view all table columns.
| Expected action | Opportunity | Result | Observation note |
|---|---|---|---|
| Confirm the active recording-definition version | Yes | Completed | Recorder read version DOC1 |
| Enter actual observation start and stop | Yes | Completed | Both times entered |
| Distinguish additional prompt from independent response | Yes | Not completed | Two prompted rows labelled independent |
| Record interruption when one occurs | No | Not applicable | No interruption in this exercise |
| Retain source rows for the reviewer | Yes | Completed | Original rows preserved |
| Summary | Four applicable observed items | 3 ÷ 4 = 75% | One not-applicable item; no unreviewed items |
The percentage only summarizes this invented checklist. Item importance is not equal in every procedure; a safety-critical omission requires separate review.
Use the sheet, tools and clinical record together
Use fictional or deidentified information in the public tools. Keep identifiable observations and completed forms in the practice-approved clinical record under the clinic’s access, retention and amendment policies.
A public worksheet helps prepare and inspect a recording format. It does not authorize a treatment, decide mastery, replace consent or turn a draft into a reviewed clinical record. A clinician should confirm definitions, denominators and interpretation before the team uses the form.
TherapyCRM serves English-language practices in Canada outside Quebec. It is practice management software with a clinical record, not a physician EMR, and has no EMR certification. It has no telehealth or video visits and does not submit insurance or government claims, including Ontario Autism Program claims.
Sources and scope of this original resource
The blank record and every worked entry are original TherapyCRM examples. Sources explain the professional topic; they do not endorse this template. School resources and research studies require interpretation by a qualified clinician for a real clinic. No standardized assessment, publisher form or licensed guideline is reproduced.
Source references
- [IOA1] Reed and Azulay: Calculating Interobserver Agreement. 2011 primary methods paper discussing count, trial, interval and duration agreement algorithms and their limitations. This guide uses original arithmetic examples and does not distribute the authors’ spreadsheet.
- [FID1] IRIS: Monitoring Fidelity of Implementation. Educational background on observing implementation and documenting steps. Checked 10 October 2026. No published intervention or fidelity checklist is copied.
Frequently asked questions
What is IOA in ABA?
Interobserver agreement describes how independently collected records compare under a named method. Report the method, valid paired denominator, windows and excluded observations.
Is IOA the same as treatment fidelity?
No. IOA compares observations; fidelity compares implementation with an approved procedure. They need separate records and interpretations.
Can total-count IOA be 100% while events differ?
Yes. Equal aggregate counts can occur in different intervals. The completed example has 100% total-count agreement and 50% exact count-per-interval agreement.
What do I do when both total counts are zero?
Total-count division is undefined at 0 ÷ 0. Report the zero-event window and method limitation instead of assigning automatic 100% agreement.
Should a missing paired trial count as disagreement?
Preserve the missing state and disclose the exclusion. Do not silently convert it to a scored response or disagreement; follow the prewritten analysis rule.
Does a fidelity percentage establish safe or effective care?
No. It summarizes the particular observed checklist and denominator. It does not validate the plan, replace safety review or establish a treatment outcome.