Bayesian Networks in Educational Assessment


Session III: Bayes Net with R

Duanli Yan, Diego Zapata, ETS

Russell Almond, FSU

2021 NCME Tutorial: Bayesian Networks in Educational Assessment

SESSION __ __ __ __ TOPIC __ __ __ __ __ __ __ __ __ __ __ __ __ __ PRESENTERS

Session 1 : Evidence Centered Design Diego Zapata Bayesian Networks

Session 2 : Bayes Net Applications Duanli Yan & ACED: ECD in Action Russell Almond

Session 3 : Bayes Nets with R Russell Almond & Duanli Yan

Session 4 : Refining Bayes Nets with Duanli Yan & Data Russell Almond

Discrete Partial Credit Model: A generic framework for building CPTs

Russell Almond

Conditional Probability Tables


Too many parameters to comfortably elicit

Certain cases might be rare in population \(</span> <span style="color:#000000">Very High </span> <span style="color:#000000">on </span> <span style="color:#000000"> _Skill 1_ </span> <span style="color:#000000"> and </span> <span style="color:#000000">Very Low </span> <span style="color:#000000">on </span> <span style="color:#000000"> _Skill 2_ </span> <span style="color:#000000">\)

Want to capture intuition of experts on how skills interact to generate performace.

Reduced Parameter Models

Noisy-And (Or)

All input skills needed to solve problem

Bypass parameter for Skill j , q j

Slip probability \(overall\), q 0

Probability of correct outcome


Noisy Min (Max)

Discrete IRT (2PL) model

DiBello–Samejima Models

Single parent version

Map each level of parent state to “effective theta” on IRT \(N\(0\,1\)) scale,

Now plug into Samejima graded response model to get probability of outcome

Uses standard IRT parameters, “difficulty” and “discrimination”

DiBello--Normal model uses regression model rather than graded response

Various Combination Rules

Effective Thetas for Compensatory Relationship

equally spaced normal quantiles

Effective Theta to CPT

Introduce new parameter d inc _ _ as spread between difficulties in Samejima model

b i,Full _ = b_ j _ + d_ inc /2 b j,Partial _ = b_ j _ - d_ inc /2

Conditional probability table for _ d_ inc _ _ = 1 is then…

CPTtools framework

Parent level effective thetas

Effective theta scale is a logit scale corresponds to mean 0 SD 1 in a “standard” population.

Want the effective theta values to be equally spaced on this scale

Want the marginal distribution implied by the effective thetas to be uniform \(unit of the combination operator\)

What the effective theta transformation to be effectively invertible \(this is reason to add the 1\.7 to the IRT equation\).

Equally spaced quantiles of the normal distribution

Suppose variable has M states: 0,…,M-1

Want the midpoint of the interval going from probability m/M to \(m\+1\)/M .

Solution is to map state m onto

R code: qnorm\(\(1:M\)-.5)/M)