ACED.scores {CPTtools} | R Documentation |
ACED (Adaptive Content with Evidence-Based Diagnosis; Shute, Hansen and Almond, 2008) is a Bayes net based assessment system which featured: (a) adaptive item selection and (b) extended feedback for incorrect items. This data contains both item level and pretest/posttest data from a field trial of the ACED system.
data("ACED")
ACED contains 3 data.frame
objects and one explanatory variable.
ACED.scores
is data frame with 230 observations on 74
variables. These are mostly high-level scores from the Bayesian
network.
Cond_code
a numeric vector giving the experimental
condition for this student, see also Cond
Seq
a factor describing whether the sequence of items
was Linear
or Adaptive
FB
a factor describing whether the feedback for
incorrect items was Extended
or AccuracyOnly
All_Items
a numeric vector giving the number of items in ACED
Correct
a numeric vector giving the number of items the student got correct
Incorr
a numeric vector giving the number of items the student got incorrect
Remain
a numeric vector giving the number of items not reached or skipped
ElapTime
a numeric vector giving the total time spent on ACED
The next group of columns give “scores” for each of the nodes in
the Bayesian network. Each node has four scores, and the columns are
names p
nodeScoreType where node is
replaced by one of the codes in ACED.allSkills
.
p
nodeH
a numeric vector giving the probability node is in the high state
p
nodeM
a numeric vector giving the probability node is in the medium state
p
nodeL
a numeric vector giving the probability node is in the low state
EAP
nodethe expected a posteriori value of node
assuming an equal interval scale, where L=1
, M=2
and
H=3
MAP
nodea factor vector giving maximum a
posteriori value of node, i.e.,
which.max(p
nodeH, p
nodeM, p
nodeL)
.
After a number of columns with this pattern, the last column is:
Cond
a factor describing the experimental condition
with levels Adaptive/Accuracy
, Adaptive/Extended
and
Linear/Extended
ACED.skillNames
is a character vector giving the abbreviations
used for the node names. Here are the interpretations:
sgp
Solve Geometric Problems. This is the highest level variable for the field trial data.
arg
Algebraic Rule Geometric
cr
Find Common Ratio
dt
Distinguish Types of series
exa
Examples (Geometric)
exp
Explicit Rule (Geometric)
ext
Extend Series (Geometric)
ind
Induce Rules (Geometric)
mod
Model (Geometric)
rec
Recursive Rules (Geometric)
tab
Tabular Representations (Geometric)
ver
Verbal Rules (Geometric)
pic
Pictorial Representations (Geometric)
ACED.items
is data frame with 230 observations on 73
variables. These are mostly item-level scores from the field trial.
Cond_code
a numeric vector giving the experimental
condition for this student, see also Cond
Seq
a factor describing whether the sequence of items
was Linear
or Adaptive
FB
a factor describing whether the feedback for
incorrect items was Extended
or AccuracyOnly
All_Items
a numeric vector giving the number of items in ACED
Correct
a numeric vector giving the number of items the student got correct
Incorr
a numeric vector giving the number of items the student got incorrect
Remain
a numeric vector giving the number of items not reached or skipped
ElapTime
a numeric vector giving the total time spent on ACED
The next 63 columns represent the items from the ACED assessment.
All are factor variables, with possible valued Incorrect
and
Correct
. The variables are named all named t
(for
task) followed by the name of one or more variables tapped by the
task (if there is more than one, then the first one is
“primary”.) This is followed by a numeric code, 1, 2 or 3,
giving the difficulty (easy, medium or hard) and a letter (a, b or
c) used to indicate alternate tasks following the same task model.
Finally, following a period, there is a version number (all of the
tasks are version 1).
After the variables, the last column is:
Cond
a factor describing the experimental condition
with levels Adaptive/Accuracy
, Adaptive/Extended
and
Linear/Extended
ACED.prePost
is data frame with 290 observations on 32
variables giving the results of the pretest and posttest.
Cond_code
a numeric vector giving the experimental
condition for this student, see also Cond
Seq
a factor describing whether the sequence of items
was Linear
or Adaptive
FB
a factor describing whether the feedback for
incorrect items was Extended
or AccuracyOnly
All_Items
a numeric vector giving the number of items in ACED
Form_Order
a factor variables describing whether
(AB
) Form A was the pretest and Form B was the posttest or
(BA
) vise versa.
Level_Code
a factor variable describing the academic
track of the student with levels Honors
, Academic
,
Regular
, Part 1
, Part 2
and ELL
. The
codes Part 1
and Part 2
refer to special education
students in Part 1 (mainstream classroom) or Part 2 (sequestered).
PreACorr
corrected score on Form A for students who took Form A as a pretest
PostBCorr
corrected score on Form B for students who took Form B as a posttest
PreBCorr
corrected score on Form B for students who took Form B as a pretest
PostACorr
corrected score on Form A for students who took Form A as a posttest
PreScore
a numeric vector with either the non-missing
value from PreACorr
and PreBCorr
PostScore
a numeric vector with either the non-missing
value from PostACorr
and PostBCorr
Gender
a factor variable giving the (self-reported) gender of the student (codebook is lost)
Race
a factor variable giving the (self-reported) race of the student (codebook is lost)
Gain
PostScore - PreScore
preacorr_adj
PreACorr
adjusted to put forms A
and B on the same scale
postbcorr_adj
PostBCorr
adjusted to put forms A
and B on the same scale
prebcorr_adj
PreBCorr
adjusted to put forms A
and B on the same scale
postacorr_adj
PostACorr
adjusted to put forms A
and B on the same scale
Zpreacorr_adj
standardized version of preacorr_adj
Zpostbcorr_adj
standardized version of postbcorr_adj
Zprebcorr_adj
standardized version of prebcorr_adj
Zpostacorr_adj
standardized version of postacorr_adj
scale_prea
score on Form A for students who took Form A as a pretest scaled to range 0-100
scale_preb
score on Form B for students who took Form B as a pretest scaled to range 0-100
pre_scaled
scale score on pretest (whichever form)
scale_posta
score on Form A for students who took Form A as a posttest scaled to range 0-100
scale_postb
score on Form B for students who took Form B as a posttest scaled to range 0-100
post_scaled
scale score on pretest (whichever form)
gain_scaled
post_scaled - pre_scaled
Flagged
a logical variable (codebook lost)
Cond
a factor describing the experimental condition
with levels Adaptive/Accuracy
, Adaptive/Extended
,
Linear/Extended
and Control
ACED is a Bayesian network based Assessment for Learning learning system, thus it served as both a assessment and a tutoring system. It had two novel features which could be turned on and off, elaborated feedback (turned off, it provided accuracy only feedback) and adaptive sequencing of items (turned off, it scheduled items in a fixed linear sequence).
It was originally built to cover all algebraic sequences (arithmetic, geometric and other recursive), but only the branch of the system using geometric sequences was tested. Shute, Hansen and Almond (2008) describe the field trial. Students from a local middle school (who studied arithmetic, but not geometric sequences as part of their algebra curriculum) were recruited for the study. The students were randomized into one of four groups:
Adaptive/Accuracy
Adaptive sequencing was used, but students only received correct/incorrect feedback.
Adaptive/Extended
Adaptive sequencing was used, but students received extended feedback for incorrect items.
Linear/Extended
The fixed linear sequencing was used, but students received extended feedback for incorrect items.
Control
The students did independent study and did not use ACED.
Because students in the control group were not exposed to the ACED
task, neither the Bayes net level scores nor the item level scores are
available for those groups, and those students are excluded from
ACED.scores
and ACED.items
. The students are in the
same order in all of the data sets, with the 60 control students
tacked onto the end of the ACED.prePost
data set.
All of the students (including the control students) were given a
25-item pretest and a 25-item posttest with items similar to the ones
used in ACED. The design was counterbalanced, with half of the
students receiving Form A as the pretest and Form B as the posttest
and the other half the other way around, to allow the two forms to be
equated using the pretest data. The details are buried in
ACED.prePost
.
Note that some irregularities were observed with the English Language
Learner (ACED.prePost$Level_code=="ELL"
) students. Their
teachers were allowed to translated words for the students, but in
many cases actually wound up giving instruction as part of the
translation.
Shute, V. J., Hansen, E. G., & Almond, R. G. (2008). You can't fatten a hog by weighing it—Or can you? Evaluating an assessment for learning system called ACED. International Journal of Artificial Intelligence and Education, 18(4), 289-316.
Thanks to Val Shute for permission to use the data.
ACED development and data collection was sponsored by National Science Foundation Grant No. 0313202.
A more detailed description, including a Q-matrix can be found at the ECD Wiki: http://ecd.ralmond.net/ecdwiki/ACED/ACED.
data(ACED)