ACED.scores {CPTtools}R Documentation

Data from ACED field trial

Description

ACED (Adaptive Content with Evidence-Based Diagnosis; Shute, Hansen and Almond, 2008) is a Bayes net based assessment system which featured: (a) adaptive item selection and (b) extended feedback for incorrect items. This data contains both item level and pretest/posttest data from a field trial of the ACED system.

Usage

data("ACED")

Format

ACED contains 3 data.frame objects and one explanatory variable.

ACED.scores is data frame with 230 observations on 74 variables. These are mostly high-level scores from the Bayesian network.

Cond_code

a numeric vector giving the experimental condition for this student, see also Cond

Seq

a factor describing whether the sequence of items was Linear or Adaptive

FB

a factor describing whether the feedback for incorrect items was Extended or AccuracyOnly

All_Items

a numeric vector giving the number of items in ACED

Correct

a numeric vector giving the number of items the student got correct

Incorr

a numeric vector giving the number of items the student got incorrect

Remain

a numeric vector giving the number of items not reached or skipped

ElapTime

a numeric vector giving the total time spent on ACED

The next group of columns give “scores” for each of the nodes in the Bayesian network. Each node has four scores, and the columns are names pnodeScoreType where node is replaced by one of the codes in ACED.allSkills.

pnodeH

a numeric vector giving the probability node is in the high state

pnodeM

a numeric vector giving the probability node is in the medium state

pnodeL

a numeric vector giving the probability node is in the low state

EAPnode

the expected a posteriori value of node assuming an equal interval scale, where L=1, M=2 and H=3

MAPnode

a factor vector giving maximum a posteriori value of node, i.e., which.max(pnodeH, pnodeM, pnodeL).

After a number of columns with this pattern, the last column is:

Cond

a factor describing the experimental condition with levels Adaptive/Accuracy, Adaptive/Extended and Linear/Extended

ACED.skillNames is a character vector giving the abbreviations used for the node names. Here are the interpretations:

sgp

Solve Geometric Problems. This is the highest level variable for the field trial data.

arg

Algebraic Rule Geometric

cr

Find Common Ratio

dt

Distinguish Types of series

exa

Examples (Geometric)

exp

Explicit Rule (Geometric)

ext

Extend Series (Geometric)

ind

Induce Rules (Geometric)

mod

Model (Geometric)

rec

Recursive Rules (Geometric)

tab

Tabular Representations (Geometric)

ver

Verbal Rules (Geometric)

pic

Pictorial Representations (Geometric)

ACED.items is data frame with 230 observations on 73 variables. These are mostly item-level scores from the field trial.

Cond_code

a numeric vector giving the experimental condition for this student, see also Cond

Seq

a factor describing whether the sequence of items was Linear or Adaptive

FB

a factor describing whether the feedback for incorrect items was Extended or AccuracyOnly

All_Items

a numeric vector giving the number of items in ACED

Correct

a numeric vector giving the number of items the student got correct

Incorr

a numeric vector giving the number of items the student got incorrect

Remain

a numeric vector giving the number of items not reached or skipped

ElapTime

a numeric vector giving the total time spent on ACED

The next 63 columns represent the items from the ACED assessment. All are factor variables, with possible valued Incorrect and Correct. The variables are named all named t (for task) followed by the name of one or more variables tapped by the task (if there is more than one, then the first one is “primary”.) This is followed by a numeric code, 1, 2 or 3, giving the difficulty (easy, medium or hard) and a letter (a, b or c) used to indicate alternate tasks following the same task model. Finally, following a period, there is a version number (all of the tasks are version 1).

After the variables, the last column is:

Cond

a factor describing the experimental condition with levels Adaptive/Accuracy, Adaptive/Extended and Linear/Extended

ACED.prePost is data frame with 290 observations on 32 variables giving the results of the pretest and posttest.

Cond_code

a numeric vector giving the experimental condition for this student, see also Cond

Seq

a factor describing whether the sequence of items was Linear or Adaptive

FB

a factor describing whether the feedback for incorrect items was Extended or AccuracyOnly

All_Items

a numeric vector giving the number of items in ACED

Form_Order

a factor variables describing whether (AB) Form A was the pretest and Form B was the posttest or (BA) vise versa.

Level_Code

a factor variable describing the academic track of the student with levels Honors, Academic, Regular, Part 1, Part 2 and ELL. The codes Part 1 and Part 2 refer to special education students in Part 1 (mainstream classroom) or Part 2 (sequestered).

PreACorr

corrected score on Form A for students who took Form A as a pretest

PostBCorr

corrected score on Form B for students who took Form B as a posttest

PreBCorr

corrected score on Form B for students who took Form B as a pretest

PostACorr

corrected score on Form A for students who took Form A as a posttest

PreScore

a numeric vector with either the non-missing value from PreACorr and PreBCorr

PostScore

a numeric vector with either the non-missing value from PostACorr and PostBCorr

Gender

a factor variable giving the (self-reported) gender of the student (codebook is lost)

Race

a factor variable giving the (self-reported) race of the student (codebook is lost)

Gain

PostScore - PreScore

preacorr_adj

PreACorr adjusted to put forms A and B on the same scale

postbcorr_adj

PostBCorr adjusted to put forms A and B on the same scale

prebcorr_adj

PreBCorr adjusted to put forms A and B on the same scale

postacorr_adj

PostACorr adjusted to put forms A and B on the same scale

Zpreacorr_adj

standardized version of preacorr_adj

Zpostbcorr_adj

standardized version of postbcorr_adj

Zprebcorr_adj

standardized version of prebcorr_adj

Zpostacorr_adj

standardized version of postacorr_adj

scale_prea

score on Form A for students who took Form A as a pretest scaled to range 0-100

scale_preb

score on Form B for students who took Form B as a pretest scaled to range 0-100

pre_scaled

scale score on pretest (whichever form)

scale_posta

score on Form A for students who took Form A as a posttest scaled to range 0-100

scale_postb

score on Form B for students who took Form B as a posttest scaled to range 0-100

post_scaled

scale score on pretest (whichever form)

gain_scaled

post_scaled - pre_scaled

Flagged

a logical variable (codebook lost)

Cond

a factor describing the experimental condition with levels Adaptive/Accuracy, Adaptive/Extended, Linear/Extended and Control

Details

ACED is a Bayesian network based Assessment for Learning learning system, thus it served as both a assessment and a tutoring system. It had two novel features which could be turned on and off, elaborated feedback (turned off, it provided accuracy only feedback) and adaptive sequencing of items (turned off, it scheduled items in a fixed linear sequence).

It was originally built to cover all algebraic sequences (arithmetic, geometric and other recursive), but only the branch of the system using geometric sequences was tested. Shute, Hansen and Almond (2008) describe the field trial. Students from a local middle school (who studied arithmetic, but not geometric sequences as part of their algebra curriculum) were recruited for the study. The students were randomized into one of four groups:

Adaptive/Accuracy

Adaptive sequencing was used, but students only received correct/incorrect feedback.

Adaptive/Extended

Adaptive sequencing was used, but students received extended feedback for incorrect items.

Linear/Extended

The fixed linear sequencing was used, but students received extended feedback for incorrect items.

Control

The students did independent study and did not use ACED.

Because students in the control group were not exposed to the ACED task, neither the Bayes net level scores nor the item level scores are available for those groups, and those students are excluded from ACED.scores and ACED.items. The students are in the same order in all of the data sets, with the 60 control students tacked onto the end of the ACED.prePost data set.

All of the students (including the control students) were given a 25-item pretest and a 25-item posttest with items similar to the ones used in ACED. The design was counterbalanced, with half of the students receiving Form A as the pretest and Form B as the posttest and the other half the other way around, to allow the two forms to be equated using the pretest data. The details are buried in ACED.prePost.

Note that some irregularities were observed with the English Language Learner (ACED.prePost$Level_code=="ELL") students. Their teachers were allowed to translated words for the students, but in many cases actually wound up giving instruction as part of the translation.

Source

Shute, V. J., Hansen, E. G., & Almond, R. G. (2008). You can't fatten a hog by weighing it—Or can you? Evaluating an assessment for learning system called ACED. International Journal of Artificial Intelligence and Education, 18(4), 289-316.

Thanks to Val Shute for permission to use the data.

ACED development and data collection was sponsored by National Science Foundation Grant No. 0313202.

References

A more detailed description, including a Q-matrix can be found at the ECD Wiki: http://ecd.ralmond.net/ecdwiki/ACED/ACED.

Examples

data(ACED)

[Package CPTtools version 0.7-2 Index]