NetworkTester-class {RNetica} | R Documentation |
"NetworkTester"
The Network tester is special object that simulates data from a Bayes
net and then tests how well the values of the targetNodes
can be
recovered from evidence in the remaining nodes (excluding the nodes in
the ignoreNodes
.
The fuction testNetwork
behaves a follows: It opens a
NetworkTester
object to hold the results. It
then runs through the case streams. For each case, it instantiates
each node in the network to the findings as found in the case stream,
except for those nodes in targetNodes
or ignoreNodes
.
It then looks at the NodeBeliefs
associated with the
targetNodes
and compares these with the actual value of the
target nodes as found in the case stream. It accumulates statistics
related to the accuracy.
After all case streams are processed, the
NetworkTester
object is returned and its
statistics can be queried. See testerConfusion
for a
list of available results.
Objects can be created by calling testNetwork(targetNodes,
dataStreams, ingoreNodes)
. Here dataStreams
should be a
list of active CaseStream
objects
containing generated test data for the network.
Net
:Object of class "NeticaBN"
which
represents the network to be tested.
targetNodes
:A list of active NeticaNode
objects which represent the nodes to be tested.
ignoreNodes
:A (possibly empty) list of active
NeticaNode
objects whose values will be ignored
(not instantiated) during the tests.
data
:A list of active (open)
CaseStream
objects containing the test data.
errorRate
:A vector corresponding to the target nodes
which contains the error rates from the test (see
testerErrorRate
).
logLoss
:A vector corresponding to the target nodes
which contains the log loss rates from the test (see
testerLogLoss
).
quadraticLoss
:A vector corresponding to the target nodes
which contains the quadratic loss rates from the test (see
testerQuadraticLoss
).
confusion
:A list of matrixes corresponding to the
target nodes. Each matrix is a list corresponding to the states
of the target node with rows representing the predicted values and
actual representing the actual (simulated) value. (See
testerConfusion
).
testNetwork(targetNodes, dataStreams,
ingoreNodes=list())
:This is the de facto constructor, it runs a test on the network and stores the results in tester object.
testerNet(tester)
:Accessor for the network.
testerTarget(tester)
:Accessor for the target nodes.
testerIgnore(tester)
:Accessor for the ignored nodes.
testerErrorRate(tester,node=NULL)
:Accessor for
the error rate, if node
is supplied, will give specific
error rate, otherwise, it will give all of them.
testerLogLoss(tester,node=NULL)
:Accessor for
the log loss, if node
is supplied, will give specific
log loss, otherwise, it will give all of them.
testerQuadraticLoss(tester,node=NULL)
:Accessor for
the quadratic loss, if node
is supplied, will give specific
loss, otherwise, it will give all of them.
testerConfusion(tester,node=NULL)
:Accessor for
the confusion matrixes, if node
is supplied, will give specific
confusion matrix, otherwise, it will give a list of all of them.
testerKappa(tester,node=NULL,
weights=c("None","Linear","Quadratic"), W=NULL)
:Will calculate Cohen's Kappa for the specified node or for all of them.
testerLambda(tester,node=NULL,
weights=c("None","Linear","Quadratic"), W=NULL)
:Will calculate Goodman and Kruskal's Lambda for the specified node or for all of them.
summary.NetworkTester(object,...)
:Produces a matrix giving the error rate, log loss, quadratic loss, about both weighted and unweighted kappas and lambdas.
[Note: These are implmemented as ordinary functions and not as methods.]
Although the Netica API defines a NetTester object, this object is
both created and destroyed in the C code for
testNetwork
. That code creates the tester object, runs
the tests and extracts the results and places the cached results in an
R object. This avoids memory management issues with foreign pointers.
Note that new data cannot simply be added to the tester, as works using the Netica API. However, tester objects can have multiple data streams.
Russell Almond
Almond, R.G., Mislevy, R.J. Steinberg, L.S., Yan, D. and Willamson, D. M. (2015). Bayesian Networks in Educational Assessment. Springer. Chapter 7.
http://norsys.com/onLineAPIManual/index.html: NewNetTester_bn(), DeleteNetTester_bn(), TestWithCaseset_bn(), GetTestConfusion_bn(), GetTestErrorRate_bn(), GetTestLogLoss_bn(), GetTestQuadraticLoss_bn()
The output of testNetwork
is NetworkTester
.
Accessors for NetworkTester
testerConfusion
, testerErrorRate
,
testerIgnore
, testerKappa
,
testerLambda
, testerLogLoss
,
testerNet
, testerQuadraticLoss
,
testerTarget
sess <- NeticaSession() startSession(sess) irt5 <- ReadNetworks(file.path(library(help="RNetica")$path, "sampleNets","IRT5.dne"), session=sess) irt5.theta <- NetworkFindNode(irt5,"Theta") irt5.x <- NetworkFindNode(irt5,paste("Item",1:5,sep="_")) CompileNetwork(irt5) ## This generates a fixed series of random cases and saves them to a ## file. N <- 100L rnodes <- c(list(irt5.theta),irt5.x) casefile <- tempfile("irt5testcase",fileext=".cas") filestream <- CaseFileStream(casefile, session=sess) rng <- NewNeticaRNG(123456779, session=sess) WithOpenCaseStream(filestream, WithRNG(rng, for (n in 1L:N) { GenerateRandomCase(rnodes,rng=rng) WriteFindings(rnodes,filestream,n) lapply(rnodes,RetractNodeFinding) # Only retract findings for # generated nodes })) irt5.test <- testNetwork(list(irt5.theta),OpenCaseStream(filestream)) summary(irt5.test) DeleteNetwork(irt5) stopSession(sess)