NetworkTester-class {RNetica}R Documentation

Class "NetworkTester"

Description

The Network tester is special object that simulates data from a Bayes net and then tests how well the values of the targetNodes can be recovered from evidence in the remaining nodes (excluding the nodes in the ignoreNodes.

Details

The fuction testNetwork behaves a follows: It opens a NetworkTester object to hold the results. It then runs through the case streams. For each case, it instantiates each node in the network to the findings as found in the case stream, except for those nodes in targetNodes or ignoreNodes. It then looks at the NodeBeliefs associated with the targetNodes and compares these with the actual value of the target nodes as found in the case stream. It accumulates statistics related to the accuracy.

After all case streams are processed, the NetworkTester object is returned and its statistics can be queried. See testerConfusion for a list of available results.

Objects from the Class

Objects can be created by calling testNetwork(targetNodes, dataStreams, ingoreNodes). Here dataStreams should be a list of active CaseStream objects containing generated test data for the network.

Slots

Net:

Object of class "NeticaBN" which represents the network to be tested.

targetNodes:

A list of active NeticaNode objects which represent the nodes to be tested.

ignoreNodes:

A (possibly empty) list of active NeticaNode objects whose values will be ignored (not instantiated) during the tests.

data:

A list of active (open) CaseStream objects containing the test data.

errorRate:

A vector corresponding to the target nodes which contains the error rates from the test (see testerErrorRate).

logLoss:

A vector corresponding to the target nodes which contains the log loss rates from the test (see testerLogLoss).

quadraticLoss:

A vector corresponding to the target nodes which contains the quadratic loss rates from the test (see testerQuadraticLoss).

confusion:

A list of matrixes corresponding to the target nodes. Each matrix is a list corresponding to the states of the target node with rows representing the predicted values and actual representing the actual (simulated) value. (See testerConfusion).

Methods

testNetwork(targetNodes, dataStreams, ingoreNodes=list()):

This is the de facto constructor, it runs a test on the network and stores the results in tester object.

testerNet(tester):

Accessor for the network.

testerTarget(tester):

Accessor for the target nodes.

testerIgnore(tester):

Accessor for the ignored nodes.

testerErrorRate(tester,node=NULL):

Accessor for the error rate, if node is supplied, will give specific error rate, otherwise, it will give all of them.

testerLogLoss(tester,node=NULL):

Accessor for the log loss, if node is supplied, will give specific log loss, otherwise, it will give all of them.

testerQuadraticLoss(tester,node=NULL):

Accessor for the quadratic loss, if node is supplied, will give specific loss, otherwise, it will give all of them.

testerConfusion(tester,node=NULL):

Accessor for the confusion matrixes, if node is supplied, will give specific confusion matrix, otherwise, it will give a list of all of them.

testerKappa(tester,node=NULL, weights=c("None","Linear","Quadratic"), W=NULL):

Will calculate Cohen's Kappa for the specified node or for all of them.

testerLambda(tester,node=NULL, weights=c("None","Linear","Quadratic"), W=NULL):

Will calculate Goodman and Kruskal's Lambda for the specified node or for all of them.

summary.NetworkTester(object,...):

Produces a matrix giving the error rate, log loss, quadratic loss, about both weighted and unweighted kappas and lambdas.

[Note: These are implmemented as ordinary functions and not as methods.]

Note

Although the Netica API defines a NetTester object, this object is both created and destroyed in the C code for testNetwork. That code creates the tester object, runs the tests and extracts the results and places the cached results in an R object. This avoids memory management issues with foreign pointers.

Note that new data cannot simply be added to the tester, as works using the Netica API. However, tester objects can have multiple data streams.

Author(s)

Russell Almond

References

Almond, R.G., Mislevy, R.J. Steinberg, L.S., Yan, D. and Willamson, D. M. (2015). Bayesian Networks in Educational Assessment. Springer. Chapter 7.

http://norsys.com/onLineAPIManual/index.html: NewNetTester_bn(), DeleteNetTester_bn(), TestWithCaseset_bn(), GetTestConfusion_bn(), GetTestErrorRate_bn(), GetTestLogLoss_bn(), GetTestQuadraticLoss_bn()

See Also

The output of testNetwork is NetworkTester.

Accessors for NetworkTester testerConfusion, testerErrorRate, testerIgnore, testerKappa, testerLambda, testerLogLoss, testerNet, testerQuadraticLoss, testerTarget

Examples

sess <- NeticaSession()
startSession(sess)

irt5 <- ReadNetworks(file.path(library(help="RNetica")$path,
                           "sampleNets","IRT5.dne"), session=sess)

irt5.theta <- NetworkFindNode(irt5,"Theta")
irt5.x <- NetworkFindNode(irt5,paste("Item",1:5,sep="_"))

CompileNetwork(irt5)


## This generates a fixed series of random cases and saves them to a
## file. 
N <- 100L
rnodes <- c(list(irt5.theta),irt5.x)
casefile <- tempfile("irt5testcase",fileext=".cas")
filestream <- CaseFileStream(casefile, session=sess)
rng <- NewNeticaRNG(123456779, session=sess)
WithOpenCaseStream(filestream,
  WithRNG(rng,
    for (n in 1L:N) {
      GenerateRandomCase(rnodes,rng=rng)
      WriteFindings(rnodes,filestream,n)
      lapply(rnodes,RetractNodeFinding) # Only retract findings for
                                        # generated nodes
    }))


irt5.test <- testNetwork(list(irt5.theta),OpenCaseStream(filestream))
summary(irt5.test)


DeleteNetwork(irt5)
stopSession(sess)


[Package RNetica version 0.7-2 Index]