Markov Decision Processes (MDPs) in RR

[Up] [Top]

Documentation for package ‘MDP’ version 1.1

Help Pages

MDP-package Markov Decision Processes (MDPs) in R
actionIdxDf Info about the actions in the HMDP model under consideration.
actionIdxMat Info about the actions in the HMDP model under consideration.
actionInfo Info about the actions in the HMDP model under consideration.
actionWeightMat Info about the weights of the actions in the HMDP model under consideration.
binaryMDPWriter Function for writing an HMDP model to binary files. The function defineds subfunctions which can be used to define an HMDP model saved in a set of binary files.
calcRPO Calculate the rentention payoff (RPO) or opportunity cost for some states.
calcSteadyStatePr Calculate the steady state transition probabilities for the founder process (level 0).
calcWeights Calculate weights based on current policy. Normally run after an optimal policy has been found.
convertBinary2HMP Convert a HMDP model stored in binary format to a hmp (xml) file. The function simply parse the binary files and create hmp files using the hmpMDPWriter.
convertHMP2Binary Convert a HMDP model stored in a hmp (xml) file to binary file format.
fixAction Fix the action of a state. That is, the other actions are removed from the HMDP.
getActionTransIdS Get the ids of the transition states of an action.
getActionTransPr Get the transition probabilities of the transition states of an action.
getActionW Get the weights of an action.
getIdS Return ids for states having index string in idxS.
getIdSStages Return ids for states in a stage.
getLabel Return the label of states having id idS.
getPolicy Get parts of the optimal policy.
getPolicyW Get parts of the optimal policy weights.
getStrIdxS Return the index strings for states having id idS.
getTransPr Get the transition probability matrix P for the founder process (level 0).
getWIdx Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.
hmpMDPWriter Function for writing an HMDP model to a hmp file (XML). The function define subfunctions which can be used to define an HMDP model stored in a hmp file.
hypergf The state-expanded hypergraph as a matrix
info Information about a state and its corresponding actions
loadMDP Create the HMDP defined in the binary files. The model are created in memory using the external C++ library.
MDP Markov Decision Processes (MDPs) in R
policyIteAve Perform policy iteration (average criterion) on the MDP.
policyIteDiscount Perform policy iteration (discount criterion) on the MDP.
removeAction Remove the action of a state from the HMDP.
resetActions Reset the actions of a state.
setActionWeight Set the weight of an action.
setPolicy Set the current policy.
setPolicyAction Set the action of a state to be in the current policy.
setStateWeight Set the weight of a state.
stateIdxDf Info about the states in the HMDP model under consideration.
stateIdxMat Info about the states in the HMDP model under consideration.
transProbMat Info about the transition probabilities in the HMDP model under consideration.
valueIte Perform value iteration on the MDP.
weightNames Names of weights used in actions.