Markov LIMID processes for representing and solving renewal problems

Erik Jørgensen; Anders Ringgaard Kristensen; Dennis  Nilsson

doi:10.1007/s10479-012-1220-4

Markov LIMID processes for representing and solving renewal problems

Erik Jørgensen, Anders Ringgaard Kristensen, Dennis Nilsson

1 Citationer (Scopus)

Abstract

In this paper a new tool for simultaneous optimisation of decisions on multiple time scales is presented. The tool combines the dynamic properties of Markov decision processes with the flexible and compact state space representation of LImited Memory Influence Diagrams (Limids). A temporal version of Limids, TemLimids, is defined by adding time-related functions to utility nodes. As a result, expected discounted utility, as well as expected relative utility might be used as optimisation criteria in TemLimids. Optimisation proceeds as in ordinary Limids. A sequence of such TemLimids can be used to model a Markov Limid Process, where each TemLimid represents a macro action. Algorithms are presented to find optimal plans for a sequence of such macro actions. Use of algorithms is illustrated based on an extended version of an example from pig production originally used to introduce the Limid concept.

Originalsprog	Engelsk
Tidsskrift	Annals of Operations Research
Vol/bind	219
Udgave nummer	1
Sider (fra-til)	63-84
Antal sider	22
ISSN	0254-5330
DOI	https://doi.org/10.1007/s10479-012-1220-4
Status	Udgivet - aug. 2014

Adgang til dokumentet

10.1007/s10479-012-1220-4

Citationsformater

@article{0d501f549bc84d67b353ae9cf8749213,

title = "Markov LIMID processes for representing and solving renewal problems",

abstract = "In this paper a new tool for simultaneous optimisation of decisions on multiple time scales is presented. The tool combines the dynamic properties of Markov decision processes with the flexible and compact state space representation of LImited Memory Influence Diagrams (Limids). A temporal version of Limids, TemLimids, is defined by adding time-related functions to utility nodes. As a result, expected discounted utility, as well as expected relative utility might be used as optimisation criteria in TemLimids. Optimisation proceeds as in ordinary Limids. A sequence of such TemLimids can be used to model a Markov Limid Process, where each TemLimid represents a macro action. Algorithms are presented to find optimal plans for a sequence of such macro actions. Use of algorithms is illustrated based on an extended version of an example from pig production originally used to introduce the Limid concept.",

author = "Erik J{\o}rgensen and Kristensen, {Anders Ringgaard} and Dennis Nilsson",

year = "2014",

month = aug,

doi = "10.1007/s10479-012-1220-4",

language = "English",

volume = "219",

pages = "63--84",

journal = "Annals of Operations Research",

issn = "0254-5330",

publisher = "Springer",

number = "1",

}

TY - JOUR

T1 - Markov LIMID processes for representing and solving renewal problems

AU - Jørgensen, Erik

AU - Kristensen, Anders Ringgaard

AU - Nilsson, Dennis

PY - 2014/8

Y1 - 2014/8

N2 - In this paper a new tool for simultaneous optimisation of decisions on multiple time scales is presented. The tool combines the dynamic properties of Markov decision processes with the flexible and compact state space representation of LImited Memory Influence Diagrams (Limids). A temporal version of Limids, TemLimids, is defined by adding time-related functions to utility nodes. As a result, expected discounted utility, as well as expected relative utility might be used as optimisation criteria in TemLimids. Optimisation proceeds as in ordinary Limids. A sequence of such TemLimids can be used to model a Markov Limid Process, where each TemLimid represents a macro action. Algorithms are presented to find optimal plans for a sequence of such macro actions. Use of algorithms is illustrated based on an extended version of an example from pig production originally used to introduce the Limid concept.

AB - In this paper a new tool for simultaneous optimisation of decisions on multiple time scales is presented. The tool combines the dynamic properties of Markov decision processes with the flexible and compact state space representation of LImited Memory Influence Diagrams (Limids). A temporal version of Limids, TemLimids, is defined by adding time-related functions to utility nodes. As a result, expected discounted utility, as well as expected relative utility might be used as optimisation criteria in TemLimids. Optimisation proceeds as in ordinary Limids. A sequence of such TemLimids can be used to model a Markov Limid Process, where each TemLimid represents a macro action. Algorithms are presented to find optimal plans for a sequence of such macro actions. Use of algorithms is illustrated based on an extended version of an example from pig production originally used to introduce the Limid concept.

U2 - 10.1007/s10479-012-1220-4

DO - 10.1007/s10479-012-1220-4

M3 - Journal article

SN - 0254-5330

VL - 219

SP - 63

EP - 84

JO - Annals of Operations Research

JF - Annals of Operations Research

IS - 1

ER -