Markov LIMID processes for representing and solving renewal problems

Erik Jørgensen, Anders Ringgaard Kristensen, Dennis Nilsson

    1 Citation (Scopus)

    Abstract

    In this paper a new tool for simultaneous optimisation of decisions on multiple time scales is presented. The tool combines the dynamic properties of Markov decision processes with the flexible and compact state space representation of LImited Memory Influence Diagrams (Limids). A temporal version of Limids, TemLimids, is defined by adding time-related functions to utility nodes. As a result, expected discounted utility, as well as expected relative utility might be used as optimisation criteria in TemLimids. Optimisation proceeds as in ordinary Limids. A sequence of such TemLimids can be used to model a Markov Limid Process, where each TemLimid represents a macro action. Algorithms are presented to find optimal plans for a sequence of such macro actions. Use of algorithms is illustrated based on an extended version of an example from pig production originally used to introduce the Limid concept.

    Original languageEnglish
    JournalAnnals of Operations Research
    Volume219
    Issue number1
    Pages (from-to)63-84
    Number of pages22
    ISSN0254-5330
    DOIs
    Publication statusPublished - Aug 2014

    Fingerprint

    Dive into the research topics of 'Markov LIMID processes for representing and solving renewal problems'. Together they form a unique fingerprint.

    Cite this