A curious robot: An explorative-exploitive inference algorithm

Kim Steenstrup Pedersen; Peter Johansen

A curious robot: An explorative-exploitive inference algorithm

Department of Computer Science

Abstract

We propose a sequential learning algorithm with a focus on robot
control. It is initialised by a teacher who directs the robot
through a series of example solutions of a problem. Left alone, the
control chooses its next action by prediction based on a variable
order Markov chain model selected to minimise a MDL criterion based
on generalised code length L_a of the past robot-environment
interaction. The user specifies the parameter a and as a
result, the robot can be directed towards exploratory behaviour if
confidence in the teacher is low (a<0), and towards
goal-seeking exploitive behaviour if confidence in the teacher is
high (a>0). The novelty of the proposed method lies in the
use of generalised code length in the MDL model selection criterion.

Original language	English
Title of host publication	RoboMat 07 : Coimbra, Portugal, 17-19 September, 2007
Editors	Helder Araujo, Maria Isabel Ribeiro
Number of pages	6
Publisher	CIM (Centro Internacional de Matematica)
Publication date	2007
Pages	51-57
ISBN (Print)	9789899501133
Publication status	Published - 2007
Event	Workshop of Robotics and Mathematics (RoboMat 2007) - Coimbra, Portugal Duration: 17 Sept 2007 → 19 Sept 2007

Conference

Conference	Workshop of Robotics and Mathematics (RoboMat 2007)
Country/Territory	Portugal
City	Coimbra
Period	17/09/2007 → 19/09/2007

Access to Document

http://labvis.isr.uc.pt/robomat/program.html

Cite this

A curious robot: An explorative-exploitive inference algorithm. / Pedersen, Kim Steenstrup; Johansen, Peter.
RoboMat 07: Coimbra, Portugal, 17-19 September, 2007. ed. / Helder Araujo; Maria Isabel Ribeiro. CIM (Centro Internacional de Matematica), 2007. p. 51-57.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

@inproceedings{b3adca60b54e11dcbee902004c4f4f50,

title = "A curious robot: An explorative-exploitive inference algorithm",

abstract = "We propose a sequential learning algorithm with a focus on robotcontrol. It is initialised by a teacher who directs the robotthrough a series of example solutions of a problem. Left alone, thecontrol chooses its next action by prediction based on a variableorder Markov chain model selected to minimise a MDL criterion basedon generalised code length La of the past robot-environmentinteraction. The user specifies the parameter a and as aresult, the robot can be directed towards exploratory behaviour ifconfidence in the teacher is low (a<0), and towardsgoal-seeking exploitive behaviour if confidence in the teacher ishigh (a>0). The novelty of the proposed method lies in theuse of generalised code length in the MDL model selection criterion.",

author = "Pedersen, {Kim Steenstrup} and Peter Johansen",

year = "2007",

language = "English",

isbn = "9789899501133",

pages = "51--57",

editor = "Helder Araujo and Ribeiro, {Maria Isabel}",

booktitle = "RoboMat 07",

publisher = "CIM (Centro Internacional de Matematica)",

note = "<em>Workshop of Robotics and Mathematics (RoboMat 2007)</em> ; Conference date: 17-09-2007 Through 19-09-2007",

}

TY - GEN

T1 - A curious robot: An explorative-exploitive inference algorithm

AU - Pedersen, Kim Steenstrup

AU - Johansen, Peter

PY - 2007

Y1 - 2007

N2 - We propose a sequential learning algorithm with a focus on robotcontrol. It is initialised by a teacher who directs the robotthrough a series of example solutions of a problem. Left alone, thecontrol chooses its next action by prediction based on a variableorder Markov chain model selected to minimise a MDL criterion basedon generalised code length La of the past robot-environmentinteraction. The user specifies the parameter a and as aresult, the robot can be directed towards exploratory behaviour ifconfidence in the teacher is low (a<0), and towardsgoal-seeking exploitive behaviour if confidence in the teacher ishigh (a>0). The novelty of the proposed method lies in theuse of generalised code length in the MDL model selection criterion.

AB - We propose a sequential learning algorithm with a focus on robotcontrol. It is initialised by a teacher who directs the robotthrough a series of example solutions of a problem. Left alone, thecontrol chooses its next action by prediction based on a variableorder Markov chain model selected to minimise a MDL criterion basedon generalised code length La of the past robot-environmentinteraction. The user specifies the parameter a and as aresult, the robot can be directed towards exploratory behaviour ifconfidence in the teacher is low (a<0), and towardsgoal-seeking exploitive behaviour if confidence in the teacher ishigh (a>0). The novelty of the proposed method lies in theuse of generalised code length in the MDL model selection criterion.

M3 - Article in proceedings

SN - 9789899501133

SP - 51

EP - 57

BT - RoboMat 07

A2 - Araujo, Helder

A2 - Ribeiro, Maria Isabel

PB - CIM (Centro Internacional de Matematica)

T2 - <em>Workshop of Robotics and Mathematics (RoboMat 2007)</em>

Y2 - 17 September 2007 through 19 September 2007

ER -

A curious robot: An explorative-exploitive inference algorithm

Abstract

Conference

Access to Document

Fingerprint

Cite this