Representation Generation in An Exploratory Learning System
Paul Scott and Shaul Markovitch. Representation Generation in An Exploratory Learning System. In D. Fisher and M. Pazzani, Editors, Concept Formation: Knowledge and Experience in Unsupervised Learning. Morgan Kaufmann, 1991.
Abstract
This paper describes some of the features of DIDO, a program we have developed to investigate possible solutions to three problems that arise in exploratory learning. The first problem is the absence of an external agent to define the set of categories to be learned. DIDO's solution is to partition the domain into classes of objects whose behavior in response to actions is equivalent. Thus the representation built is inherently 'DIDOcentric' and provides the knowledge needed to plan sequences of actions to solve problems. The second problem, which also arises because of the absence of an external agent, is the need for an evaluation of the quality of a representation. DIDO's solution is to attempt to construct a representation with no uncertainty. Since uncertainty arises as a result of experience, and since DIDO's experience generator deliberately seeks experiences likely to increase the uncertainty, this leads to a representation whose predictions are both unique and consistent with the behavior of the domain. In fact the operation of DIDO can be viewed as a struggle between the experience generator, which tries to increase the uncertainty in the representation, and the representation generator which tries to minimize it. This is a struggle which the representation generator wins in all but very noisy domains.
Keywords: Relational Reinforcement Learning, Active Learning, Exploration
Secondary Keywords:
Online version:
Bibtex entry:
@incollection{Scott:1991:RGE,
Author = {Paul Scott and Shaul Markovitch},
Title = {Representation Generation in An Exploratory Learning System},
Year = {1991},
Booktitle = {Concept Formation: Knowledge and Experience in Unsupervised Learning},
Editor = {D. Fisher and M. Pazzani},
Url = {http://www.cs.technion.ac.il/~shaulm/papers/pdf/Scott-Markovitch-1991.pdf},
Keywords = {Relational Reinforcement Learning, Active Learning, Exploration},
Secondary-keywords = {Exploratory Learning},
Abstract = {
This paper describes some of the features of DIDO, a program we
have developed to investigate possible solutions to three problems
that arise in exploratory learning. The first problem is the
absence of an external agent to define the set of categories to be
learned. DIDO's solution is to partition the domain into classes
of objects whose behavior in response to actions is equivalent.
Thus the representation built is inherently 'DIDOcentric' and
provides the knowledge needed to plan sequences of actions to
solve problems. The second problem, which also arises because of
the absence of an external agent, is the need for an evaluation of
the quality of a representation. DIDO's solution is to attempt to
construct a representation with no uncertainty. Since uncertainty
arises as a result of experience, and since DIDO's experience
generator deliberately seeks experiences likely to increase the
uncertainty, this leads to a representation whose predictions are
both unique and consistent with the behavior of the domain. In
fact the operation of DIDO can be viewed as a struggle between the
experience generator, which tries to increase the uncertainty in
the representation, and the representation generator which tries
to minimize it. This is a struggle which the representation
generator wins in all but very noisy domains.
}
}