Experience Selection and Problem Choice in an Exploratory Learning System

Paul Scott and Shaul Markovitch. Experience Selection and Problem Choice in an Exploratory Learning System. Machine Learning, 12:49-67 1993.

Abstract

A fully autonomous exploratory learning system must perform two tasks that are not required of supervised learning systems: experience selection and problem choice. Experience selection is the process of choosing informative training examples from the space of all possible examples. Problem choice is the process of identifying defects in the domain theory and determining which should be remedied next. These processes are closely related because the degree to which a specific experience is informative depends on the particular defects in the domain theory that the system is attempting to remedy. In this article we propose a general control structure for exploratory learning in which problem choice by an information-theoretic "curiosity" heuristic: the problem chosen then guides the selection of training examples. An implementation of an exploratory learning system based on this control structure is described, and a series of experimental results are presented.

Keywords: Active Learning, Relational Reinforcement Learning, Exploration

Secondary Keywords:

Online version:

Bibtex entry:

 @article{Scott:1993:ESP,
  Author = {Paul Scott and Shaul Markovitch},
  Title = {Experience Selection and Problem Choice in an Exploratory Learning System},
  Year = {1993},
  Journal = {Machine Learning},
  Volume = {12},
  Pages = {49--67},
  Url = {http://www.cs.technion.ac.il/~shaulm/papers/pdf/Scott-Markovitch-mlj1993.pdf},
  Keywords = {Active Learning, Relational Reinforcement Learning, Exploration},
  Secondary-keywords = {Exploratory Learning},
  Abstract = {
    A fully autonomous exploratory learning system must perform two
    tasks that are not required of supervised learning systems:
    experience selection and problem choice. Experience selection is
    the process of choosing informative training examples from the
    space of all possible examples. Problem choice is the process of
    identifying defects in the domain theory and determining which
    should be remedied next. These processes are closely related
    because the degree to which a specific experience is informative
    depends on the particular defects in the domain theory that the
    system is attempting to remedy. In this article we propose a
    general control structure for exploratory learning in which
    problem choice by an information-theoretic "curiosity" heuristic:
    the problem chosen then guides the selection of training examples.
    An implementation of an exploratory learning system based on this
    control structure is described, and a series of experimental
    results are presented.
  }

  }