Technical Report CIS9810

Title: Feature Generation Using General Constructor Functions
Authors: Shaul Markovitch and Danny Rosenstein
Abstract: Most classification algorithms receive as input a set of attributes of the classified objects. In many cases, however, the supplied set of attributes is not sufficient for creating an accurate, succinct and comprehensible representation of the target concept. To overcome this problem, researchers have proposed algorithms for automatic construction of features. The majority of these algorithms use a limited predefined set of operators for building new features. In this paper we propose a generalized unifying framework that is capable of generating features based on any given set of constructor functions. These functions can be domain-independent such as arithmetic and logic operators, or, can be domain-dependent operators that rely on partial knowledge of the user. The paper describes an algorithm which receives as input a set of classified objects, a set of attributes, and a specification of a set of constructor functions, which contains their domains, ranges and properties. The algorithm produces as output a set of generated features that can be used by standard concept learners to create improved classifiers. The algorithm maintains a set of its best generated features and improves this set iteratively. During each iteration, the algorithm performs a beam-search over its defined feature space, and constructs new features by applying constructor functions to the members of its current feature set. The search of the algorithm is guided by general heuristic measures, that are not confined to a specific feature representation. The algorithm was applied to a variety of classification problems and was able to generate features that were strongly related to the underlying target concepts. These features also significantly improved the accuracy achieved by standard concept learners, for a variety of classification problems.
CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF files directly. The latter URLs may change without notice.

To the list of the CIS technical reports of 1998
To the main CS technical reports page

Computer science department, Technion