Curse of dimensionality

As the number of image features employed to train an algorithm increases there is a geometric increase in the number of training examples required, which is known as the curse of dimensionality.

Background

A feature is a quantity or trait based on which machine learning algorithms make predictions. Supervised machine learning algorithms take a collection of features as their inputs. Ideally, an algorithm uses the minimum number of features that can differentiate between possible answers. In the context of machine learning in clinical medicine, features may take on many different categories of variables; this may include patient demographic data (e.g. age, sex, weight), clinical characteristics (e.g. CRP, heart rate, temperature), or information derived from medical imaging (e.g. greyscale value of each individual pixel)

Intuition

As the number of features increases, the number of data points (or “training examples” in machine learning) required to train the algorithm increases exponentially. The intuition behind this can be visualized by imagining a set of 10 points on a line 10 centimeters long. If we increase the dimension of this line to 2 (i.e. it becomes a square), the number of points required for a similar density is increased to 100 (or 10). This property of an exponentially-growing requirement for input data as the number of features increases is known as the curse of dimensionality.

Importance in radiology

In the context of radiology, the number of input features can grow very large, particularly in pixel-based machine learning algorithms where each pixel (or voxel) of an input image represents a distinct feature. The process of reducing the dimension size of the input into machine learning algorithms to avoid the curse of dimensionality is known as dimensionality reduction. In medical imaging, this typically involves one or more pre-processing steps applied to inputted images aimed at extracting the most salient features of the images.