Inductive Bias

Consider

• concept learning algorithm L
• instances X, target concept c
• training examples $D_c = \{\langle x, c(x) \rangle \}$
• let L(x_i,D_c) denote the classification assigned to the instance x_i by L after training on data D_c.

Informal: Any preference on the space of all possible hypotheses other than consistency with training examples

Formal Definition:

The inductive bias of L is any minimal set of assertions B such that for any target concept c and corresponding training examples D_c

$$(\forall x_i \in X) [(B \wedge D_c \wedge x_i) \vdash L(x_i,D_c)]$$

where $A \vdash B$ means A logically entails B