Linear-probe separability increases monotonically with CNN depth

Alain & Bengio introduce the linear-classifier-probe method, fitting an independent linear probe at each layer of trained Inception-v3 and ResNet-50 [alain-bengio-2016-linear-classifier-probes] Probe accuracy, a linear-separability measurement, increases monotonically with network depth [alain-bengio-2016-linear-classifier-probes] This foundational work establishes linear probing as a standard representation-analysis technique and is purely observational [alain-bengio-2016-linear-classifier-probes] The authors caution probe accuracy is only a lower bound on the information present and can be confounded by probe capacity [alain-bengio-2016-linear-classifier-probes]