I am looking for a specific type of experimental test of active learning. Given some artificial or natural learning task that consists of classifying inputs $x$ from a large input space $X$. There is some underlying function $f: X \rightarrow \{0,1\}$ to be learned, this function can come from some restricted function class $C$. The subject knows the description of the function class, but has to learn the specific parameters that define the function. I am looking for experiments that had two conditions of this type:

1. The subject is given labeled samples $(x,f(x))$ from some distribution on $X$ that they have no control over.
2. The subject is allowed to ask for specific $x \in X$ and receive the labeled sample $(x,f(x))$.

The first condition would correspond to passive learning, while the second to active learning.

From arm-chair expectations (or formal theories like CoLT), one would expect that on certain tasks the subjects in the active condition would be able to significantly outperform (predict the underlying function more accurately) than in the passive condition.

Can you provide references to specific experiments that would fit in this paradigm? What were their results? On what sort of tasks does active learning provide an advantage? How significant is this advantage?

###Example

An example would be a subject sitting in front of a blank screen. They are told "there is a straight line that divides this screen into a red and green region. Can you find it?". Thus, the set $X$ would correspond to coordinates, and an input $x$ would be a specific coordinate, say $x = (1,3)$. The function class would be the set of straight line partitions of the screen. A specific function with $0$ meaning red and $1$ - green might be:

$$f((a,b)) = \begin{cases}0 & \mathrm{if} \; a > b \\ 1 & \mathrm{otherwise}\end{cases}$$

In the active task, the participant can click on any point $x$ on the screen to turn it from blank to either red (if $f(x) = 0$) or green (if $f(x) = 1$). In the passive trial, some number of points are selected and revealed at random to the subject.

I am looking for a specific type of experimental test of active learning. Given some artificial or natural learning task that consists of classifying inputs $x$ from a large input space $X$. There is some underlying function $f: X \rightarrow \{0,1\}$ to be learned, this function can come from some restricted function class $C$. The subject knows the description of the function class, but has to learn the specific parameters that define the function. I am looking for experiments that had two conditions of this type:

1. The subject is given labeled samples $(x,f(x))$ from some distribution on $X$ that they have no control over.
2. The subject is allowed to ask for specific $x \in X$ and receive the labeled sample $(x,f(x))$.

The first condition would correspond to passive learning, while the second to active learning.

From arm-chair expectations (or formal theories like CoLT), one would expect that on certain tasks the subjects in the active condition would be able to significantly outperform (predict the underlying function more accurately) than in the passive condition.

Can you provide references to specific experiments that would fit in this paradigm? What were their results? On what sort of tasks does active learning provide an advantage? How significant is this advantage?

Example

An example would be a subject sitting in front of a blank screen. They are told "there is a straight line that divides this screen into a red and green region. Can you find it?". Thus, the set $X$ would correspond to coordinates, and an input $x$ would be a specific coordinate, say $x = (1,3)$. The function class would be the set of straight line partitions of the screen. A specific function with $0$ meaning red and $1$ - green might be:

$$f((a,b)) = \begin{cases}0 & \mathrm{if} \; a > b \\ 1 & \mathrm{otherwise}\end{cases}$$

In the active task, the participant can click on any point $x$ on the screen to turn it from blank to either red (if $f(x) = 0$) or green (if $f(x) = 1$). In the passive trial, some number of points are selected and revealed at random to the subject.