3
$\begingroup$

Say that you've trained a network with a set of training data comprising of data points A,B,C, and D.

Can the effect of, say, point B on the network be undone without also losing the training from points C and D? In other words, can we perform some operation on the ABCD-trained network and end up with a network identical to one that has been trained only on A,C, and D?

$\endgroup$
  • $\begingroup$ For those who use SQL, I know this can be done in a backprop routine with a GROUP BY or windowing clause on the data points. This essentially spits out a different model for each unique combination of data points, although the complexity can be cut by filtering with GROUPING_ID or some similar SQL clause (I'm using T-SQL syntax here). This would of course cost computing resources, but most SQL products at least have built-in ways of managing those costs (like indexing) and shunting them towards IO, memory, or the CPU, depending on which would be optimal (using query hints for example). $\endgroup$ – SQLServerSteve Sep 30 '16 at 23:04
2
$\begingroup$

So lets be clear. (I think) we are talking about back propagation here.

Yes, you probably could get from the ABCD-trained network to an ACD-trained network.

The network adapts with each data point it is trained on. As such, the relationship between how an individual data point effects the net, changes with training. What really matters to the net is the average of all of these interactions, and this is were a lot of specific information related to individual data points is lost.

So in reality, achieving this would require you to know how the net (trained only on A) changed in response B. So we would need to be saving previous changes to weights and biases (for each data point if we wanted it to be more general/flexible).

I don't see this as particularly practical. But you could do it if you wanted.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.