So lets be clear. (I think) we are talking about back propagation here.
Yes, you probably could get from the ABCD-trained network to an ACD-trained network.
The network adapts with each data point it is trained on. As such, the relationship between how an individual data point effects the net, changes with training. What really matters to the net is the average of all of these interactions, and this is were a lot of specific information related to individual data points is lost.
So in reality, achieving this would require you to know how the net (trained only on A) changed in response B. So we would need to be saving previous changes to weights and biases (for each data point if we wanted it to be more general/flexible).
I don't see this as particularly practical. But you could do it if you wanted.