Since this question is in the Cognitive Science category, I'm assuming you're asking for ideas supported by neuroscience evidence, and not artificial neural networks. Naturally, this makes things much more theoretical and difficult to get everyone on the same page.
...so the C in an input series of A, B, C can be handled differently than the C in input series A, A, C.
If I understand your example correctly, you are asking what the role of "circular" network connections is in processing series that vary over time, and with respect to the temporal context of the inputs. This is a very clever thing to ask and has many important applications to AI.
Conceptually, the brain does do this by having a mechanism to distinguish between inputs based on context. In your example, you are right in thinking that the C should be handled differently in ABC than it should in AAC. The ability to understand C in a way that is unique to the events leading up to it is a powerful one.
Contextual time series analysis is something that Jeff Hawkin's Hierarchical Temporal Memory (HTM) model of intelligence (based loosely on biologically inspired ideas) tries to accomplish, and it may be relevant to you. However, it does not incorporate feedback connections from higher (more abstract) regions to lower ones. Keep in mind that HTM still has its pitfalls in contextual time series analysis and so the actual way the brain does it is much more robust. I offer this merely as an anecdote.
From a biological perspective, you have to understand the word "circular" to mean the feedforward and feedback connections between neurons of different levels, as opposed to lateral connections between neurons within the same level. Neurons within the same level receive connections from the same inhibitory neurons (which are overwhelmingly local connections). You mentioned that neurons connect to themselves (N1 => N1), but this is not a very commonly studied thing in biology and probably has a role in error correction, boosting its own activity, or some other kind of activity modulation.
What seems to be happening in all parts of the brain, and even the retina, is that a level's excitatory neurons will send their feedforward signals to the next higher region's inhibitory and excitatory neurons. The inhibitory neurons will laterally inhibit the least active excitatory neurons within the same level. Some excitatory neurons will then send connections to the next higher region (feedforward) and the same process repeats. Other excitatory neurons will send connections to the lower region that connects to it (can consider these connections feedback), but will generally connect in a way to inhibit the excitatory neurons of the lower region by activating the inhibitory neurons (which work locally, thus only within that region).
With many more details excluded, this leads to a self-stabilizing framework where very little fine-tuning of arbitrary parameters is required (which apparently is a problem that you are well aware of in trying to find that balance). However, a circular network is just one substrate that is in use by the brain to understand context in time series. Unfortunately, circular connections actually have very little to do with contextual time series. Lateral connections seem to have much more to do with that than feedforward and feedback connections. With that said, there is much to be gained in combining the HTM's contextual time series analysis approach with a valid feedforward and feedback connection scheme between levels.
Feedforward connections mainly serve to make input signals much more abstract. Feedback connections serve as a way to fine-tune the lower levels and "tell" those levels which activations it is allowed to have.
If you want to learn more about circular connections and their role, I strongly recommend you study the retina. It is simpler than understanding the cortex and subcortical components, and we know much more about its cognitive processing than we do of the brain. Furthermore, the retina is actually an extension of the brain. It even comes complete with neurons that process signals in complex ways. You find the circular connections of which you speak in there, inhibitory/excitatory neurons, distinct hierarchical levels, and the higher cells even demonstrate characteristics of context-dependent input series on very small timescales.