Just a couple examples:
This is the basis of @vkehayas's answer. More generally, despite integration at the soma being fairly linear up to a point, dendrites are highly non-linear as active conductances (i.e., voltage-gated channels) are necessary in addition to passive ones (i.e., neurotransmitter receptors or experimental current injections). This makes the spatial arrangement of input very important, not just the global sum over all input like a perceptron. See for example:
London, M., & Hausser, M. (2005). Dendritic computation. Annual review of neuroscience, 28(1), 503-532.
These spatial arrangements are also critical for distinct forms of inhibitory control of neurotransmission, particularly when people talk about "shunting inhibition" occurring with inhibition out on distal dendrites.
Synapses are not memoryless; short-term plasticity occurs primarily presynaptically through calcium concentrations. "Weak" synapses (=those with low release probability) tend to show facilitation, as single action potentials do not allow sufficient calcium entry into the presynaptic bouton for release, but over multiple summed action potentials in quick succession the calcium concentration rises and release probability increases.
"Strong" synapses (=those with high release probability) tend to show the opposite, short-term depression, because initial release of vesicles depletes those available to release.
See for example:
Abbott, L. F., & Regehr, W. G. (2004). Synaptic computation. Nature, 431(7010), 796-803.
(I'd also recommend Abbott's book with Peter Dayan, "Theoretical Neuroscience" as a good intermediate-level textbook on neural computation; it should be especially accessible if you are familiar with artificial networks)
Neuromodulation and diverse neurotransmitter actions
While one might take glutamatergic transmission and GABAergic transmission in the CNS and simplify these to a + and - sign, this greatly simplifies things even for just those two neurotransmitters. GABA, for example, will diffuse out of highly active synapses and activate different classes of highly sensitive GABA receptors located far from synapses. Glutamate doesn't just open excitatory channels, but also triggers G-protein receptors on both pre- and post-synaptic cells and can modulate synaptic strength over intermediate time scales.
There's enough complexity with just those two "typical" neurotransmitters before we start to think of other neuromodulators. You could fill library shelves with all the diverse ways that neuromodulators can change the behavior of a circuit. Some of the most remarkable examples are found in what are misleadingly thought of as "simple" nervous systems such as in invertebrates, but similar mechanisms also occur in mammalian nervous systems. Subtle neuromodulation at the level of a single-cell can have massive consequences when viewed at the scope of whole networks; the difference between wake and sleep, for example.
A couple reviews to start with:
Fellous, J. M., & Linster, C. (1998). Computational models of neuromodulation. Neural computation, 10(4), 771-805.
Marder, E., & Thirumalai, V. (2002). Cellular, synaptic and network effects of neuromodulation. Neural Networks, 15(4-6), 479-493.