In the cognitive sciences Alan Turing is best known for launching AI with his Computing machinery and intelligence (1950). However, this was not his first contribution to the cognitive sciences, in his unpublished 1948 technical report Intelligence Machinery he foresaw connectionism with his B-type neural networks.

The model is a recurrent neural network that is wired at random, and synchronized by a global clock. The neurons are two input $\mathrm{NAND}$-gates. The connections have one of two states: they either forward their signal perfectly ($0 \mapsto 0$ and $1 \mapsto 1$), or replace it by $1$ ($0 \mapsto 1$ and $1 \mapsto 1$). The learning algorithm adjusts the states of the connections.

Unfortunately, the director of the National Physical Laboratory rejected Turing's work and it was not published until significantly after Turing's death. The original manuscript, though predates Hebbian learning (1949) and Rosenblatt's perceptrons (1957; and they weren't as sophisticated, only doing feedforward as opposed to recurrent).

Was Turing's B-type neural networks are the earliest neural-like models of computations capable of learning?

Although by modern standard Turing's approach is dated, and has been supplanted by more realistic and general treatements (for instance ones that incorporate dynamic Hebbian updating on weighted connections without a need for central clock synchronization). When did the state of connectionism first surpasss Turing's B-type neural networks? Is there modern treatments of B-type neural networks and their learning abilities?


I am interested in this mostly from the historic perspective, and not in how accurate Turing's model was under current interpertations. Although current knowledge would help answer when other models surpassed Turing's.

  • 1
    $\begingroup$ I am not familiar with B-type neural networks, but they sound a bit like the Boltzmann machine (Hinton & Sejnowski, 1986, PDP, Vol. 1, Chapter 7) and Hopfield networks (Hopfield, 1982, 1984, Proc. Nat. Acad. Sci.) that were developed in the 1980's and continue to be influential in connectionism. $\endgroup$
    – Dan M.
    Jul 13 '12 at 17:38
  • 1
    $\begingroup$ I wasn't familiar with that work by Turing - thanks for posting it. I'm not aware of any neural networks predating that work, but mathematical models of individual neurons date back to 1907 at least. $\endgroup$
    – user3132
    Jun 14 '13 at 13:49

To my knowledge, with respect to the context of the question, the first neural-like model of computations capable of learning – or, for that matter, computational model of neural processing and learning – has been put forward in McCulloch/Pitts (1943), as is also acknowledged in some of the texts about Turing's unorganized machines (›A-/B-type neural networks‹). Turing does not refer to this paper in his 1948 report, however, constituting a central reference for Rosenblatt's perceptrons, the McCulloch/Pitts paper is also regularly ignored in historic accounts of connectionism.

Turing's original unorganized machines are generally recognized as having some limitations with regard to certain logical operations such as XOR (cf. e.g. Teuscher/Sanchez 2001). In this respect (which for brevity may just be called one of historical narrative) they are similar to the perceptron. Thus, it seems safe to assume that at least connectionism surpassed B-type unorganized machines when it was shown how to implement such functions (e.g. in the infamous Minsky/Papert 1969 – mind that this does not necessarily imply a general logical shortcoming of the original models).

The relatively few papers that have addressed the topic over the last 20 years have indeed expressed the observation that Turings unorganized machines have been largely neglected and thus constitute an interesting topic for more detailed analysis – but see Teuscher (2002) for a monograph on the subject.

• Boccato, L., Soares, E. S., Fernandes, M. M. L. P., Soriano, D. C., & Attux, R. (2011). Unorganized Machines: From Turing’s Ideas to Modern Connectionist Approaches. International Journal of Natural Computing Research, 2(4).

• Copeland, B. J., & Proudfoot, D. (1996). On Alan Turing's anticipation of connectionism. Synthese, 108(3), 361–377. doi: 10.1007/BF00413694

• Copeland, B. J., & Proudfoot, D. (1999). Alan Turing's Forgotten Ideas in Computer Science. Scientific American(280), 99–103.

• McCulloch, W. S., & Pitts, W. (1943). A Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics, 5, 115–133.

• Minsky, M., & Papert, S. (1969). Perceptrons. An Introduction to Computational Geometry. Cambridge, Mass.: MIT Press.

• Teuscher, C. (2002). Turing’s Connectionism. An Investigation of Neural Network Architectures. London: Springer-Verlag.

• Teuscher, C., & Sanchez, E. (2001). A Revival of Turing’s Forgotten Connectionist Ideas: Exploring Unorganized Machines. In R. M. French & J. P. Sougné (Eds.), Connectionist Models of Learning, Development and Evolution (pp. 153–162): Springer London.

• Webster, C. S. (2012). Alan Turing’s unorganized machines and artificial neural networks: his remarkable early work and future possibilities. Evolutionary Intelligence, 5(1), 35-43. doi: 10.1007/s12065-011-0060-5

  • $\begingroup$ "the McCulloch/Pitts paper is also regularly ignored in historic accounts of connectionism" Really? To me, McCulloch/Pitts has always been the beginning of the story. Can you refer to a case where they are ignored, and what is considered the start instead? $\endgroup$
    – Martino
    Sep 6 '19 at 12:08
  • $\begingroup$ @Martino: That is a fair question. What I meant is this: McCulloch and Pitts clearly are regularly included in the pertinent technical literature of, let's just say, cognitive science. However, brief historic accounts are frequently given in the literature of a number of disciplines. Pretty often I've seen outlines of connectionism begin with Rumelhart & McClelland's PDP, possibly with recourse to Hebb. $\endgroup$
    – huh
    Apr 22 '20 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.