Constraints on dataset size for AI-assisted connectome reconstruction

Question

From an exchange with Dr. Seung, an expert on connectomics, I learned that one of his former PhD students Viren Jain is leading the Connectomic effort at Google using supervised learning methods. Now, supervised deep learning methods usually require large amounts of data to succeed. (I know this based on experience in industry as well as the deep learning literature.) However, due to the time and effort of human annotation the datasets for circuit reconstruction are really small:

It takes about six months for humans to annotate 1 cubic millimeter of brain volume.

In fact, Dr. Seung informed me that the following dataset from a circuit reconstruction competition in 2016 which is less than five gigabytes in total is basically state of the art.

Given that augmenting training data with synthetic data is frequently used for computer vision problems, I am led to the following questions:

1. Might it theoretically be possible to augment human-annotated training data with morphologically similar synthetic data?
2. What methods in the near future might allow us to significantly increase the size of human-annotated training data?

References:

1. Michał Januszewski, Jörgen Kornfeld, Peter H. Li, Art Pope, Tim Blakely, Larry Lindsey, Jeremy Maitin-Shepard, Mike Tyka, Winfried Denk & Viren Jain. High-precision automated reconstruction of neurons with flood-filling networks. Nature. 2018.
2. Luis Perez and Jason Wang. The Effectiveness of Data Augmentation in Image Classification using Deep Learning. Arxiv. 2017.
3. Alberto Bailoni, Constantin Pape, Steffen Wolf, Thorsten Beier, Anna Kreshuk, Fred A. Hamprecht. A Generalized Framework for Agglomerative Clustering of Signed Graphs applied to Instance Segmentation. Arxiv. 2019.
4. EyeWire: mapping neurons. https://science.eyewire.org/science-mapping-neurons Accessed 21 August 2019.

Answer 1

Improving labeling

Increasing the amount of data that humans can annotate in a reasonable amount of time (your second question), one essentially needs to develop better labeling tools. Typically, this can either involve nicer interfaces or training ML algorithms to propose labels which humans can then more easily confirm or improve.

I recently attended a talk by Moritz Helmstaedter in which he presented work in both these directions. If I remember their approach correctly, they first had people trace axons and dendrites in serial electron microscope sections. Based on these traces, they then found the membrane around these axons/dendrites using an algorithm, which gave them a proposal for the segmentation. For the tracing part, they developed a nicer interface (see this paper): instead of showing only slices along the cardinal directions to their labelers, they developed a "flight mode" in which the labeler can see the data from an ego-centric view in the current direction of the neurite they are tracing. This gave them a 4- to 13-fold speedup.

Synthetic data

Concerning your first question, I'm not aware of any work on synthetic datasets in this area. I imagine generating one would be quite difficult, since cell morphology is complex. In order to generate synthetic EM images, you'd essentially need a detailed description of the cell morphologies in a cube of cortical tissue - and we usually get these descriptions from EM data. Alternatively, you could try to use deep learning (autoencoders or generative adversarial networks) to produce such synthetic data, but this recent review concluded that these will most likely not give you the quality you would need for training, since

the artificial images would likely either fail in subtle ways to be truly realistic or would fail to capture the full diversity of real-world data.

For now it seems the field will have to aim for better / faster human annotations instead of synthetic data.