Neuroscience inspired Computer Vision

Source: https://www.pinterest.com/explore/visual-cortex/

Having read the profound master piece “When breath becomes air”, by Neuroscientist – surgeon Paul Kalanithi, I was curious about how neuroscience could contribute to AI (Computer vision in particular). 

Then, I found an comprehensive article in Neuron Review journal (written by Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, Matthew Botvinick) titled “Neuroscience inspired Artificial Intelligence”.  Here goes a brief excerpt of concepts I found inspiring in that article, related to computer vision.

Past

CNNS

• How visual input is filtered and pooled into simple and complex areas of cells in area V1in visual cortex
• Hierarchical organization of mammalian cortical systems 

Object recognition 

• Transforming raw visual input into increasingly complex set of features - To achieve invariance towards pose, illumination and scale

Present

Attention 

• Visual attention shifts strategically among different objects (no equal priority for all objects) - To ignore irrelevant objects in a given scene in the presence of a clutter, multi object recognition, image to caption generation, generative models to synthasize images 

Future 

Intuitive understanding of physical world 

• Interpret and reason about scenes by decomposing them into individual objects and their relations 
• Redundency reduction (encourages the emergence of disentangled representations of independent factors such as shape and position) - To learn objectness, construct rich object models from raw inputs using deep generative models, E.g., Variational auto encoder 

Efficient Learning 

• Rapidly learn new concepts from only a handful of examples (Related with Animal learning, developmental psychology) 
• Characters challenge - distinguish novel instances of an unfamiliar hand written character from another - "Learn to learn”  networks

Transfer Learning

• Generalizing or transferring generalized knowledge gained in one context to novel previously unseen domains (E.g., Human who can drive a car drives an unfamiliar vehicle) - Progressive networks 
• Neural coding using Grid codes in Mammalian entorhinal cortex - To formulate conceptual representations that code abstract, relational information among patterns of inputs (not just invariant features) 

Virtual brain analytics 

• Increase the interpretability of AI computations, Determine response properties of units in a neural networks 
• Activity maximization - To generate synthetic images by maximizing the activity of certain classes of unit 

From AI to neuroscience

• Enhancing performances of CNNs has also yielded new insights into the nature of neural representations in high-level visual areas. E.g., 30 network architectures from AI to explain the structure of the neural representations observed in the ventral visual stream of humans and monkeys