We have a new paper with @marcusghosh @GabrielBena on why we have nonlinearity in multimodal circuits.
My lab page has links to journal, preprint, code, talk on youtube, etc.:
http://neural-reckoning.org/pub_multimodal.html
TLDR: Why is it a question that we have nonlinearity in these circuits? Well, the classical multimodal task can be solved with a linear network, so maybe those nonlinear neurons aren't actually needed?
We find that nonlinearity is very important when you consider an extension of the classical multimodal task, embedded into a noisy background, and you don't know when the multimodal signal is active. We think this is a more realistic scenario, for example in a predatory-prey interaction.
We're following up with two additional projects at the moment, looking at what happens when you have even more extended temporal structure in the task (preview: you can still do very well with fairly simple feedforward or recurrent circuits), and when you model agents navigating a multimodal environment (e.g. foraging, hunting; early results suggest recurrent circuits are more robust).
I don't think we can fully understand multimodal circuits until we start looking at more realistic, temporally extended tasks. Exciting times ahead, and we'd be happy to work with any experimental groups interested in pursuing this. Please get in touch!