The task: fly through a vast network of neuron models and visualize the propagation of “signals” across the network. My solution: generate the network procedurally, and the rest will follow.
Let’s begin with the result before looking behind the scenes.
Challenge
The award-winning VR experience Mind The Brain! by mYndstorm productions is a dream-like, non-scientific journey through the brain. Some section of the experience are programmed to react to the neural activity of the viewer. This is done through a brain-computer-interface (BCI) that converts a live electro encephalogram into usable variables for the program code.
One key scene of the experience, called “Microscopic Space”, one I was asked to create, was a minute-long flight (11 minutes to be precise) through a network of neuron 3D models. Additionally the network had to be traversed by lights, visualizing the propagation of electric signals through the brain. The frequency and speed with which these lights travel across the network had to depend on the brain activity of the viewer.
A first attempt at hand crafting a network from my 5 base neuron 3D models in a 3D application first, and importing the whole network scene into Unity was not fruitful for two reasons:
- It was a cumbersome process to arrange the six neuron models in a connected way, too laborious for the intended duration of the scene.
- It did was very polygon heavy, and with dozens and hundreds of individual neuron model copies applying different level-of-detail (LOD) versions would again be a lot of work.
Approach
It quickly became clear that generating the network procedurally by replicating instances of the base models instead of copying them was the way forward. This would save time and polygons (because LOD stand-ins had only be applied to the six base models before replication).
But that was still easier said (or thought) than done.
The six models were similar in that they each had several dendrites and one axon that would connect to the body of a neighbouring cell. But the axons were of different length, the dendrites in different directions, and every neuron had to be rotated in different angles to connect without (clearly visible) intersection of dendrites and axons.
Heureka!
The central idea that solved the problem was to move the origin of a neuron from the center of its own cell body to beyond its axons where the parent neuron’s cell body would be. In the center of each the neuron would then be three null locators pointing in the direction where a child neuron could dock.
By parenting the children (with their offset origins) to the null locators (at the body centre) the child neurons would automatically “grow away” from the parent in allowed directions.
Which in the end looks like this:
C# and Unity implementation
Because we used Unity’s own version control system I cannot easily share a link to GitHub repository. But following is an abridged version of the relevant C# code, along with a look at the central elements in Unity. I stripped the code of references to our proprietary camera control system, and of the BCI connection.
Helper methods
Nothing to write home about in these two helper functions that helped randomizing certain aspects of the network.
public static class NeuronHelper
{
// Returns a random element from a list.
public static T GetRandomElement<T>(this List<T> list)
{
return list[Random.Range(0, list.Count)];
}
// Returns a random variation of a base value. Variance in percent.
public static float GetRandomVariation(float baseValue, int variance)
{
return baseValue + baseValue * Random.Range(-variance, variance) / 100f;
}
}
Neuron network controller
The neuron network controller class…
- provides an interface for the user to set parameters for the network generation through public variables,
- orchestrates the generation of neurons by instantiating and parenting them properly,
- initiates the “firing” animation on randomly chosen neurons across the network.
public class NeuronNetworkController : MonoBehaviour
{
internal Transform NetworkTransform;
// Chance for a neuron to fire randomly
[Range(0, 1000)]
public int FireChance = 10;
// Chance for light to trigger next neuron to fire
[Range(0, 100)]
public int TriggerChance = 20;
// Speed of movement of the lights across the network
[Range(0.25f, 1.5f)]
public float LightSpeed = 0.75f;
// Variance in speed
[Range(0, 100)]
public int LightSpeedVariance = 50;
// Number of neurons in the network
[Range(0, 5000)]
public int MaxNeuronCount = 2000;
// Probability of a child neuron being generated for each potential direction
[Range(0, 100)]
public int ChildSpawnProb = 15;
public int RandomSeed = 1015;
// List of neuron prefab models from which to build the network
public List<GameObject> NeuronPrefabs;
// List of all generated neurons in the network
List<Neuron> Neurons = new List<Neuron>();
public float NeuronFireStartPosition = 5f;
public int NeuronCounter => Neurons.Count;
// Create singleton of this class
public static NeuronNetworkController Instance;
public void Awake()
{
Instance = this;
NetworkTransform = GetComponent<Transform>();
}
public void CreateNeuron(Transform newNeuronTransform, Neuron parent)
{
if (NeuronCounter > MaxNeuronCount)
{
return;
}
var nextChild = Instantiate(NeuronPrefabs.GetRandomElement(),
newNeuronTransform.position,
newNeuronTransform.rotation,
NetworkTransform
).GetComponent<Neuron>();
nextChild.NeuronNumber = NeuronCounter;
Neurons.Add(nextChild);
nextChild.Parent = parent;
parent.Children.Add(nextChild);
}
private void Update()
{
FireNeurons();
}
public void FireNeurons()
{
if (UnityEngine.Random.Range(0, 100) < FireChance % 100)
{
Neurons.GetRandomElement().Fire(NeuronHelper.GetRandomVariation(LightSpeed, LightSpeedVariance));
}
for (int i = 1; i < Math.Floor(FireChance / 100f); i++)
{
Neurons.GetRandomElement().Fire(NeuronHelper.GetRandomVariation(LightSpeed, LightSpeedVariance));
}
}
}
Single neuron
The neuron class…
- initiates the “reproduction” of child neurons on
Start(), - generates between one and three children by calling
CreateNeuronon the neuron network controller, - handles the “firing” animation,
- triggers neurons up or down the tree to start their “firing” animation.
public class Neuron : MonoBehaviour
{
// New child child neurons can be placed at these coordinates with these orientations.
public List<Transform> Directions;
public int NeuronNumber { get; set; }
internal List<Neuron> Children = new List<Neuron>();
internal Neuron Parent;
// Neuron firing is done via an animator
internal Animator Animator;
private void Awake()
{
Animator = GetComponent<Animator>();
Animator.enabled = false;
}
void Start()
{
GenerateChildren();
}
private void GenerateChildren()
{
UnityEngine.Random.seed = NeuronNetworkController.Instance.RandomSeed + NeuronNumber;
foreach (var direction in Directions)
{
if (UnityEngine.Random.Range(0, 100) < NeuronNetworkController.Instance.ChildSpawnProb)
{
NeuronNetworkController.Instance.CreateNeuron(direction, this);
}
}
if (Children.Count < 1)
NeuronNetworkController.Instance.CreateNeuron(Directions.GetRandomElement(), this);
}
internal void Fire(float speed)
{
Animator.enabled = true;
Animator.SetFloat("MotionSpeedFactor", speed);
if (UnityEngine.Random.Range(0, 2) == 1)
Animator.SetTrigger("FireNeuronDown");
else
Animator.SetTrigger("FireNeuronDown");
}
internal void TurnOffAnimator()
{
Animator.enabled = false;
}
public void TriggerDown()
{
if ((Children.Count > 0) && (UnityEngine.Random.Range(0, 100) < NeuronNetworkController.Instance.TriggerChance))
{
var rndChild = Children.GetRandomElement();
rndChild.Animator.enabled = true;
rndChild.Animator.SetFloat("MotionSpeedFactor", Animator.GetFloat("MotionSpeedFactor"));
rndChild.Animator.SetTrigger("FireNeuronDown");
}
}
public void TriggerUp()
{
if ((Parent != null) && (UnityEngine.Random.Range(0, 100) < NeuronNetworkController.Instance.TriggerChance))
{
Parent.Animator.enabled = true;
}
Parent?.Animator.SetFloat("MotionSpeedFactor", Animator.GetFloat("MotionSpeedFactor"));
Parent?.Animator.SetTrigger("FireNeuronUp");
}
}
Unity
Here is a look at Unity side of the network generation and animation.
On the interface of the neuron network controller script component, several parameters can be set: the maximum number of neurons, the probability for more than one child per neuron (which translates to the density of the network), the seed to create fixed random networks. Also, the component references the neuron prefabs that make up the network.
The most important part of individual neuron for the network generation is the script component which takes a list of Unity objects, namely the directions that the child neurons would be facing.
The neuron shader graph not only handles the general textures of the neurons but also prepares for the traveling light shifting a gradient across the model via UV animation.
Finally, the neuron animator starts and stops the “firing” animation in the appropriate direction.
