Algorithm

Basically does Vector Similarity Search

What does the hippocampus do? Here’s an attempt to approximate the algorithm:

Input: Some vector $i_t$

Output: Some vector $o_t$, in the same space as $i_t$

Storage Requirements: A list of Variational Vectors. This can be a top-k list.

Algorithm:

Loop:

1. At each timestep, get input
2. Record an element-wise Running Average and a Running Variance of the inputs. These can be exponentially-smoothed averages, which can be computed efficiently.
3. If the overall running variance drops below some threshold $\tau_r$, move to the retrieval step
4. If the overall running variance drops below some threshold $\tau_r$, move to the storage step
5. Else either of those, go to sampling

Retrieval:

1. Search the stored vector for the closest vector. Distance will be a probability, because they’re variational vectors.
2. If the distance falls under some threshold $\tau_c$, go to sampling with this closest vector

Storage:

1. Add the running average vector to the storage
2. Keep only the top k vectors, sorted by minimum variance