Summary

This podcast (https://www.eetimes.com/podcasts/brainchips-ip-for-targeting-ai-applications-at-the-edge/) was a very interesting listen. The BrainChip folks talked about their technology and the business components of how they hope that technology will be used. The company sells IP for larger systems that accelerate AI workloads at the edge. They specifically implement a neural network they call Temporal Encoded Neural Networks or TENNs. These are spike(ish) networks that get the benefit of both CNNs and RNNs by encoding the space as a variable as a context input (similar to an RNN) but with CNN-like performance and parallelization but with much smaller networks. It all sounded a little too good to be true. Their core focus seemed to be on applications with sequential data where you'd typically see RNN-like networks (LSTMs, GRUs, etc.) like eye tracking, audio denoising, and ASR. They also stated it works well for small language models. They seemed to be top performers in a lot of competitions.

Technology Used

Legendre Multipliers

There were a couple of interesting technologies used. They discussed using Legendre multipliers which constrain neural networks physically. The example that was used was ghost cars, where in CNNs an object is identified and then tracked afterward, but that object tracking can turn off and on in ways that, if you were following the rules of physics, could never happen. Legendre multipliers impose physical limits on things so this sort of ghost effect doesn't happen. It's always cool to have rules like this, but it's also fun with neural networks to see interpretation without these limitations. That's kind of the beginning of creativity in some aspects where you decide to question the rules. In the technical recap, the specialist says he likes when AI hallucinates because it's being creative in applying new mappings to previously seen results. I liked this and think it's an interesting bit of all this emerging technology when we start trying to understand where heuristics are just being followed and where true creativity is happening.

Bio-Inspired Approach (Augmented SNN)

It also seems that they use a bio-inspired approach and try to take from the mechanisms of the brain where it benefits and use a more digital architecture approach where it's appropriate. It's a worthy endeavor to recreate an abstract version of the brain, but this feels very far off and they're looking at commercialization. One area where this was clear was in using an SNN like we talked about in class but with amplitudes. The brain uses spikes without regard to the shape of the spike or the amplitude, just when it happened. They're essentially going the opposite way as NVIDIA in increasing the amount of bits they're using for a more bio-inspired design rather than a pure emulation of the brain.

NoC

They also talked about how they use a mesh network-on-chip, something we've discussed in class. I didn't show a ton of interest in my networking class, as I never thought I'd get into IT, when I took it a million years ago, but I wish I had viewed it as more of a conceptual undertaking. Probably a lesson there for me to focus because you never know when these things are going to pop back up. Neural networks were a curiosity once, and now they run the world.

Parallelization → Localization

As discussed, the authors competed with their algorithm in a bunch of competitions and did well in all. They also outperformed Mamba, a similar algorithm built for GPUs I need to look into with more state space tracking, in some LLM and SLM competitions. In this, they discussed how the natural progression of ML required huge amounts of parallelization where all data was sent through entire networks every time. The brain, on the other hand, uses only parts of its network and saves power, and I believe the authors are attempting a similar approach, which is also biologically inspired. Seems great for power if it's achievable, i.e., doesn't require too much overhead.

Commercialization

It seems as though they may be having a hard time getting the Akida adopted even with all its success and state-of-the-art status. They even brought up one field (biotech, that I happen to work in) that could particularly benefit from a high-performance, low-power system like this. As they discussed, a huge development ecosystem is needed for adoption for a commercial chip like this. People won't use things they don't understand, and given the presentation and even the expert's discussion at the end on the confusing elements of this, this chip is hard to understand. How is it getting all the benefits from both RNNs and CNNs without any of the tradeoffs? Does the SNN-ish nature of this with the spike amplitudes make it worth it with the sparsity benefits, or is a GPU with all of its modern tools and ecosystem still better? Does this even matter? People use AI all the time without understanding transformers, but for commercial use, I think companies will likely need to understand the network, and tools around it will need some time to mature outside niche academic interest. Overall sounds very promising.