I just finished reading a white-paper from a recent AI startup. The company, Deep Brain Chain, wants to distribute neural network training over computers worldwide, using blockchain technology. Here is the whitepaper

From my understanding of machine learning and artificial neural networks, the communication bandwidth is the biggest bottleneck. I would assume that anyone needing infrastructure AI services is using some pretty big models. The bigger the training batch, the more likely it is that data transfer speed is a bottleneck.

Now, attempting to paralyze this across multiple nodes "in the cloud" does not seem like it would scale well. Far, far from linear at least.

Utilizing a distributed network of blockchain mining computers seems like an additional step towards imperfect parallelization. The internet connection speed is bound to be 10-100x times slower than speeds achievable in a cloud datacenter.

Plus, none of the computers in a distributed network of blockchain miners are running the same hardware specifications. I'm not sure, but this seems to add lots of difficulty to the parallelization and efficiency issues.

It is my understanding that neural networks are not highly parallelizable to begin with, much less suited for this type of distribution.

However, I've come here to ask these questions: Are there any (known) training algorithms that can avoid the problems I've posted above? Can there be distributed neural network training systems that utilize a massive network of home PCs and data centers alike?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.