1
$\begingroup$

Given a neural network(assuming no regularisation/dropout), I want to determine the computational cost of doing a forward and a backward pass of a datapoint. I want the measure to be of independent of the programming language/operating system involved. I can think of it as a graph model, with some activations applied on nodes, then I think it can be estimated by hand. Also is there any online tool/package which would do the estimate if I input the neural network ? I am new to computer science, so probably it is a very naive question to most of the users, I am offering my apologies it is so.

$\endgroup$
1
  • 1
    $\begingroup$ Hava Siegelmann and her collaborators have done a lot of work on computational complexity of specific types of neural networks. The early work is collected in her book, but much of it is based on papers that you can find at Siegelmann's website or elsewhere. I don't think this work is directly relevant to your question, which is very specific and may not map onto the sorts of NNs that Siegelmann et al analyze. However, you might get insight from her methods, or find references to more relevant work. $\endgroup$ – Mars Oct 13 '20 at 16:45
1
$\begingroup$

It takes $O(n)$ time, where $n$ is the size of the neural network (e.g., number of nodes plus number of connections; in a graph model, number of vertices plus number of edges).

$\endgroup$
1
  • $\begingroup$ Can you add a source or reference? $\endgroup$ – Rexcirus Jan 10 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.