New answers tagged neural-networks
1
There are a few issues to consider.
If P!=NP, this approach is doomed from the start because you can't try every one of the infinite number of possible algorithms in finite time. That's I think the main deal-breaker here. Unless P=NP, which most people doubt, then the best you could ever do is say "my AI failed to find a polynomial-time algorithm in X ...
1
It is not quite clear to me what you have in mind with infinite size + depth limits for your neural net, but once you have sorted that out you should end up with the following game:
Player I plays a program p, and Player II plays a program q.
Player 1 wins iff p halts on input q, and answer Yes if q halts on input q, and No otherwise. Player 2 wins iff ...
1
Turing proved that the halting problem is undecidable.
No algorithm that you can come up with will solve the halting problem.
0
There are many papers on document structure analysis. I suggest you read them, rather than trying to reinvent the wheel. As they say, a week in the lab can save a day in the library.
I think there may be some misconceptions. The way we evaluate neural networks is by trying them on a representative workload to see how well they perform. There are ...
2
My impression is that someone wanted to multiply x by a function f(x) that goes smoothly from 0 to 1, and experimented until they found an expression using elementary functions that did this, with no mathematical reason behind the choice of functions.
After choosing a parameter t, let $p_t(x) = 1/2 + (3 / 4t)x - x^3 / (4t^3)$, then $p_t(0) = 1/2$, $p_t(t) =...
3
OP points to a particular implementation of the mish activation function for accuracy specifications, so I had to characterize this first. That implementation uses single precision (float), and is stable and accurate in the positive half-plane. In the negative half-plane, because it uses logf instead of log1pf, relative error quickly grows a $x\to-\infty$. ...
4
With some algebraic manipulation (as pointed out in @orlp's answer), we can deduce the following:
$$f(x) = \tanh(\log(1+e^x)) \tag{1}$$
$$ = x\frac{(1+e^x)^2 - 1}{(1+e^x)^2 + 1} = x\frac{e^{2x} + 2e^x}{e^{2x} + 2e^x + 2}\tag{2}$$
$$ = x - \frac{2x}{(1 + e^x)^2 + 1} \tag{3}$$
The expression $(3)$ works great when $x$ is greater than $-5$ with very little ...
3
There's no need to perform the logarithm. If you let $p = 1+\exp(x)$ then we have
$f(x) = x\cdot\dfrac{p^2-1}{p^2+1}$ or alternatively $f(x) = x - \dfrac{2x}{p^2+1}$.
Top 50 recent answers are included
