New answers tagged

1

There are a few issues to consider. If P!=NP, this approach is doomed from the start because you can't try every one of the infinite number of possible algorithms in finite time. That's I think the main deal-breaker here. Unless P=NP, which most people doubt, then the best you could ever do is say "my AI failed to find a polynomial-time algorithm in X ...


1

It is not quite clear to me what you have in mind with infinite size + depth limits for your neural net, but once you have sorted that out you should end up with the following game: Player I plays a program p, and Player II plays a program q. Player 1 wins iff p halts on input q, and answer Yes if q halts on input q, and No otherwise. Player 2 wins iff ...


1

Turing proved that the halting problem is undecidable. No algorithm that you can come up with will solve the halting problem.


0

There are many papers on document structure analysis. I suggest you read them, rather than trying to reinvent the wheel. As they say, a week in the lab can save a day in the library. I think there may be some misconceptions. The way we evaluate neural networks is by trying them on a representative workload to see how well they perform. There are ...


2

My impression is that someone wanted to multiply x by a function f(x) that goes smoothly from 0 to 1, and experimented until they found an expression using elementary functions that did this, with no mathematical reason behind the choice of functions. After choosing a parameter t, let $p_t(x) = 1/2 + (3 / 4t)x - x^3 / (4t^3)$, then $p_t(0) = 1/2$, $p_t(t) =...


3

OP points to a particular implementation of the mish activation function for accuracy specifications, so I had to characterize this first. That implementation uses single precision (float), and is stable and accurate in the positive half-plane. In the negative half-plane, because it uses logf instead of log1pf, relative error quickly grows a $x\to-\infty$. ...


4

With some algebraic manipulation (as pointed out in @orlp's answer), we can deduce the following: $$f(x) = \tanh(\log(1+e^x)) \tag{1}$$ $$ = x\frac{(1+e^x)^2 - 1}{(1+e^x)^2 + 1} = x\frac{e^{2x} + 2e^x}{e^{2x} + 2e^x + 2}\tag{2}$$ $$ = x - \frac{2x}{(1 + e^x)^2 + 1} \tag{3}$$ The expression $(3)$ works great when $x$ is greater than $-5$ with very little ...


3

There's no need to perform the logarithm. If you let $p = 1+\exp(x)$ then we have $f(x) = x\cdot\dfrac{p^2-1}{p^2+1}$ or alternatively $f(x) = x - \dfrac{2x}{p^2+1}$.


Top 50 recent answers are included