# Tag Info

5

If you're only changing the weights in the last layer, then effectively you have a neural network with a single layer, preceded by some preprocessing step. Single-layer neural networks (also known as perceptrons) aren't so powerful. Why isn't the network learning anything? It doesn't learn much because it can't -- a single-layer network can't learn ...

4

You are right. While you could backpropagate for all samples and then update the weights, you don't have to. Alternatively, you can iterate through the samples and, for each sample, backpropagate for just that sample and then update the weights. The latter turns out to be more effective (empirically). It is called stochastic gradient descent. Today, ...

4

You are calculating the so-called binary cross-entropy. Let $f(\cdot)$ be a sigmoid function. The binary cross-entropy between $y$ and $f(t)$ is $$F(t,y) = H(y,f(t)) = -y\log f(t) - (1-y)\log(1-f(t)).$$ When you compute the partial derivative of $F(t,y)$ with respect to $t$, you get: $$\frac{\partial F(t,y)}{\partial t} = f(t) - y$$ You reach ...

4

We often take the logarithm because: Maximizing $\log \Phi(x)$ is equivalent to maximizing $\Phi(x)$, so in maximum-likelihood problems, we can maximize the log of the likelihood instead of maximizing the likelihood directly and the result will be equivalent. The logarithm converts multiplication to addition, and the derivative of a sum is "nicer" than the ...

4

You can't. The optimal solution is not well-defined. In general there will be a range of tradeoffs between minimizing time and uncompressed size, and you haven't specified anything that would allow uniquely identifying which you want. Your first step is that you need to specify a single objective function, where the goal is to find a solution that ...

4

There is no universal learning rate. It depends on your problem space (Are you solving a problem with many local minima or just one? Does your problem’s solution vary dramatically based on a slight change to your input or does the solution gradually shift?, etc.), and it depends on your network architecture (number of layers, number of hidden layers. Do you ...

4

The cost function $C(w,\lambda)$ is not a function of learning rate $\eta$. You can't compute the gradient of $\eta$. $\lambda$ is part of the cost function. You can indeed compute its gradient and apply SGD on $\lambda$. However, when you do that, $\lambda$ will keep decreasing because its optimal value is obviously $-\infty$. Note that it is possible to ...

3

You say you want to understand how $\lambda$ and $\eta$ affect the cost function. If you hold the weights $w$ fixed, the equation for $C$ tells you how $\lambda$ affects the cost function, and $\eta$ doesn't affect the cost function; it only affects the sequence of steps taken by gradient descent. But the tricky thing is that the final weights $w$ depend ...

3

I guess $f_2(x)$ is some sort of oracle that tells you if $\Phi_2(x)$ is greater than $t$? If the set $\{\mathbf{x} : \Phi_2(x) \le t\}$ is convex, then I think projected gradient algorithm might be helpful. Could you provide more information on $\Phi_2(x)$? The question is kinda confusing with the information provided.

3

I just found out the answer and it's actually pretty simple: while there is a good linear policy for the mountain car task, the value function itself is non-linear. The state space of this task is like a spiral, and there is no linear approximation possible even for a mediocre value function. If you increase the value for going right and reaching the top, ...

2

This is known as stochastic gradient descent. I suggest you read some background material on stochastic gradient descent; then I think you will understand. There is lots written on that subject in many standard places. Compare: In standard gradient descent, we let $E(w)$ denote the total error across all training samples (as a function of the weights $w$)...

2

Just concatenate all of the variables into a single, long vector. In your case, you'll have a $2n$-dimensional vector: $$v = (v_{1,x},v_{1,y},v_{2,x},v_{2,y},\dots,v_{n,x},v_{n,y})$$ where $v_{i,x}$ is the x-coordinate of $v_i$ and $v_{i,y}$ is the y-coordinate of $v_i$. Now treat this as a function of $v$, i.e., f(v) = \sum_{i,j} (||v_i - v_j|| - d_{...

2

Big yes. In fact, gradient descent is one of the fundamental tools used by many supervised learning models. You can check out a very nice docs page about how it is used in scikit-learn.

2

First of all, if we have a descent direction, we can always find a step size $\tau$ that is arbitrary small, such that "the sufficient descent criterion" is satisfied (see the Wikipedia article 'Backtracking line search'). This essentially rules out the infinite loop issue. For example, consider Armijo condition as "the sufficient descent ...

1

No, there are no guarantees. SGD finds a local optimum but not a global optimum, and the solution it finds can be arbitrarily bad, if you have an unfriendly objective function. The only results I've seen that provide guarantees start from strong assumptions about the objective function (e.g., convexity).

1

The relative performance of different optimization algorithms depends a lot on the particular function you are minimizing. We certainly can't tell you whether it is really that good for your particular function without knowing what specific objective function you are looking at, but it certainly seems possible to me. There are some functions where Newton's ...

1

Actually, derivative methods such as random search shorten the time allocated for function evaluation if the problem is big. On the other hand, derivative-free methods take much time to complete function evaluation that leads to a dramatic increase in optimization time.

1

If you overshoot by 10% you are fine, for example if the solution is 0 and the starting value 1000, you go to -100, +10, -1, +0.1 etc. If you overshoot by 80%, you converge but much smaller. If you overshoot by 110% you go from 1,000 to -1,100, +1,210, -1,331 etc. - divergence.

1

You would have to look at multiobjective optimisation, with these methods you can optimise two functions at the same time. Just be aware that, depending on the method that you end up using, you might have in the end a number of solutions equally good (trade offs) which you would have to choose from.

Only top voted, non community-wiki answers of a minimum length are eligible