Skip to main content
8 votes
Accepted

Why do we use the log in gradient-based reinforcement algorithms?

We often take the logarithm because: Maximizing $\log \Phi(x)$ is equivalent to maximizing $\Phi(x)$, so in maximum-likelihood problems, we can maximize the log of the likelihood instead of ...
D.W.'s user avatar
  • 162k
6 votes

Find minimum of a function only knowing the ordering of a set of input points

You can't. The function could be anything. For example, consider $$f(x) = \begin{cases} g(x) &\text{if }x \ne \alpha\\ -10^{100} &\text{if } x = \alpha \end{cases}$$ where $\alpha \in \...
D.W.'s user avatar
  • 162k
5 votes
Accepted

How to show that cross entropy is minimized?

You are calculating the so-called binary cross-entropy. Let $f(\cdot)$ be a sigmoid function. The binary cross-entropy between $y$ and $f(t)$ is $$ F(t,y) = H(y,f(t)) = -y\log f(t) - (1-y)\log(1-f(...
user172818's user avatar
5 votes

Why updating only a part of all neural network weights does not work?

If you're only changing the weights in the last layer, then effectively you have a neural network with a single layer, preceded by some preprocessing step. Single-layer neural networks (also known as ...
Yuval Filmus's user avatar
4 votes
Accepted

Why update weights and biases after training a Neural Network on whole set of training samples

You are right. While you could backpropagate for all samples and then update the weights, you don't have to. Alternatively, you can iterate through the samples and, for each sample, backpropagate ...
D.W.'s user avatar
  • 162k
4 votes
Accepted

Is there a universal learning rate for NeuralNetworks?

There is no universal learning rate. It depends on your problem space (Are you solving a problem with many local minima or just one? Does your problem’s solution vary dramatically based on a slight ...
ScottK's user avatar
  • 208
4 votes

What are the challenges of using gradient descent on hyper parameters λ and η to find out their optimum values?

The cost function $C(w,\lambda)$ is not a function of learning rate $\eta$. You can't compute the gradient of $\eta$. $\lambda$ is part of the cost function. You can indeed compute its gradient and ...
user172818's user avatar
3 votes
Accepted

What does RSGD stand for?

It's probably Riemannian stochastic gradient descent (R-SGD), (Stochastic Gradient Descent on Riemannian Manifolds). You'll find several articles on the subject by searching for this term.
John Kemeny's user avatar
  • 16.7k
3 votes

What are the challenges of using gradient descent on hyper parameters λ and η to find out their optimum values?

You say you want to understand how $\lambda$ and $\eta$ affect the cost function. If you hold the weights $w$ fixed, the equation for $C$ tells you how $\lambda$ affects the cost function, and $\eta$ ...
D.W.'s user avatar
  • 162k
3 votes

Is it possible to solve the Mountain Car reinforcement learning task with linear Q-Learning using the state as direct input?

I just found out the answer and it's actually pretty simple: while there is a good linear policy for the mountain car task, the value function itself is non-linear. The state space of this task is ...
rcpinto's user avatar
  • 471
2 votes

Find minimum of a function only knowing the ordering of a set of input points

The answer by @D.W. is correct that any search space will need some structure in order to make progress. However, I think it would be pessimistic to conclude that the No Free Lunch theorems apply; ...
Warbo's user avatar
  • 632
2 votes

Is SGD used in machine learning libraries?

Big yes. In fact, gradient descent is one of the fundamental tools used by many supervised learning models. You can check out a very nice docs page about how it is used in scikit-learn.
Victor Valente's user avatar
2 votes

Calculating gradient in a neural net using batches

This is known as stochastic gradient descent. I suggest you read some background material on stochastic gradient descent; then I think you will understand. There is lots written on that subject in ...
D.W.'s user avatar
  • 162k
2 votes

MDS minimization with gradient descent

Just concatenate all of the variables into a single, long vector. In your case, you'll have a $2n$-dimensional vector: $$v = (v_{1,x},v_{1,y},v_{2,x},v_{2,y},\dots,v_{n,x},v_{n,y})$$ where $v_{i,x}$...
D.W.'s user avatar
  • 162k
2 votes

Line-search does not guarantee convergence so how to use it?

First of all, if we have a descent direction, we can always find a step size $\tau$ that is arbitrary small, such that "the sufficient descent criterion" is satisfied (see the Wikipedia ...
baris_esmer's user avatar
2 votes
Accepted

How does Gradient Descent treat multiple features?

That's correct. The derivative of $x_2$ with respect to $x_1$ is 0. A little context: with words like derivative and slope, you are describing how gradient descent works in one dimension (with only ...
D.W.'s user avatar
  • 162k
1 vote

SGD statistical guarantee

No, there are no guarantees. SGD finds a local optimum but not a global optimum, and the solution it finds can be arbitrarily bad, if you have an unfriendly objective function. The only results I've ...
D.W.'s user avatar
  • 162k
1 vote
Accepted

Is Newton's algorithm really this much better than conjugate gradient descent?

The relative performance of different optimization algorithms depends a lot on the particular function you are minimizing. We certainly can't tell you whether it is really that good for your ...
D.W.'s user avatar
  • 162k
1 vote

What is the difference between derivative free optimization and derivative optimization in terms of advantages/disadvantages?

Actually, derivative methods such as random search shorten the time allocated for function evaluation if the problem is big. On the other hand, derivative-free methods take much time to complete ...
m. öztürk's user avatar
1 vote

Gradient descent overshoot - why does it diverge?

If you overshoot by 10% you are fine, for example if the solution is 0 and the starting value 1000, you go to -100, +10, -1, +0.1 etc. If you overshoot by 80%, you converge but much smaller. If you ...
gnasher729's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible