The tag has no usage guidance.

35 questions
Filter by
Sorted by
Tagged with
6 views

### Prophet Model in Batch Implementation

I am newby at Prophet model. I was wondering Prophet uses stochastic or batch approach while in training. If it uses stochastic that means there is nothing to do convert and compute mini batch ...
17 views

### SGD statistical guarantee

I have a question regard online learning with SGD. Is there a way to give a statistical guarantee that the value obtained after $n$ samples deviates at most $\epsilon$ from the real value? Thank you ...
37 views

### In a machine learning system, why use differentially private SGD if our input data is already perturbed by a DP mechanism?

I'm trying to implement my own version of a deep neural network with differential privacy to preserve the privacy of the parties involved in the training dataset. I'm using the method by Abadi et al. ...
21 views

46 views

### Is Newton's algorithm really this much better than conjugate gradient descent?

I have a function I'm minimizing. I'm using conjugate gradient descent and the Newton algorithm. I am experiencing that the Newton algorithm is absurdly faster. Like, it finishes it 5-6 iterations, ...
119 views

### Line-search does not guarantee convergence so how to use it?

Line-search/backtracking in gradient descent methods essentially boils down to picking the current estimate $\theta_n$ (which depends on the stepsize $\gamma$ and the prior estimate $\theta_{n-1}$) by ...
140 views

### What is the difference between derivative free optimization and derivative optimization in terms of advantages/disadvantages?

I understand the basic operation of the algorithms however i'm unclear as to when to use one over the other and what advantages/disadvantages they offer over each other. Also as an aside, if anyone ...
52 views

### Solving analytic gradient of loss function for neural networks [closed]

Please note that I am talking in about theory rather than ''what someone would do in a real, practical situation''. Given a multi-layer Perceptron with at least 1 hidden layer, and sigmoid (or other ...
45 views

### Is SGD used in machine learning libraries?

SGD (Stochastic Gradient Descent) is used in most libraries of different programming languages. Is it also used in machine learning libraries?
53 views

### How is momentum an approximation of Hessian based optimization?

In the answer to "what is the Hessian" at this site: https://stackoverflow.com/questions/23297090/how-calculating-hessian-works-for-neural-network-learning the person answering the question ...
26 views

### Inverse kinematics step

I am working on an implementation of inverse kinematics using the jacobian transpose method. The implementation seems to be working as it does find the "theta" vector, although sometimes it might take ...
71 views

### Why don't Artificial Neural Networks Commonly Diverge?

Introduction: I'm using divergence here as to mean that the gradient is getting further and further from zero in stochastic gradient descent. I've written my own feed-forward neural network and tried ...