I have a question regard online learning with SGD. Is there a way to give a statistical guarantee that the value obtained after $n$ samples deviates at most $\epsilon$ from the real value?
Thank you guys.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
No, there are no guarantees. SGD finds a local optimum but not a global optimum, and the solution it finds can be arbitrarily bad, if you have an unfriendly objective function.
The only results I've seen that provide guarantees start from strong assumptions about the objective function (e.g., convexity).
