I have a question regard online learning with SGD. Is there a way to give a statistical guarantee that the value obtained after $n$ samples deviates at most $\epsilon$ from the real value?
Thank you guys.
Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. It only takes a minute to sign up.Sign up to join this community
No, there are no guarantees. SGD finds a local optimum but not a global optimum, and the solution it finds can be arbitrarily bad, if you have an unfriendly objective function.
The only results I've seen that provide guarantees start from strong assumptions about the objective function (e.g., convexity).