I am a PhD student in Computer Science who switched his PhD a little bit towards ML algorithms combined with something else... I am an expert in that something else, say image processing, but not an expert in Machine Learning. What should I read/learn to get the ML mathematics right? Especially for adaptive learning algorithms.
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ What do you mean by "adaptive learning algorithms"? and out of curiosity, What is the "something else"? $\endgroup$– NickSep 16, 2012 at 21:02
-
$\begingroup$ something else = semantic multimedia stuff... as for adaptive - well genetic algorithms are adaptive... there is a process through which you can adapt to changes... but there also algorithms I hardly know about that are still called adaptive...just to give some examples: slideshare.net/gladysCJ/slidesPhDthesisGCastillo or ms.cc.sunysb.edu/~ecarcelespov/otherfiles/learning.pdf and so on - you find lots of examples on google - however most of them already have lots of mathematics and I'm left scratching my head as to what math prerequisites I should master... $\endgroup$– paxRomanSep 16, 2012 at 21:44
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The precise mathematical skills that you'll need for your studies will vary depending on exactly which area of ML you'll be working in. In general though Bayes theorem, probability theory and statistics will be your constant companion. Getting a good grasp of these is extremely helpful when it comes to fully understanding how learning algorithms work, and how they can be modified.
Partial differential equations, linear algebra, and mathematical induction are also very important. However beyond these it is difficult to say exactly which areas you should brush up on. In my opinion it depends on what problem you are trying to solve.
For instance is it a regression problem? Is it a classification problem? Are you clustering data? You mention adaptive algorithms, but many ML algorithms could be considered adaptive. As you become familiar with the ML field you'll find that people use different terms to describe exactly the same thing :) a little frustrating to be honest! But the second link you provided is from a non - computer science journal, so just be aware the definition of adaptive in this paper may mean something different in ML. For instance I don't work in adaptive algorithms, but my work has to adapt to high throughput data streams so is most certainly adaptive. But it falls under the category of Semi-supervised learning.
I would recommend taking a look at the following books for more information:
Machine Learning by Tom M. Mitchell, 1997 - This is a great and relatively easy going introduction to machine learning. Despite being over ten years old, this book covers the key topic areas, and introduces you to important concepts. Indirectly this will introduce you to the mathematics of Machine Learning. You'll be able to tell from this book what type of mathematics skills your particular problem may require.
Artificial Intelligence: A Modern Approach by Stuart Russell & Peter Norvig, 3rd Edition, 2009 - All encompassing text (1000+ pages) covers machine learning in great detail. Authoritative text.
Pattern Recognition and Machine Learning (Information Science and Statistics), Christopher M. Bishop, 2006 - Excellent authoritative text that goes in to ML in great depth. This includes introductory chapters that get you up to speed with the mathematics required to understand learning algorithms (including probability). Can be hard going if you have no mathematics experience.
To support any reading you do I suggest you take a look at the Andrew Ng machine learning course on Coursera. I watched some of these before starting my own study, so too did many of my friends.
