I have read some articles which state that basic algorithms such as dynamic programming , graph algorithms etc are not required int machine learning fields such as deep learning , reinforcement learning etc. Is it true that learning basic algorithms is not necessary for learning machine learning concepts ?

  • 1
    $\begingroup$ Dynamic programming is used. $\endgroup$ – pdexter Jul 15 '16 at 14:17
  • 1
    $\begingroup$ Depends on what you're deep learning. My friend was just telling me about a deep learning problem for matching different information about a person from noisy sources, that used neural nets as classifiers, but involved a lot of fancy graph theory to know what to feed the algorithm. I think knowing learning algorithms without the solid, deterministic algorithms is akin to... knowing how to build a V8 engine, but not knowing how to make an axel or wheels to make it go anywhere. $\endgroup$ – Alex Meiburg Jul 15 '16 at 16:32
  • 1
    $\begingroup$ This is a broad question, as machine learning spans many different things, and "basic algorithms" spans many topics as well. It might help if you edited it to clarify what aspects of machine learning you are thinking of. Are you talking about just being a user of machine learning algorithms, or designing new machine learning algorithms? Are you asking about what's needed to take a course in the subject, to use it in software development, to do research in that area, something else? Anything you can do to narrow down the question might help; otherwise, the answer is probably "it depends". $\endgroup$ – D.W. Jul 15 '16 at 23:34
  • $\begingroup$ "basic algorithms such as dynamic programming , graph algorithms" -- you name a technique and a whole class of algorithms. $\endgroup$ – Raphael Jul 16 '16 at 13:19
  • 1
    $\begingroup$ You seem to be asking "are basic algorithms really important for learning about these other algorithms?" I don't know how to respond to that. $\endgroup$ – Raphael Jul 16 '16 at 13:21

I would say that machine learning is a part of computer science.

Machine learning is defined by Tom Mitchell:

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

This means that an algorithm on a defined task (e.g. classification, regression) on which you define a measure (e.g. accuracy, mean squared error) is a machine learning algorithm when it improves with more data.

Which algorithms are used?

  • Deep learning: This is machine learning with neural networks. There is much more in machine learning than neural networks, but currently they give the most astonishing results and a lot of progress is happening with neural networks. Neural networks are trained with gradient descent; a numeric optimization algorithm known for a long time.
  • Dynamic programming: A very simple way to classify is by comparing a new pattern with all known patterns. For time series, there is an algorithm called "dynamic time warping" which calculates the minimum distance to map a series of points to another series of points. So DP is used in machine learning.
  • Graph algorithms: Clustering is a machine learning task. CLARANS is a clustering algorithm which operates on a virtual graph. Good luck understanding this algorithm without knowing what a graph is. Or in automatic speech recognition beam search.
  • Sorting: There are fancy approaches like Neural Turing Machines which are able to learn to sort. In fact, they are theoretically(!) able to learn any algorithm just by data. However, they are MUCH worse in any respect than simple quicksort. We don't even talk there about Timsort.
  • A lot more. And a general knowledge of algorithms and datastructures is very important. Machine learning is awesome if you have really complicated problems where you can't find a solution analytically. Many computer vision tasks, automatic speech recognition, translation, some control problems are examples for this task. However, there are many more tasks which can be solved in an optimal way without machine learning. Knowing about algorithms and when to apply which approach is crucial if you want to design good software.
| cite | improve this answer | |
  • $\begingroup$ The backpropagation algorithm itself is also an example of dynamic programming. $\endgroup$ – Jeremy List May 3 '19 at 4:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.