11
$\begingroup$

The general question, as the title suggests, is:

  • What is the difference between DS and OR/optimization.

On a conceptual level I understand that DS tries to extract knowledge from the available data and uses mostly Statistical, Machine Learning techniques. On the other hand, OR uses the data in order to make decisions based on the data, for example by optimizing some objective function (criterion) over the data (input).

I wonder, how these two paradigms compare.

  • Is one subset of the other?
  • Are they consider complementary fields?
  • Are there examples that one field complements the other or they are used in conjuction?

In particular, I am interested in the following:

Is there any example where OR techniques are used to solve a Data Science question/problem?

$\endgroup$
  • 3
    $\begingroup$ I'm not sure this is really a question about computer science but I suppose it's close enough. I edited out the part about what people on one side think of the other, since that seems to be entirely a matter of opinion. $\endgroup$ – David Richerby Mar 14 '17 at 14:54
  • $\begingroup$ @DavidRicherby thanks. I agree with you that it could be a matter of opinion. Traditionally, both disciplines are been taught, and emerged, from the CS community so, I suppose, this is the correct place to ask. $\endgroup$ – PsySp Mar 14 '17 at 15:06
  • $\begingroup$ en.wikipedia.org/wiki/Operations_research, en.wikipedia.org/wiki/Data_science $\endgroup$ – D.W. Mar 14 '17 at 15:59
  • $\begingroup$ @D.W. thank you. I have read the articles and to be honest I fail to see any discussion about overlap and/or differences between the two mentioned fields. In particular, how one complements the other. $\endgroup$ – PsySp Mar 14 '17 at 16:04
  • 1
    $\begingroup$ Data Science is mainly about doing work to find information via data. Operations Research is mainly about doing work to improve decision making. You can often view OR as using methods to find an optimal policy for use in decision making. Some methods used in OR can be classified as Reinforcement Learning methods in the CS community, though not all OR problems are of this type. $\endgroup$ – spektr Mar 14 '17 at 20:21
9
$\begingroup$

While both Operations Research and Data Science both cover a large amount of topics and areas, I'll try to give my perspective on what I see as the most representative and mainstream parts of each.

As others have pointed out, the bulk of Operations Research is primarily concerned with making decisions. While there are many different ways to determine how to make decisions, the most mainstream parts of OR (in my opinion) are focused on modelling decision problems in a mathematical programming framework. In these kinds of frameworks, you typically have a set of decision variables, constraints over these variables, and an objective function dependent on your decision variables that you are trying to minimize or maximize. When the decision variables can take values in $\mathbb{R}$, the constraints are linear inequalities over your decisions variables, and the objective function is a linear function of the decision variables, then you have a linear program -- the main workhorse of OR for the past sixty years. If you have other kinds of objective functions or constraints, you find yourself in the realm of integer programming, quadratic programming, semi-definite programming, etc...

Data Science, on the other hand, is mostly concerned with making inferences. Here, you're typically starting with a big pile of data and you'd like to infer something about data you haven't seen yet in your big pile. The typical sorts of things you see here are: 1) the big pile of data represents the past results of two different options and you'd like to know which option will yield the best results, 2) the big pile of data represents a time series and you'd like to know how that time series will extend into the future, 3) the big pile of data represents a labelled set of observations and you'd like infer labels for new, unlabelled observations. The first two examples fall squarely into classical statistical areas (hypothesis testing and time-series forecasting, respectively) while the third example I think is more closely associated with modern machine learning topics (classification). Fun trivia: I believe that for a long time, the job title at Google for people doing what we now call Data Science was (is?) "Statistician".

So, in my opinion, Operations Research and Data Science are mostly orthogonal disciplines, although there is some overlap. In particular, I think that time-series forecasting appears in a non-trivial amount in OR; it's one of the more significant, non-math programming-based parts of OR. Operations Research is where you turn if you have a known relationship between inputs and outputs; Data Science is where you turn if you're trying to determine that relationship (for some definition of input and output).

$\endgroup$
  • $\begingroup$ Thank you for the clear answer. I was wondering if, for example, one could use any OR techniques to solve DS problems. I would be interested in such an example but, from your answer, I doubt there is any. $\endgroup$ – PsySp Mar 17 '17 at 23:08
  • $\begingroup$ @Psysp Eh, maybe? I can't think of any off the top of my head but that's far from definitive. $\endgroup$ – mhum Mar 17 '17 at 23:58
  • 1
    $\begingroup$ I don't think the division between OR and DS is a strict as you believe, but this might be because I consider topics as machine learning and datamining as parts of DS instead of considering DS a synonym of Statistics. (Unfortunatly, as DS is a buzzword, it has no widely accepted definition, as far as I know) However, the tasks of descision and inference need not be mutually exclusive. Machine learning is precisely the field where both are combined: sometimes clever decisions must be made to make decent inferences, at other times clever inferences are used for good decisions. $\endgroup$ – Discrete lizard Mar 20 '17 at 23:23
  • $\begingroup$ @Discretelizard Sure, I agree to some extent. I am presenting a rather stark division (maybe almost a caricature?) and concentrating on the core parts of each field in order to highlight the differences in the types of problems each field is typically tuned for. The edges of both fields can be pretty fuzzy (especially in DS which is a lot newer) and there is probably more overlap there. Also, I agree that a lot of the mainstream of DS includes ML stuff but I wasn't sure exactly how divided DS is from ML. $\endgroup$ – mhum Mar 21 '17 at 16:42
4
$\begingroup$

This isn't a full answer, since mhum's is quite good in contrasting the differing aims of OR vs DS.

Rather, I want to address this comment of yours:

I was wondering if, for example, one could use any OR techniques to solve DS problems.

The answer is yes. The clearest example that comes to mind is Support Vector Machines (SVMs).

To "fit" an SVM model to some data (which must be done before you can use it to infer predictions), the following optimisation problem must be solved:

Maximize the dual,

$$ g(a) = \sum_{i=1}^{m} \alpha_i - \frac{1}{2} \sum_{i=1}^{m} \sum_{j=1}^{m} \alpha_i \alpha_j y_i y_j x_i^T x_j,$$

subject to the constraints

$$ 0 \leq \alpha_i \leq C, \qquad \sum_{i=1}^n y_i \alpha_i = 0$$

This is a constrained optimisation problem, just like many in the field of OR, and it is solved using quadratic programming methods or interior point methods. These are generally associated with the field of OR rather than DS but this is an example of their wider applicability.

More generally, optimisation is key to many of the statistical and machine learning models employed in the field of DS, since the process of training these models can typically be formulated as a minimisation problem involving a loss/regret function - from the humble centuries-old linear regression model to the very latest deep learning neural network.

A good reference on SVMs is Bishop.

$\endgroup$
2
$\begingroup$

As a strategist, I've had the opportunity to work with both sides of the discipline. In trying to explain what OR and DS are to a qualitative MBA executive, my (overly) simplistic one line introduction for each

OR: economists that know how to code
DS: statisticians that know how to code.

In practical terms, how the two groups typically come together: the OR side develops the decision model, and the DS side figures out the appropriate data implementation to feed the model.

Each on their own, will rely on the theoretical traditions of their disciplines - together, they conduct experimentation to structure the data and refine the model in order to get to the true insights needed for optimal decisions. As each gets to know the other, their thinking and their language will typically converge.

$\endgroup$
  • 1
    $\begingroup$ I understand the practical description of DS as 'statisticians who code', but the description for OR seems a bit strange to me. OR includes logistics and related routing problems. That doesn't really look like a natural place for an economist to me. Perhaps you could elaborate on why OR is done by economists in practice? $\endgroup$ – Discrete lizard May 4 '18 at 10:03
  • 1
    $\begingroup$ @Discretelizard I don't doubt that economists do OR, but there is, as you say, a heck of a lot of OR that has nothing to do with economics and is done by computer scientists, mathematicians and others. $\endgroup$ – David Richerby May 4 '18 at 16:09
0
$\begingroup$

Data science is a broad field that deals with data in general. If this sounds vague it is normal because it really is. It has been a buzz word for quite some years now. Essentially, it tries to find a way to exploit data: what can I do with my data (what insights can I get from it?).

Operations Research is the science of mathematical optimization: you model a problem into “equations”, solve this mathematical model and translate the solutions back into your initial problem setting. It is a tool to help to make decisions: what should/can I do to obtain this or that.

Many business problems can be viewed as a optimization problem. Given that I am trying to maximize my revenue, given the resources constraints, how exactly would I carry out my business, of what values should I set for my decision variables. Problems such as scheduling, facility planning, supply chain management...etc all leverage optimization techniques.

Portfolio optimization is also a classic example where optimization is being used. Suppose that I can invest in several different assets in my portfolio, each with non-deterministic returns, how should I balance my portfolio so that I minimize the risk of my overall portfolio while maintaining a level of monetary return. In this setting, the objective function often becomes the risk/variance of the portfolio, and the constraints are the required rate of return on the investment, as well as the amount of money you have.

$\endgroup$
  • 3
    $\begingroup$ You only list brief summaries of both fields. This answer doesn't address the differences and/or similarities between DS and OR, for which the question specifically asked. You can improve your answer by focussing on that part $\endgroup$ – Discrete lizard Jan 30 '18 at 8:12
-1
$\begingroup$

If you count ML and AI driven by ML as a part of Data Science(which some people do and some dont according to my experience, for instance Microsoft professional program in AI contains key aspects of Data Science+Machine learning(with both DL and RL) while Higher School of Economics presents practically same advanced parts of Microsoft cuuriculum as Advanced Machine Learning) then there are many similarities in mathematics that is used in both fields. For instance: Nonlinear Programming(Lagrange multipliers, KKT conditions...)-->used for derivation of Support Vector Machines...Econometrics which is mostly based on Regressions---> Regressions are key part of both Data Scinece in general and more specifically Supervised Learning...Statistics(normally found in OR Curriculum)--->key for Data Science and Machine Learning as well...Stochastic Processes--->very important in Reinforcement Learning...Dynamic Programming--->again found in Reinforcement Learning...So,I would say there are some similarities with Data Science in general and pretty much similarities with ML. Of course, goals of these disciplines are different but there is a lot of similarities in mathematics that is being used in these disciplines.

$\endgroup$
  • $\begingroup$ How does it answer the question? $\endgroup$ – Evil Mar 31 at 18:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.