Let's say I have a database of freight orders. The job is to match freight carriers with customers who need their freight moved. I have the customer's information, the freight carrier's information, and all details related to the freight orders including date ordered, date shipped, the amount of money it took to hire the freight carrier, and whether a carrier was even found to ship the order.

If I have thousands of these past freight orders, could I use machine learning to look at future freight orders to predict whether or not a freight carrier will be found to move it?

Bonus: If it is possible, what steps would I need to take to find the best data points to focus on? From what I understand, I need to convert everything to a number in order to train the classifier, but I am having trouble figuring out what data features are going to help make these types of predictions.

I have been studying how to do machine learning and I am not looking for somebody to tell me everything there is to know on the subject, I just don't know how to determine what data points are going to be useful and am also looking for an answer to whether or not this is something machine learning can do(or if it's something a beginner in machine learning can do). Sorry if the question seems vague, it's kind of hard to articulate on a subject you are just starting to learn about. If anybody has materials they can link that would help me to better understand these things, I would appreciate that as well..

  • 3
    $\begingroup$ This sounds too broad to me. Community votes? (Your first step is to study how machine learning works. It sounds to me like you are asking "I don't know anything about machine learning; can you tell me everything I need to know about the field to solve this problem?" That sounds too broad to me. I'd suggest you start by doing some self-study, then come back when you can formulate a more specific question. Alternatively, can you edit the question to ask about a specific aspect of the problem?) $\endgroup$ – D.W. Sep 16 '15 at 15:45
  • $\begingroup$ Well I have spent hours studying machine learning, and there is very little information I have found that is helpful due to not knowing much about the subject, so finding places to start is difficult to say the least. I know how to use the library I am using to do what I need to do, the problem is that I don't know much about statistics and I don't know what data points are going to give me the best predictions. The rest of it I can figure out. I don't need any help with the library. $\endgroup$ – Duck Puncher Sep 16 '15 at 16:18
  • $\begingroup$ @DavidRicherby, The official question is, "Can I use Machine Learning to Solve this Problem?". Listing steps I would need to take to get good predictions would just be icing on the cake, but I think the question is definitely on-topic. I guess the library wouldn't matter honestly, I am more concerned with if machine learning can solve the problem or not. $\endgroup$ – Duck Puncher Sep 16 '15 at 16:19
  • $\begingroup$ @DuckPuncher I agree that the edited question is wholly on-topic. $\endgroup$ – David Richerby Sep 16 '15 at 16:51
  • 1
    $\begingroup$ "What data features are going to be useful?" are not computer science questions. Those are questions that relate to your particular application domain: freight ordering. So, I don't see how we could possibly tell answer that for you -- you'll have to tell us. But in a similar vein: Have you read about feature selection? $\endgroup$ – D.W. Sep 16 '15 at 18:18

Here's the thing to know: machine learning assumes that there is some sort of statistical distribution, and you "learn" that distribution, in order to get the probability of some event.

The common saying in machine learning is "garbage in, garbage out." You have a bunch of random variables, but if those variables are all statistically independent, then you won't get anything useful from machine learning.

Say, for example, that there was a strong correlation between the amount of money paid for the carrier, and whether the item is shipped. Machine learning would likely be able to discover this relationship.

Or, if there were certain periods of the year where an item was more or less likely to be shipped. Machine learning could find this out.

But, if there's no underlying pattern in the data you give to your training algorithm, then you will get no useful information out of it.

For your question of finding the best data points: don't. That's what the machine learning algorithm is for. You give it your data, and it looks at which ones are statistically relevant, given some threshold. The whole point of machine learning is letting the algorithm do that for you, and more importantly, that's all the algorithm can really do for you.

Generally, these algorithms will work better when they have more data, so don't go out of your way to remove what data you're giving it.

And, remember, that ALL these algorithms are doing is statistics. If it says that the item is shipped with probability 0.6, don't be surprised when it isn't shipped. And, it's possible that it will say the item is shipped with probability 0.99, but it won't ship, because there's some variable which is hugely important in real life, which you didn't have recorded for your data set. If that variable isn't correlated with your data, then your model will be no good.

| cite | improve this answer | |
  • $\begingroup$ this is exactly what I was looking for! I have troves of data and we think there are patterns at play here. Thanks! $\endgroup$ – Duck Puncher Sep 16 '15 at 18:30
  • $\begingroup$ You might want to look at WEKA as a toolkit for doing your tasks. I'm guessing you'll want to start with decision trees, SVMs, and Naive Bayes as the simplest classifying algorithms. But that's not more StackOverflow territory than CS.SE. $\endgroup$ – jmite Sep 16 '15 at 18:34
  • $\begingroup$ thanks. I am actually using scikit-learn along with a few other python modules to tackle this, but if it doesn't seem to do the job I need it to do, I will give it a shot. I'm willing to try anything at this point. Thanks again! $\endgroup$ – Duck Puncher Sep 16 '15 at 18:43
  • $\begingroup$ Yeah, Python has a library for everthing, so there's a good chance scikit-learn will do the trick for you. Good luck! $\endgroup$ – jmite Sep 16 '15 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.