Is there a methodical procedure for determining the optimal timeout / retry strategy for dealing with a remote server that handles processes responses for requests, given some probability distribution of response time?

For example, suppose I have a client program which requests a web page from some server, and I know roughly the probability distribution of its response time, e.g. 95% of the time I'll get a response with a gamma distribution, and 5% of the time my request will get lost and a response never sent. So maybe 80% of the time I'll get a response within 100 milliseconds, and 90% of the time I'll get a response within 500 milliseconds.

On one hand, I could set a timeout of 500 milliseconds and retry with another request after then. Or I could set a timeout of 100 milliseconds. A longer timeout means I have more chance of success, but I have to wait longer. A shorter timeout means I can reduce my wait time potentially, at the cost of increasing the chance of failure (I give up too early; the server will respond). Or I could just send two identical requests up front and take whichever response arrives first -- this seems like it might be the optimal strategy, but something seems wrong here. (why then wouldn't I send 100 identical requests? I guess I'm not modeling the fact that lots of additional requests would clog the server queue.)

Please help; I know this is a fairly general question, but I have no idea where to look for more information of this type. I never took any classes on distributed systems in college + am trying to improve responsiveness of a client program.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.