So I have a large number of commands, say 500,000, that I want to send and run on a server somewhere else, and get the answers back. All of these commands together takes a long time to execute - roughly 30 seconds. I could group all those commands together and send them off, have the server execute them all, and then get my results. But that's significantly slower than grouping those commands into batches, which I send off all at the same time, and can then execute on that server in parallel.
However, I can't make the batches too small, because I'm doing this from a browser, and most modern browsers can only keep ~6 concurrent open connections with the same server. So you get a trade off - for large batch sizes, I'm not sending enough concurrent commands to the server. For small batch sizes, I'm sending so many that the browser gets backed up.
I've tried this experimentally, and I wind up with data that looks something like this (pardon the terrible drawing):
On the x axis is the number of batches I use, and on the y axis is the time the whole resulting calculation takes. I haven't succeeded in finding a model that really works for this data, only a basic crude approximation.
All I really want to know is, is there research on this kind of thing I can look up? Some way that, given code execution times on the server, and delay times for getting there, I can calculate the optimal batch size? I can't be the only person who's wanted to solve this problem. If there isn't, then what do you think?