Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. It only takes a minute to sign up.
Sign up to join this community
I have some experiences of working on clusters and grids for parallel computing. But I wonder what distributed computing can do besides parallel computing? If I understand correctly, parallel computing is to solve a common task by multiple processors (computers).
Are the Internet and a intranet considered distributed systems? They don't necessarily involve parallel computing, or solving a common task by multiple computers.
Distributed computing can be seen as a special case of parallel computing with some additional features and restrictions, some of which I'll outline here.
Multiply IO throughput
In some settings, e.g. databases, IO is the main limiting factor of performance, e.g. hard drive accesses. By serving requests on many machines, you can parallelise hard drive accesses (not possible in typical many-core machines).
Avoid slow IO
Real machines have technical limits regarding quick memory. These limits change constantly, but at any given point in time every machine can only hold so much data in RAM. A distributed system can be scaled so that all data resides in RAM, thus eliminating the need to resort to slower mediums.
Focus on throughput over response time
When faced with many small requests already dominated by, say, IO, you can not speed up individual requests anymore. Having multiple nodes serve requests can increase throughput.
Redundancy
Many nodes should be able to serve any given request so you can perform load balancing.
Consistency
When a requests not only retrieves but also changes data, many nodes may need to be informed of these changes. This is probably the factor with the most impact on performance.
Failure tolerance
The system as a whole should be robust against failure of individual nodes, i.e. continue to work (if with reduced throughput). This is not trivial since you can typically not hold all data on all nodes.
Communication cost
In a distributed system, nodes typically communicate over some kind of network. Delays in the order of milliseconds (typical response times in Ethernet) are prohibitive for systems/algorithms that require frequent communication. In specialised network implementations, wire length actually becomes a significant factor in delay times even inside a single server farm.
Security
If you don't control all aspects of the network connecting your nodes, the inner workings of your system are open to outside attack.
Note that all of these are interconnected.
