0
$\begingroup$

I was reading the following computer systems paper:

https://www.usenix.org/system/files/conference/osdi12/osdi12-final-75.pdf

And I was trying to understand why it claims that it does not need data locality for it be perform well.

Basically, in the abstract it says:

"...FDS multiplexes an application's large scale I/O across the available throughput and latency budget of every disk in a closer. FDS therefore makes many optimizations around data locality unnecessary."

What I was not sure was, why does multiplexing an applications I/O across the servers in the datacenter, make it unnecessary to have to optimize in terms of data locality?

Basically, as I read the paper they make a big deal that locality is something they don't need and that they can achieve the benefits of it without having servers being close to the clients they provide data access. The part I am not sure is, what is the crux of what makes them not depend on data locality? Is it the CLOS network? Is it because they don't serve clients around the world? Is it because their servers are all in one data center anyways and not scattered around the world? What is it that makes their performance be so amazing?


Paper Reference:

Title: Flat Datacenter Storage

Author(s): Edmund B. Nightingale, Jeremy Elson, Jinliang Fan, Owen Hofmann, Jon Howell, Yutaka Suzue

Institution(s): Microsoft Research, Univeristy of Texas at Austin

$\endgroup$
  • 1
    $\begingroup$ Would you mind adding a full reference (paper title, author names, and conference name) so that this question is more readily findable by search and thus more likely to be useful to others? $\endgroup$ – D.W. Mar 28 '14 at 4:26
  • $\begingroup$ Thanks for the reminders! :) Its a great Idea to do that, don't know why it didn't occur to me. $\endgroup$ – Pinocchio Mar 28 '14 at 4:59
3
$\begingroup$

The paper you cited is exploring the consequences of the question "what if datacenter network bandwidth was free?" There are two reasons you might care about data locality. One is latency and the other is bandwidth. The workloads they are looking at (mostly) don't care about latency, and they've made bandwidth irrelevant, hence locality doesn't matter. (Note however that at the end of Section 6.2 (on Page 12) they discuss how the inefficiency of non-local accesses forced them to use more compute nodes for their sorting algorithm.)

They also aren't completely eliminating locality optimizations. Each compute node is still caching/buffering reads and writes in its local DRAM. The rule of "avoid unnecessary I/O" still stands. It's just that you don't make a distinction when you do a disk access of whether the disk is near or far from your compute node. All disk accesses are assumed to include the extra latency of a round-trip network traversal.

Here's a picture to show the difference. The authors' "flat" storage model is on the left, the traditional "distributed" storage model is on the right:

flat vs. distributed storage model

In the distributed model (on the right) data is either on a local disk or a remote disk. In the flat model (on the left) data is all remote. But since the network bandwidth is (effectively) infinite, this doesn't reduce your maximum data bandwidth.

$\endgroup$
  • $\begingroup$ How are we going to decrease network round trip time? I mean, what kind of logic is best to ensure that mostly, the data needed by a compute node is available with it without the need of traversing the network? $\endgroup$ – JavaTechnical Jan 17 at 13:00
0
$\begingroup$

Their design is based on a "non-oversubscribed full bisection-bandwith network" (the CLOS network you mentioned). And due to this assumption you can consider accessing remote disks as fast as local ones.

Now comes the trick: if you regard every blob as a file, then this "file" is essentially scattered uniformly on a bunch of servers (disks) . And you can read/write this file parallely with a aggregated bandwidth (thanks to the full-bisection-bandwidth assumption) of all these servers. By the way, another important reason that enables this parallism is that: since the data distributing scheme is generated by a deterministic function, it is unnecessary to contact the metadata server most times.

On the contrary, although traditional designs store the whole dataset distributedly, each "file" (or anything that represents the data unit) is stored on a single server (as well as some replicas, maybe), and that exactly causes the problem of data locality.

Strictly speaking, each "tract" in the FDS is stored locally in a tractserver. But a tract could be so small compared with the blob size that, it's unlikely to do some computation with a single tract -- you always deal with plenty of tracts within some blob, which by design are scattered around, thus eliminate locality problems.

This viedo explains the thinking behind the FDS neatly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.