0
$\begingroup$

enter image description here

In an example of the application of MapReduce provided by University of Utah, it says that Map() function emits <"hello", 1> every time it sees hello where the reduce function counts the number of instances "hello" occurs

My question is if this is the case, why isn't reduce doing <"hello", {1,1,1,1,1,1,1,1,1,1,1,1...}>, where each 1 is an instance Map() emits a key,value pair? In the example it wrote something like <"hello", (3,5,2,7)>, what does it mean?

Also, why do you need MapReduce to do this? I can just use an linked list on my computer.

Thanks

$\endgroup$
5
  • $\begingroup$ "Also, why do you need MapReduce to do this? I can just use an linked list on my computer." Of course you dont need MapReduce for this or anything else, just as you don't need to use a linked list instead. It's just an example and, since it's an example for teaching, it's deliberately a simple example. $\endgroup$ Nov 24 '14 at 21:28
  • $\begingroup$ This is a programming question, not a computer science question. I'm not sure whether you'd want it on Stack Overflow or maybe Computational Science, but it's definitely offtopic here. $\endgroup$
    – Raphael
    Nov 24 '14 at 22:15
  • $\begingroup$ @Raphael, might the programmers stack exchange also be a good fit? It seems more conceptual than the code-related questions I'm accustomed to seeing on SO. (I agree it doesn't really fit here.) $\endgroup$ Nov 24 '14 at 22:36
  • $\begingroup$ @SeanEaster I don't think so, but I'm not an expert of the scope of either site. The question here is essentially "please explain this code" which should imho be ontopic on Stack Overflow. $\endgroup$
    – Raphael
    Nov 25 '14 at 7:14
  • $\begingroup$ @Raphael Fair interpretation, and a sound enough reason to migrate, from my view. $\endgroup$ Nov 26 '14 at 14:53
2
$\begingroup$

[...] it says that Map() function emits <"hello", 1> every time it sees hello where the reduce function counts the number of instances "hello" occurs

Not quite: It appears the mapper reads each file, counts the number of times a word appears, and outputs a single (word, count) pair per file, rather than per occurrence of the word. The reduce step then sums these. ("hello", 1) indicates that "hello" appeared once in a given file, ("hello", 3) indicates three appearances in a file, etc.

In the example for the reduce step, it appears four files were mapped, and that "hello" appeared 3 times in the first, 5 in the second, etc.

Also, why do you need MapReduce to do this?

Via wiki MapReduce is "for processing parallelizable problems across huge datasets using a large number of computers[.]" Meaning, if your task is to count the number of times "hello" appears in four small documents, you likely don't need MapReduce. But if your task is to count the appearances of all words that appear in a large set of documents, then the only way to accomplish this is a practically useful time may require distributing across multiple processors.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.