Multithreading – distributed computing and threading

How similar are distributed computing and threads? I found that two papers came to the opposite conclusion:

"Multithreading is easier than network. How simple is thread, similar to network code"

http://software.intel.com/file/14723

(this gives me the impression that they are very similar. After encapsulation, the two methods can be completed with the same code - but maybe I'm wrong)

"Description of distributed computing"

http://research.sun.com/techrep/1994/abstract-29.html

(there's a big difference)

I'm sure the facts are somewhere in between What is the golden mean? Is there a technology that can unify these two paradigms? Or did these attempts fail because of the fundamental difference between networking and concurrency?

Solution

I never found them very similar Let me define a "node" for the purpose of this article, which is a hardware thread running on a machine Therefore, a quad - core machine is a cluster of four nodes and four single - processor boxes

Each node usually runs some processing and requires some type of cross node communication Typically, the first instance of this communication is telling the node what to do For this communication, I can use shared memory, semaphores, shared files, named pipes, sockets, remote procedure calls, Distributed COM, etc But for the simplest use, shared memory and semaphores are usually not available over the network Shared files may be available, but performance is often poor Socket is often the most common and flexible choice on the network, rather than a more complex mechanism At this point, you must deal with the details of the network architecture, including latency, bandwidth, packet loss, network topology, etc

If you start with a work queue, nodes on the same machine can use simple shared memory to complete their work You can even write it as unlocked. It works seamlessly Where do you place queues through nodes on the network? If it is centralized, the machine may bear a very high bandwidth cost Trying to distribute it, things get very complicated very quickly

In general, I find that people dealing with this parallel architecture tend to choose embarrassing parallel problems to solve Thought of ray tracing In addition to job assignment, there is not much cross node communication required To be sure, there are many such problems, but I find that distributed computing is basically the same as threading, which is a little dishonest

Now, if you want to write the same write thread as a distributed system, use pure messaging, and don't assume that any thread is "primary" and so on, yes, they will be very similar But all you do is pretend you have a distributed architecture and implement it in threads The problem is that threading is a simpler case of parallelism than real distributed computing You can abstract these two problems into one problem, but choose the more difficult version and strictly abide by it When all nodes are local to the machine, the results will not be as good as they are You didn't take advantage of this exception