At home I've got a pretty extensive lan, consisting of an xbox two desktops, upto 3 laptops via wifi...is there a way of linking their spare processing power to my main desktop, linked via ethernet, in some sort of cloud computing way - similar to what they have at CERN hehe...

Woud it be easy to develop your own software to do this sort of thing?

I believe what you are looking for is a BeoWolf cluster.

Cool, thanks...will have a look at that tonight...

One more question - when does a cluster become a (distributed) network?

Hey. It all depends on what you want to do with this sort of thing! I am very interested in parallel computing but when I came to learn about (I very much learn by doing, ie, think of a program and learn how to do it while attempting to program it!) it the only program I could think of that would be in any way useful to me was a password cracking program. I had also been learning about python sockets and threads at that time and so have almost managed to complete a parallel password cracker in python using a server client method where the clients would connect to the server, find out between what characters they were iterating in search of the clear text for the hash and if none was found, go back and get another set. This is sort of a simple way to do it. A more proper and industry standard way of doing it is using MPICH. Its an open source implementation of MPI (message passing interface) and is used in alot of the big super computers. It has libraries and can be written in Fortran, C and C++.
I see your a linux person so depending on your distro you may be able to install mpich from a repository. Alternatively, and if you want to run your stuff on more than one OS (mac or windows) you can get the code and compile it from here: mpich home page

I have a book, which, honestly i haven't really got started into yet, but looks good! Its "Using MPI second edition. Portable Parallel Programming with the Message-Passing interface" which seems quite good. I think you can get it on amazon for ~£15.
They take you though basics and work you up with programming examples for all of the 3 supported languages.

So have a look on the MPICH website and read about it there, i would then recommend the book to get started on doing some cluster programming, and then start to work on your own stuff, what ever you can think of to do. I'm not entirely sure what way MPI works but I'm pretty sure its more a one program, multiple CPU's sort of model rather than a more traditional and depending on what your trying to do, maybe better, client server model. So have a look, have a go and get back to us with what you end up with, your experiences and all such like!


My lan consists of 3 computers the same as a small beowulf cluster that I built for learning about this stuff, with Q6600's and 2GB ram each and with two trunked Gb lan each and my main computer with 3 (2.4) TB storage, nice big screen for developing and a well OC'd E6700 with 2GB ram also (though I'm probably going to upgrade this for other video work I want to do on it.) and the dual trunked lan. I also have a netgear smart switch to do proper IEEE port trunking. (Though this isn't properly tested, i think there was something funny when i first tried to do it so i ended up not, now i know more about it i may get it to work)
(then two mac laptops and a little ITX thing for playing with)

I used to make pretty fractals on a distributed framework.
The master dished out portions of the fractal to calculate to the slaves that where running on the lan.
Depending on the speed of the clients they would finish in a certain amount of time and get a new packet of work (or be very slow and doing just one packet in total)

To get all of this done the program used a custom system with unix ports etc. We are talking 1992 here, mind you.

Distributed computing is usualy used for tasks that are easy to sub-divide, has a limited dataset, and limited communication between the nodes (or segments of your data-space).
The mentioned fractal is one good example of data that is very suited for distributed computing.
Compilation or rendering of 3d scenes is also done in such environments, but due to the communication between the various segments (eg: an object in segment 1 is reflected in segment 2) this is much more difficult.

One more question: are you looking for specific applications to use this computing grid for, or are you more interested in distributed computing in a general and perhaps user-transparent manner ?

Well I basically wanted to make it completely invisible by making the other processors on the other machines work in parallel via the lan, I suppose this medium wouldn't be quick enough for "live" rather than batch processing as the user would notice the transmission time.

I mean, it would obviously be benificial to have this system if you were doing several 1000 movie conversions as fast as possible...

Say if you were playing a particularly taxing game on the processor, there isn't really a way of offloading some of the extra operations onto the lan processors efficiently?

Well, for starts you'd need at least Gb lan, as I say, I have two Gb lines in each joined together for mine.
Secondly, if you are writing the game, then yes, you could do it. You could make it run on a cluster but if its simply a game you bought out of the shop or downloaded or whatever that you want to run on it, well I think you would need some insanely smart code and a virtual layer for it to sit on so you could hijack the code on the way to the normal processor to split it up and send it to lan clients. You would undoubtedly be better just upgrading your processor though, it would be much faster for games processing to make it run on one MB with its fun fast bus speeds than to try and send it over the much slower lan speeds. This is why the more high end cluster computers in the world, the proprietary ones, use specialised hardware and networking equipment and even specially designed boards and rack layouts to maximise networking bandwidth and information transfer between compute processors and nodes. Parallel processing is a lot about mass data processing so a lot of its focus is on passing data to be computed around. Hence one specification is called the Message Passing Interface (MPI)!