Old Article - How to Build a Diskless Folding Farm

A forum just for SPCR's folding team... by request.

Moderators: NeilBlanchard, Ralf Hutter, sthayashi, Lawrence Lee

Post Reply
aristide1
*Lifetime Patron*
Posts: 4284
Joined: Fri Apr 04, 2003 6:21 pm
Location: Undisclosed but sober in US

Old Article - How to Build a Diskless Folding Farm

Post by aristide1 » Wed Mar 05, 2008 4:26 pm

Ran across this over where I check our stats:

http://www.extremeoverclocking.com/arti ... arm_1.html

I also sent them a donation since they need to upgrade their hardware to keep up with the growing demand of processing stats in a timely manner. I always loved his constantly updated graphs, even our teams graph, which is going straight down! It looks like the crash of '29.

KansaKilla
Friend of SPCR
Posts: 381
Joined: Fri Aug 01, 2003 12:13 pm
Location: Rochester, MN

Post by KansaKilla » Thu Mar 06, 2008 7:53 am

That looks really interesting. Thanks for posting it--methinks I might want to try it at some point in the near future.

iganu
Posts: 11
Joined: Wed Dec 26, 2007 10:29 pm

Post by iganu » Thu Mar 06, 2008 8:51 pm

I have a setup kind of like this for my folding machines. They're on my main network, and they netboot from my main machine. Every 30 minutes, they tar up the folding data files and tftp them back to the server machine. When a client is started it checks to see if there are saved files on the server, if so it grabs them and un-tars them otherwise it just sets up a new directory and starts from scratch.

This setup has been working well for about 2.5 months now.

It really is a lot easier to administer. There are no need for hard drives, which keeps noise, cost, and power consumption down. It's also much easier to add a new folding box, since there's no need to install an OS (all the clients boot the same kernel image). Just a couple of quick configuration things on the router and server, and a new box is ready to go in about 5 minutes.

I built my linux from sources. It's a very minimal setup, just the kernel (non-modular), busybox (lightweight command line utilities), and dropbear (for remote access via ssh). All this runs from an initramfs packaged with the kernel. The whole thing (kernel and filesystem) is less than 4MB in size, so the netboot is fast, and not much RAM is used for the kernel and filesystem. And it folds a bit faster than ubuntu as well, but the difference isn't much.

aristide1
*Lifetime Patron*
Posts: 4284
Joined: Fri Apr 04, 2003 6:21 pm
Location: Undisclosed but sober in US

Post by aristide1 » Fri Mar 07, 2008 3:30 pm

Welcome Iganu.

Interesting info. I was wondering 2 things about what you said in regards to the farm.

1. How do you monitor individual motherboards? How can you tell if its still running?

2. Are checkpoints kept on the server? You don't lose a WU if you have a power outage, do you? I suspect not but still I ask.

iganu
Posts: 11
Joined: Wed Dec 26, 2007 10:29 pm

Post by iganu » Sat Mar 08, 2008 8:47 am

I usually keep an ssh session open in a terminal window to each client and check the logs a few times a day to make sure things are still running (but they almost always are). Maybe once a month I need to fix something.

Work unit files are backed up to my main machine every half hour. So if there's a power outage I loose at most 1/2 hour plus the time since the last checkpoint. But the big cost of a power outage for me is that my machines aren't setup to automatically reboot when the power comes back on, so if I'm not around they could be down for some time. But fortunately power outages are fairly rare where I live.

VanWaGuy
Posts: 299
Joined: Mon Sep 11, 2006 1:01 am
Location: Vancouver Wa USA

Post by VanWaGuy » Sat Mar 08, 2008 9:08 am

Hey Iganu,

nice to see more posts! (Half of them in this thread.)

Congrats on hitting 1 mill points ahead of me. (We had a little race going.)

You did not mention the reason that you built your own kernel from source. :)

iganu
Posts: 11
Joined: Wed Dec 26, 2007 10:29 pm

Post by iganu » Sat Mar 08, 2008 2:55 pm

Congratulations to you Viking for reaching the 1 million point milestone.

I built my kernel from source because I wanted a small non-modular kernel with just the stuff I needed to support folding on my hardware.

But it did turn out to have other benefits :)

KansaKilla
Friend of SPCR
Posts: 381
Joined: Fri Aug 01, 2003 12:13 pm
Location: Rochester, MN

Post by KansaKilla » Sun Mar 09, 2008 6:37 pm

Hi, iganu. How hard is it for a Linux newbie to build his (or her) own folding-centered distribution? I'm interested in doing it, but I've only messed around with SuSE 10.2 about nine months ago or so. I'm thinking I might have more time beginning in July or so, and this might be a project into which I could get interested.

aristide1
*Lifetime Patron*
Posts: 4284
Joined: Fri Apr 04, 2003 6:21 pm
Location: Undisclosed but sober in US

Post by aristide1 » Sun Mar 09, 2008 7:19 pm

KansaKilla wrote:Hi, iganu. How hard is it for a Linux newbie to build his (or her) own folding-centered distribution? I'm interested in doing it, but I've only messed around with SuSE 10.2 about nine months ago or so. I'm thinking I might have more time beginning in July or so, and this might be a project into which I could get interested.
Me too, except I only have limited Ubuntu experience. I may get some Knoppix experience in my current class.

Am I speaking Unix yet? :roll:

floffe
Posts: 497
Joined: Mon May 08, 2006 4:36 am
Location: Linköping, Sweden

Post by floffe » Wed Mar 12, 2008 12:42 pm

Linux From Scratch has a decent enough guide to what you need to get up and running. It'll require quite a bit of command line wizardry, but it'll also build you the slimmest distro possible. I haven't used it myself, but from what I understand the guide is pretty comprehensive. IRC is a useful way of getting help with linux distros in general, and command-line-based ones in particular, so I'd suggest idling in the LFS support channel while installing, so you can ask questions there.

Bear in mind that having no package manager means that you'll have to upgrade everything yourself, though. I guess that's not much of an issue for folding machines which only interface with your own network and stanford, though.

Post Reply