Supermicro 4-Socket G34 Folding for Team Icrontic

LeonardoLeonardo Wake up and smell the glaciersEagle River, Alaska Icrontian
edited January 2012 in Folding@Home
"Greyhound 5":
Supermicro H8QGL-iF
4 X Opteron 6136 2.4@2.7GHz
64GB G.Skill DDR3 1600
Corsair AX750
4 X Cooler Master Hyper 212+ home-fabricated retention brackets
Rocketfish/Lian Li highly modified for SWTX format motherboard
Linux Ubuntu 10.10 optimized for Folding@Home

View through the side panel window:

image

Side panel off:

image

Closeup:

image


my macro photography leaves a lot to be desired - can't seem to optimize the lighting, sorry about that
«1

Comments

  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    Custom mounted Cooler Master Hyper 212+ heatsinks. I fabricated the heatsink retention/motherboard mounting bars from shelf standard (see the guide here):

    image

    image

    image
  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian
    What a gorgeous build!
  • TushonTushon I'm scared, Coach Alexandria, VA Icrontian
    Freaking awesome, Leo! I'm glad to have been a part of the Ubuntu train for you.
  • GargGarg Purveyor of Lincoln Nightmares Icrontian
    That is pure folding pornography. Nice build, Leo! What's the PPD like on that beast?
  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian
    Just so I'm clear here—this is 32 cores, right?
  • _k_k P-Town, Texas Icrontian
    Yup, the Opteron 6136s are 8-core 2.4GHz(MC cores). They are the fastest 8 core 6100 series opterons that AMD has.

    Leo has a board with a hacked BIOS which allows for overclocking thus the reason they are at 2.7GHz, which is insane on so many levels given the hardware.
  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian
    That is tremendous.
  • ErrorNullTurnipErrorNullTurnip Illinois Icrontian
    Holy cow! Mind=blown
  • quake101quake101 Ohio Icrontian
    :eek:

    I think I just poo'd my pants... Wowzers!

    Question, what Ubuntu optimizations can be done?
  • TushonTushon I'm scared, Coach Alexandria, VA Icrontian
    I've got them all listed in my upcoming article, right @primesuspect?
  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian
    Yep!
  • PirateNinjaPirateNinja Icrontian
    Leo that is amazing on a human being level and a technical/build level. Thank you so much for all the folding you too, what an a generous hobby.
  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    Oh, hi guys! Sorry to reply late, I guess I didn't haven't thread update notifications turned on. My favorite comment:
    That is pure folding pornography.
    thank you, thank you. I took pains for the appearance. For me, aesthetics are important even if no one else notices. Aesthetics can be a pleasure, even in this case, where the computer is in my garage surrounded by a full pegboard of tools and a grease-stained workbench. Concerning appearance: 1) I've cleaned up the cables even more, bundling the cables on the bottom better and color matching the SATA cable with the red components, and 2) the back of the case, where I had to do significant cutting for cooling and motherboard (SWTX format) fit, is not so fancy.

    Production. The beast turned out to be as powerful as I had hoped. Project P6903 typically runs at a rate of around 17:00 to 17:05 minutes per frame (TPF), which, at today's bonus rate (QRB), equates to about 370-375K PPD. The P6904 should be better, but I have not been fortunate enough to download one yet.

    In preparation for this build, I sold off a boatload of GTS 450 video cards that I previously had Folding. (If you missed my sale at Icrontic's http://icrontic.com/categories/buy-sell-trade, hey, what can I say? Go on over there and take a look!) Our home power bill now is about $100/month less than what it was one year ago, even though we are having a colder than average winter. The AMD 2P and 4P rigs are a tremendous bang for the buck in production and power efficiency. On the efficiency scale, nothing comes close.
  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    Alright, let's address some other questions and comments (_k_ and Tushon already addressed some of this):

    * PPD: best so far is 375K PPD
    * power draw (no one asked, but should have): about 600W, full load
    * Ubuntu optimizations (Tushon is hot on this as well): 1) "Langouste," which is an automated background app that uploads completed Folding work units simultaneously as new ones download, 2) "The Kraken," which is a load balancer, that keeps all the CPU cores sharing the Folding load as far as possible - scaling is excellent, 3) and a couple scripts that allow Linux to engage the overclocking settings (please don't ask how it works - it just does!)
    * CPUs (_K_ was correct): 4 times AMD Opteron (Magny Cours), each 8 physical cores, default 2.4GHz, overclocked 12.5% to 2.7GHz (NOTE -- The RAM must also be flashed, otherwise the factory JDEC settings will not be sufficient for motherboard's custom BIOS.)


    On a personal note, Team 93 comrades, I fully understand that we've all got different levels of enthusiasm and different budgets. It took me over 10 years (yes over 10 years) of Folding before I had the resources to put together a rig like this. I'm past the age where 'impressing' people is important for me (hmm, unless a promotion can be earned...). My intent with this thread is to inspire you Folders to bigger and greater things - it doesn't have to be the top. Just one PPD improvement is still improvement. Another reason for posting this thread was simply to share with you. This project has probably been the most satisfying computer hardware endeavor I've had since I accomplished my first home-built computer in 1998.
  • LincLinc Owner Detroit Icrontian
    That is a beautiful build, but the thing that grabbed me was the 10 years comment. Have we really been dancing around this site that long Leo? (Well, *almost* 10 for me, ya old-timer ;) ). And I still haven't ever shaken your hand. :)
  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    I started folding in 2001. I Folded for a small team for about three weeks at another site - don't remember my user name, the team name, or site name. I liked it right away, as it seemed a natural extension of custom computer building and overclocking. I broached the subject at Icrontic, and we started Team 93 (the low number should tell how far back we go :) ). So yeah, more than a decade for both Team 93 and Leonardo. Yes, over 10 years for me at Icrontic. I formally joined right after the site changed its name from "Apu's Hardware." LOL What, I think that was 1999 or 2000.
  • LincLinc Owner Detroit Icrontian
    I know! I remember seeing in an old archive.org capture that you had user # 293 or something like that on the original Icrontic. :)
  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    No, it couldn't have been 293. We are 93 and we formed after the first team I was on, which would have had a lower number. I just did a sequential search of lower numbered teams at the Stanford main site. It may have been Team 49, "ForumOC." ForumOC is no longer a forum but appears to be some kind of defunct/neglected security site. I don't know, I never went back to it after Team 93 became active.
  • ardichokeardichoke Icrontian
    This reminds me... I need to fill the 2nd C32 socket in my server, if for no other reason than more PPD.
  • LincLinc Owner Detroit Icrontian
    edited January 2012
    @Leonardo No, I meant you had UserID 293 on the original Icrontic forum prior to the 2003 crash. :) You were one of the first to sign up after the move from Apu's. I was talking about how long you'd been around the forum, not folding specifically.
  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    Oh, yes, right! Sorry, I'm scatterbrained as of late (wife would say, "as usual").

    Actually, time Folding and time as an Apu's/Icrontic member are pretty close.
  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    The computer finally downloaded a P6904.
    TPF average: 23:12
    PPD: 388K
  • TushonTushon I'm scared, Coach Alexandria, VA Icrontian
    That is nuts

    /endthread
  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian
    388k? :eek2:
  • LeonardoLeonardo Wake up and smell the glaciers Eagle River, Alaska Icrontian
    edited January 2012
    Yes.

    That was my reaction as well, when a few months ago I started seeing Folders experimenting with Intel/EVGA SR2 and AMD G34 rigs. "No way!" Yes, way!

    _k_ has a couple of angry dragons as well - A Supermicro (Tyan?) 2P G34 and an Asus 2P G34.
  • _k_k P-Town, Texas Icrontian
    Tyan
  • aliasveealiasvee
    edited March 2012
    Hi there,
    I am folding for the hardware.no team 37651, as -alias-

    This is my new G34 rig that I built recently. The case had to be modified a little, and so had the CPU coolers, but all together it works fine and are very silent, and in my opinion it looks not bad either.

    - Xigmatek Elysium case
    - Tyan S8812 Motherboard
    - 4 x Opteron 6272 CPU
    - Antec Kühler H2O 620
    - 64Gb Corsair 1600MHz RAM
    - Chieftec BPS-950C 80 PLUS Silver
    - OCZ 2.5 "Flash Disc / SSD 60GB Agility
    1116816
  • TushonTushon I'm scared, Coach Alexandria, VA Icrontian
    Nice build! You have an insane PPD ... how much does just this machine get?
  • aliasveealiasvee
    edited March 2012
    Thanks..
    Not as much as you might think.
    For P6904 349K PPD. It is about the same or better PPD for 6903 with 352K, and 280K for the 6901.

    Opteron 6272 have only 8 x FPU, so we're talking 32-core for folding even if the rig has 64 cores. Power consumption from the wall is, however, only 550watt and I'm happy with that. All together I have 8 rigs going 24/7, but I'm about to replace the 2600k (4) and 980X (2) rigs. Most likely I will replace the 6 with one 48P G34 rig.
  • TushonTushon I'm scared, Coach Alexandria, VA Icrontian
    Very nice. Thanks for the info and stop by.
Sign In or Register to comment.