Team 93 back on a downward slide

primesuspectprimesuspect Beepin n' BoopinDetroit, MI Icrontian
edited March 2016 in Folding@Home

Well the excitement was nice while it lasted, but we fizzled. We're back down to 28 active folders from 36, and our points rally has stalled. What happened? Stability issues? temperature problems? Ask for help. What can we do to get back to an upward trend?

BuddyJ
«1

Comments

  • GargGarg Purveyor of Lincoln Nightmares Icrontian
    edited March 2016

    I turned off my GPU while away for the weekend, but its 100K PPD are back in business as of last night.

    primesuspectSonorous
  • ardichokeardichoke Icrontian

    Power costs money. Money is tight. Only folding on my server (because I don't pay the electricity bill in the data center)

    primesuspect
  • BlackHawkBlackHawk Bible music connoisseur There's no place like 127.0.0.1 Icrontian

    I'm not sure about GPU folding after killing my video card and the ambient temperature is rising so the A/C is on instead of the heater.

    primesuspect
  • _k_k P-Town, Texas Icrontian
    edited March 2016

    A lot of it is because one person is "off". Around the weekends and Tuesdays we will always see dips as people have spare time to game.

    I had a power hit at work Saturday morning which resulted in a loss of 100k a day. Then I was busy tearing through 10+ hours of games over the weekend.

    One of the big issues being that most of the heavy folders are going to be gamers so they might have paused over the weekend. You have to build a personal culture around f@h to be successful long term. Remember to pause/unpause, every couple of days check on clients even if numbers are good, watch your stats daily to stay on top of clients, watch for heat waves to turn ACs on or just run with the windows open, plan for the summer and what clients use the least electricity and produce the least heat but have a high yield, bring clocks down or shut off CPU clients to buy room on your heat envelope, laying fans on the backside of video cards to keep them cooler or pulling side panels, when is turning WUs in more important than using your computer for other things.

    primesuspectGargSonorous
  • RequitRequit That one guy Somewhere over there, I don't know Icrontian

    I recently purchased a few new GPU's but have had issues with the drivers causing BSODs. As soon as I get that taken care of...

    primesuspectGargSonorous
  • SnarkasmSnarkasm Madison, WI Icrontian

    Yes, I'm sorry. I'm on my honeymoon, and I left my PC folding, but it crashed like the second day I left. I monitor it every few hours when I'm in town, but nothing I can do from this far away.

    So yes, stability. Even with what I think is a -20% underclock on frequency and a +10% boost in power (in theory, giving it more than enough power to handle what it should be calculating), I still encounter frequent hangs. Stabilize me and you'll have all my PPD more regularly.

    primesuspectGnomeQueenGargSonorous
  • drasnordrasnor Starship Operator Hawthorne, CA Icrontian
    edited March 2016

    My thing seems to be working.
    [23:04:01] * OS: Microsoft Windows 10 Pro
    [23:04:01] * CPU: Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz (3.00 GHz)
    [23:04:01] * RAM: 16347 MB Total (10262 MB Free)
    [23:04:01] * VGA: NVIDIA GeForce GTX 980
    [23:04:01] * Uptime: 637.01 Hours

    BlueTattooGargSnarkasmSonorous
  • drasnordrasnor Starship Operator Hawthorne, CA Icrontian

    @_k said:
    Remember to pause/unpause.

    Easy as pie.

    fah.JPG 100.3K
  • SignalSignal Icrontian

    For me, starting to become to warm outside to fold. Folding turning my rig into a space heater.

    primesuspect
  • SonorousSonorous F@H Fanatic US Icrontian

    I added a GTX 970 to my arsenal so that should be another 250k-300k from me daily. I'll try and stick the 660ti in something and get that 60k back in the mix too.

    primesuspect
  • _k_k P-Town, Texas Icrontian

    @Sonorous I have two 970s running currently, would only bank on a long term average of 250k assuming factory clocks.

  • SonorousSonorous F@H Fanatic US Icrontian

    @_k said:
    @Sonorous I have two 970s running currently, would only bank on a long term average of 250k assuming factory clocks.

    What are "factory clocks"? :wink:

    Garg
  • JokkeJokke Bergen, Norway Icrontian

    Having a hard time folding while at work. Internet is too unstable, and down-/uploading work could take from hours to day(s).
    Lately, whenever I run F@H on my home computer, it suddenly shuts down, and a burnt smell rises from the case. Stopping F@H seems to fix this.. Broken PSU?

  • Mt_GoatMt_Goat Head Cheezy Knob Pflugerville (north of Austin) Icrontian
    edited March 2016

    @Jokke said:
    ......
    Lately, whenever I run F@H on my home computer, it suddenly shuts down, and a burnt smell rises from the case. Stopping F@H seems to fix this.. Broken PSU?

    Nah, that's just the dust from not being used burning off. ;D

    primesuspectbells
  • SazbeanSazbean Paladins. Lawful good & violent about it. Chelsea, MI Icrontian

    I'm a damn hippie and rather conserve power by turning off my machine than have it running. F@H was really running the fan even on low power and making a lot of noise. I'd prefer if they let me have more control over when it runs (specific times and what it can do hardware usage). I think they need to make smaller compute packages to make it worthwhile for me (as well as letting me run it quietly from something like 2am-6am). I was excited about trying to help SCIENCE!!! tho. :sad:

  • _k_k P-Town, Texas Icrontian
    edited March 2016

    You could set-up a bat script to launch the program already set to a desired level of utilization(idle/light/medium) and then kill everything at a set time.

  • SnarkasmSnarkasm Madison, WI Icrontian

    Picked back up as of yesterday.

    primesuspect
  • SnarkasmSnarkasm Madison, WI Icrontian

    That said, I still can't guarantee this into perpetuity, though I'll certainly try. We're moving into a new house soon, going to be testing the waters with new electricity bills, etc. I'm probably not trending up, either, unless the algorithms are getting better and my existing hardware will process them faster.

    I guess what I'm trying to say is, if you're depending on my 600-800k PPD to be happy with where we're at with regards to Folding, I can't be that reliable, and we should grow more people so I'm not generating 15-20% of all points on my single point of failure.

    primesuspectGarg
  • SonorousSonorous F@H Fanatic US Icrontian

    @Snarkasm said:

    I helped a little with that increase by swapping the GTX 660ti out for a GTX 970. I have been a little low on my PPD the last few days due to installing OSX on my main rig. You're correct though. We need more people folding overall.

    primesuspect
  • _k_k P-Town, Texas Icrontian

    I turned off a 970 for the summer. My theoretical ppd is 340k.

    SonorousGarg
  • RequitRequit That one guy Somewhere over there, I don't know Icrontian

    Alright, I'm back to folding again. The frequent BSODs have stopped, however I've only got one card running and my screen is flickering constantly. I'll work on those later.

    SonorousGargprimesuspect
  • SazbeanSazbean Paladins. Lawful good & violent about it. Chelsea, MI Icrontian

    @_k said:
    You could set-up a bat script to launch the program already set to a desired level of utilization(idle/light/medium) and then kill everything at a set time.

    True -- but that sounds a lot like work, and I have enough of that already :(

    Garg
  • sgstairsgstair Reverse Engineer Redmond, WA Icrontian

    Ack, my systems have fallen into various states of disarray in the last two weeks without me noticing. There are a few FAH client bugs that I seem to hit every now and then - Problematic systems are being remediated.

    Garg
  • SnarkasmSnarkasm Madison, WI Icrontian

    My PPD have plummeted to the mid-400s for no obviously discernible reason. Rough patch of WUs?

    Garg
  • SonorousSonorous F@H Fanatic US Icrontian

    @Snarkasm said:
    My PPD have plummeted to the mid-400s for no obviously discernible reason. Rough patch of WUs?

    I don't know what is going on. I'm having issues as well. Seems like my client folds fine for a while and then after submitting a WU it has an issue.

  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian

    I'm also having issues. I can no longer "pause" the client. I have to force kill it and then restart it manually.

  • _k_k P-Town, Texas Icrontian

    mmmwwwwwahahahahahahahah. Dance my puppets, toil and struggle! :pirate:

    This is where the v7 client helps and hurts. Previously on the old SMP clients you would end up cleaning out the AppData directory and install directory when things started to get wonky. Now with the v7 client they hide things a little more and when instability hits it makes it harder to resolve.

    @Snarkasm @Sonorous Now is the time to make sure all your clients are connected back to a main v7 install so you can monitor/monitor WU logs ensuring that your frames, percentages, are not have a crazy variance in time to complete. If it is not a issue on frames then make sure when you finish a WU that the upload and reassignment happens timely. You can help build buffers by setting an advanced option to download the next WU at X% of the currently active unit. Finally the last thing to monitor is GPU utilization ensuring that the fahcore process is running at full steam on the system; a mix of GPU-z and HWMonitor is good for this since you guys know how to indirectly know load percentages.

    @primesuspect You might need to dump the slot and make it again. You can do a few things. If the GPU slot doesn't pause then verify the same behavior is seen on a CPU slot. Also you need to make sure that shutting down the client and rebooting doesn't dump the work progress which should show if the clients are shutting down successfully when under fahControl.

    f@h clients have always need to be bumped from time to time. The v7 client is insanely robust, almost as good as the old hacked in bigadv linux SMP client; kraken was the shit for months.

    Garg
  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian

    When I fire up FAHcontrol, it just says "connecting". it won't connect to the client.

  • SonorousSonorous F@H Fanatic US Icrontian

    @_k said:
    mmmwwwwwahahahahahahahah. Dance my puppets, toil and struggle! :pirate:

    This is where the v7 client helps and hurts. Previously on the old SMP clients you would end up cleaning out the AppData directory and install directory when things started to get wonky. Now with the v7 client they hide things a little more and when instability hits it makes it harder to resolve.

    @Snarkasm @Sonorous Now is the time to make sure all your clients are connected back to a main v7 install so you can monitor/monitor WU logs ensuring that your frames, percentages, are not have a crazy variance in time to complete. If it is not a issue on frames then make sure when you finish a WU that the upload and reassignment happens timely. You can help build buffers by setting an advanced option to download the next WU at X% of the currently active unit. Finally the last thing to monitor is GPU utilization ensuring that the fahcore process is running at full steam on the system; a mix of GPU-z and HWMonitor is good for this since you guys know how to indirectly know load percentages.

    @primesuspect You might need to dump the slot and make it again. You can do a few things. If the GPU slot doesn't pause then verify the same behavior is seen on a CPU slot. Also you need to make sure that shutting down the client and rebooting doesn't dump the work progress which should show if the clients are shutting down successfully when under fahControl.

    f@h clients have always need to be bumped from time to time. The v7 client is insanely robust, almost as good as the old hacked in bigadv linux SMP client; kraken was the shit for months.

    I'm seeing %100 GPU utilization of the GTX 970 for a few hours then the core clock drops to about 950mhz and stays there. I reinstalled my drivers and removed my overclock on the GPU and that hasn't fixed the issue. This seemed to start after the last GeForce driver update. I'll dissect your post and try to sort out what's going on.

  • primesuspectprimesuspect Beepin n' Boopin Detroit, MI Icrontian

    I dunno. I rebooted and now it's folding again.

Sign In or Register to comment.