Stinkers
drasnor
Starship OperatorHawthorne, CA Icrontian
I'm working on a script that when set up as a scheduled task will detect and remove Tinkers from my Folding farm. I started working on this script today after I found out that the two 1.1GHz Pentium III machines I set up last weekend folded two Gromacs WUs between Sunday and Monday and have been crawling the rest of the week on a single (s)Tinker. I'd like to not have to drive 120 miles each week to find out that the things are Folding those.
I'm posting this because I have two questions. First, is anyone else interested in this script for their use, and second will anyone over at FoldingCommunity have a fit that there's a tool available that streamlines removing slow WU from computers?
-drasnor
I'm posting this because I have two questions. First, is anyone else interested in this script for their use, and second will anyone over at FoldingCommunity have a fit that there's a tool available that streamlines removing slow WU from computers?
-drasnor
0
This discussion has been closed.
Comments
Quite frankly I don't think it's a good idea.
-Stephen
What if everybody did this, or at least a few people did this to any tinker WUs.
Dont post something like this at the Community. Not joking here.
I don't know what is the right answer and I won't condemn anyone who dumps sTinkers off of Intel machines though as it's just not a very efficient processor for sTinker.
I wouldn't dream of posting anything at the Community ever again. They almost killed me for my post asking for help with my stock dual Opteron eating WU. Most of them didn't even read my post, just told me to stop overclocking it and not nicely.
Suggestion: send Tinkers to Team Google Toolbar and Team Anonymous. They don't care.
-drasnor
It might be something for the team to think about though whether they start tagging processors for their ability and trying to assign as many tinkers as possible to procs that will get them done faster. Its better for them (faster results) and better for those with the boxes that stink on them (*cough* intel *cough*). It would be less beneficial to those with AMD boxen but the increased load shouldn't hurt much ... and it would be for the good of the project.
[14:18:07] Updating shared core-client information
[14:18:07] - Writing "work/wudata_01.key": (overwrite)successful.
[14:18:07] Key file to update shared file: work/wudata_01.key
[14:18:07] keyfile: 0 147 200 15 2 1 0
[14:18:07] Protein: Des2/pdb1h97.20.spa
[14:18:07] - Frames Completed: 0, Remaining: 600
[14:18:07] - Dynamic steps required: 120000
[14:18:07]
[14:18:07] Printed current.prm
[14:18:07] Writing local files:
[14:18:07] - Writing "work/wudata_01.key": (overwrite)successful.
[14:18:07] - Writing "work/wudata_01.xyz": (overwrite)successful.
[14:18:07] - Writing "work/wudata_01.prm": (overwrite)successful.
[14:18:08] Starting design engine
[14:18:08] [SPG] project name: work/wudata_01.
[14:18:08] [SPG] 1 0
[14:18:08] [SPG] Initializing protein design engine
[14:18:08] [SPG] seed = 0
[14:18:08] [SPG] Initialization complete
[14:18:08] [SPG] Writing current.pdb, chainlength = 147
[14:18:08] [SPG] Writing current.xyz
[14:18:08] [SPG] Preprocessing . . .
[14:18:08] [SPG] 147 positions in protein
[14:20:42] [SPG] Preprocessing complete
[14:20:42] Iterations: 0 of 600
[14:20:43] Finished
[14:20:43] [SPG] Native chi angles stored
[14:20:45] [SPG] Rotamers read
[14:20:45] [SPG] Starting genetic algorithm
[14:26:23] [SPG] seed: 14280159
[14:26:23] [SPG] Designing protein sequence 1 of 30
[14:34:30] [SPG] 10.0
[14:41:38] [SPG] 20.0
[14:51:09] [SPG] 30.0
[14:57:33] [SPG] 40.0
[15:03:57] [SPG] 50.0
[15:11:40] [SPG] 60.0
etc. etc. etc. But you gots to admit that Rotamers and Native chi angles could be very interesting!!
Iteration that algorithm Jack
That's the problem I'm having.
-drasnor
The reason? Stanford had found some very interesting results in data already returned and wanted to pursue that particular type of protein in more detail.
We all like the points and the competition, but let's not lose sight of the main objective. We've all seen our production fall (and threat lists grow) due to having our comps bogged down with dog WU's. It evens out in the end.
Any and all help for the team is appriciated
They could be doing better if you cared more tho.
I don't know how, I leave the comp on all day because it's folding. If anybody could tell me how to get it to do more I'd do it. Maybe when I get my next, faster, CPU?
My XP1600 (at 1.47) out folds my 1.4 (at 1.6) by a factor of 1.4x.
Yes, I wish that I could decide which wu's my boxes got. My old tbird box doesn't do too bad with tinkers. But part of the deal is living with what the project decides to give us.
Lately my production way up. Five days in a row over 200 pts. I'll take what ever I get.
Also, I know what you are talking about with the weiners over there who are always blaming any problem you are having on overclocking, like Bruce who is an admin there, among many others. :rolleyes2 You can tell with their answers that they don't have the stones or whatever to learn enough about overclocking stably to even comment on stability problems while overclocking but don't hesitate to add their snide comments(fricking idiots). :rolleyes2 I think that if you posted that you came down with a case of the crabs that some nitwit there would blame it on overclocking.;D
Side note... How does Standford determine points for WU's? Difficulty? Why would a tinker that takes several days be worth less than 20 Points when some of the other WU's get over 50 pts and take just a day?
The keyword in the "adding effciency" is ADDING. The benchmarks used are not thrown out with effciency substituted. Second, the efficiency is arunning number, IE if you get one WU that is a Tinker and you get a Gromacs after, the the efficency is the average of the two, and if you get another Gromacs, your efficiency goes up to the average of the first two plus the performance of the tinker (average of all three). Benchmarks are used to help determine the base offering, if the Gromacs servers are not real busy when your client calls in.
The base setting for the efficency is some number that could be 0 or could be .5 or could be .8. Bench plus efficiency is used to determine what is offered next. Efficiency is actually how much of the deadline time is left, and the number sent is rolled in to an average that is recalced as an average based on efficiency average including average after last completed WU plus the actual efficiency when you next turn in your WU, divided by two.
So, if you have an efficiency that averages out as about .98, you know that on average you completed WUs in 2 percent of deadline time. If you have an efficiency of .70 you know you have been folding, on average, WUs that complete in 30% of deadline time.
So, lets look at this:
WU 1 for box completes in 10% of deadline, yields a .90 efficiency. WU two completes in 30% of deadline time. Efficiency does NOT become .7, it becomes (.90+.70)/2 or 1.6/2 or .8. Third WU is finished in 5% of time of credit deadline. Efficiency for that WU is .95. your running average is now (.8 + .95)/2 or 1.75/2 or .875.
They did this to eliminate the problem of one bad WU tossing what your box gets to all tinkers after one WU.
How did I figure this out??? By looking at actual effciency for 40+ WUs versus the efficency logged and derived what mathced the actual changes shown. running averages actually evens out output as your box accumulates WUs completed. I printed out the assignment sheets, took complete time versus start in terms of what percentage of time it took, calced 1-that percentage, then did a true average and compared to the running average. With 40-50 WUs I got a very close match between the two figures. I had two boxes, so what I actually compared was a running efficency for each box divided by two versus the average of all WUs (effciency of each divided by number of WUs) and the first two didits matched, the third was close, the others did nto matter.
If, on average, your box runs WUs to completion in 20% of credit deadline time or less, you will probably get Gromacs 90% of the time or more.
My boxes have running efficency averages that are about .984-.985 (range) now....
IF previous WU effectiveness determined next WU sent, then you would have a very heavy complaint and it would be valid, but it is not calced that way. Thought you might like to know what the numbers say is actually happening. They allowed for Tinkers and odd inefficencies of single WU results when they figured out how to do the effectiveness ratings system.
Worry only if you get more than 3-4 Tinkers in a row when a client is first installed. WUs that are killed do not count toward inefficiency or efficiency, and the results will be used if turned in as much as two times the credit deadline or in some cases up to three times the credit deadline.
You will get SOME points if you turn in a WU within credit deadline, so each box will contribute to folding more than the points show if you get a lot of ineffective WUs for your box. Results are used anyway, lots of times. Those of you with farms, I would simply let them fold.
Note, the -verbosity 9 switch is used to have your client tell you its efficiency rating and lots of other things.
John D.
-drasnor
Haha, I'm with you man. I just let it fold, check the log every couple days to make sure it's still going, and check my stats to make sure my other machines are pushing work units, if my score fluctuates it's not a big deal.
KingFish
I think Stanford gives more points because they are higher on the priority list. At least comparing tinker vs gromac.
Actually, when I heard they were going to do this, I shot a PM on the folding community to Vijay to ask it be done that way. He never replied, but it DID get done that way. One other trick, the client DOES delete an old FAHlog-Prev.txt when it has a full FAHlog.txt or when it gets SIGTERM killed in some cases. What I do, to keep them, is RENAME them when I notice them appearing on my dailly to three day check of folder that folding stuff is in (I have a desktop shortcut to that folder). For the Barton, the most recent was renamed to FAHlog-Prev34-Barton.txt. I like my most recent log to have a lower number, so I started with FAHlog-Prev50-Barton.txt fro numbering.
Thus, I can mine back logs anytime I want to see how things are going and what back work changed effectiveness badly. I PM Vijay with some ideas of patterns I see, not each minor flux. If I get a bunch of things that are noticably slower on same box, I tell him what WU by project number, the hardware, and to please forward appropriately. If I get things that noticably lblze, I tell him that. Gotten some thanks for the latter, those guys like to know tunings are working or not so they can tune better.
mmonnin:
For the things that CAN be Gromac'd, they can throw more variables in at once into a WU, more steps also. Normally they will take previous work, keep what is good, add something more (usually one variable MORE), with a big Gromacs. They can play with combos of variable changes in one WU. I get a lot of humongous WUs proportionately, 2 million and 2.5 million step WUs that take 16 hours and on a slower box would take weeks. They come out to up to 70 points each, and as little as 33-40 depending on how complex they are and how long they take. The baby(insofar as complexity goes) 11 to 18 Gromacs that are just coming into the distribution stream recently actually are pretty time to points effective WUs, they also can run on slower boxes as they are less complex, and might be gradually phased in and Tinkers gradually phased out for many things. They are one answer to slower boxes not completing Gromacs in time for credit.
Gromacs let folding deal with cumulative research. I have gotten big beasts of WUs that use numbering that has TWO base project numbers in it, they in fact are fixing certain changes that worked somewhat and adding anohter change-- new change is new project number, codes between tell some of that it is, and the old project is a successful one for what they looked at that is kept so they can judge how mixes work. Expect more and more of these as things get zeroed in to better combo solutions. The Gromacs folks (it was dev'd in the NetherLands, aka Holland) also license the GROMAC style WU dev software, some of the folding boxes not used to actively run the distribution part also dev WUs with guidance from PandeGroup scientists.
Tinkers have to be limited (limited numbr of steps), and some universities that feed projects to folding use only the Tinker dev stuff. THOSE folks need many reps or clones to get enough data and change things one by one in different amounts, to do base research that lets them then use the research successes as a takeoff point to see what is more likely to really beat the cancer beast they are studying right then. (Yes, Tinker has a DEV environment, licensed, also-- more cheaply than Gromacs dev stuff is, and it can gen WUS that run nice on older boxes.)
Everyone:
I do not know that Stanford gets free use of both dev and client end for boht kinds of WU, but suspect so, as using it lets Stanford give feedback to the devs to make the dev and client end things work better, so the devs get feedback debugging help and also get the benefits of a more slaable thing to other folks later on. I think the other uni's get mostly to pay some to use the environments for dev. Thus, Stanford's folding effort does get SOME income above and beyond things for teaching folks how to run a distributed net with hands-on time and teaching, and the folding effort gets soem funds for expandign the distributed core hardware.
They could however, also use donations when folks think about tax time, just as the teams could. If anyone wants to know about that, or knows where to get info as to whether monetary donations direct to Stanford that are directed explicitly to folding's efforts are Federal Income Tax deductible as charitable, let me know, OK???
John D.