The DC Zone

DC Projects => World Community Grid => Topic started by: PiNkY on June 23, 2006, 06:23:36 PM



Title: HPF2 Begins
Post by: PiNkY on June 23, 2006, 06:23:36 PM
From Dr Bonneau at WCG...

Quote

HI FROM THE NYU CENTER FOR COMPARATIVE FUNCTIONAL GENOMICS!
ok, i'm a bit excited ...

HPF2 is launched. This is very exiting to me and to my lab members. We’ve been working with IBM for a while now and were very excited when they told us that we could put this project on the grid.

HPF2 is different from HPF1, but in a way that might not mean much to those who don’t think about structure prediction. HPF2 is a higher resolution version of HPF1. In HPF1 we modeled the protein at the level of overall shape (each part of the molecule was represented by a few atoms). In HPF2 we try to model all atoms in the protein and use a mode of Rosetta that pays attention to roughly twice the number of atoms modeled in HPF1. This means we have to spend more compute time per protein and that we have to narrow our focus for HPF2.

If you’re new to HPF then check out the FAQ for HPF2 and HPF1. There is a lot of information in the descriptions of HPF2 that refers back to HPF1 so check out the HPF1 faq and description first if your interested.

########
more details:
########

HPF phase II:

HPF phase-2 will refine, using Rosetta in a mode that accounts for greater atomic detail, the structures resulting from the first phase of the Human Proteome Folding Project (HPF phase1). The project will focus on human secreted proteins (proteins in the blood and the spaces between cells). These proteins can be important for signaling between cells and are often key markers for diagnosis. These proteins have even ended up being useful as drugs (when synthesized and given by doctors to people lacking the pro-teins). The project will also focus on key secreted pathogenic protein. This project dove-tails with efforts at the ISB in Seattle to support predictive, preventative and personalized medicine (under the assumption that these secreted proteins will be key elements of this medicine of the future).

This project continues where the Human Proteome Folding Project leaves off. With the Human proteome Folding project we aimed to get protein function. With the second phase we would aim to increase the resolution of a select subset of Human proteins. Better reso-lution is important for a number of applications including but not limited to virtual screening of drug targets with docking procedures and protein design. The second phase of the pro-ject will also serve to improve our understanding of the physics of protein structure and ad-vance the state of the art in protein structure prediction (help us to further develop our program, Rosetta).

The two main objectives are to: 1) obtain higher resolution structures for specific hu-man proteins and pathogen proteins and 2) further explore the limits of protein structure prediction by further developing Rosetta structure prediction. Thus, the project would ad-dress two very important parallel imperatives, one biological and one biophysical.

The Human Proteome Folding Project Phase-2 will use the computer power of millions of computers to predict the shape of Human proteins for which researchers currently know little. From this detailed shape scientists hope to learn about the function of these proteins, as the shape of proteins is inherently related to how they function in our bodies. This data-base of protein structures and putative functions will let scientists take the next steps un-derstanding how diseases that involve these proteins work. Proteins are the most important molecules in living beings. Just about everything in your body involves or is made out of pro-teins. Protein structure is key to understanding the functions of this diverse class of bio-molecule. Thus we hope that our work on HPF 1 and HPF 2 will contribute to critical pub-lic infrastructure to the biological and biomedical community.


Title: Re: HPF2 Begins
Post by: Teletraan on June 24, 2006, 10:42:00 AM
Thanks for the heads up, you sneaky little mouse :P


Title: Re: HPF2 Begins
Post by: Stephenish on June 24, 2006, 11:14:24 AM
Wonderful news, higher resolution is almost always good. And in this case I am certain it will be.


Title: Re: HPF2 Begins
Post by: john galbraith on June 25, 2006, 07:43:51 PM
The HPF2 agent is initially a 9+ Mb download and includes a surprize, a cpu throttle that automatically maxes cpu usage @ 60% forthe agent.  If you runs any apps, the % will drop even further, even down to 0%.  There's more info about this plus a tool download so that you can control cpu usage your self at the WCG forums.

Note:  The cpu throttle only applies to those using ud software.  BOINC is not affected.


Title: Re: HPF2 Begins
Post by: john galbraith on June 26, 2006, 10:53:52 AM
The Phase 2 units are very fast.  All I've had so far finish in around 2 hours.  Pretty quick for a 1 meg d/l.


Title: Re: HPF2 Begins
Post by: Teletraan on June 26, 2006, 12:51:26 PM
Some people are reporting inconclusive results and errors with the new work units. I've had 2 out of 3 not complete with a valid result. I'm going to wait on running any more until they get the bugs ironed out. For those running BOINC, keep an eye on your results and be sure you're returning valid ones.


Title: Re: HPF2 Begins
Post by: Stephenish on June 26, 2006, 12:57:52 PM
Is there some way to identify the HPF2 WUs by name? I'm using BOINC client to do my WCG. I'd like to know what type of work I'm doing.


Title: Re: HPF2 Begins
Post by: Teletraan on June 26, 2006, 01:01:00 PM
If you open your BOINC manager you click on the "work" tab. You'll see under "Application" a listing for hpf2 5.06 if you're running the HPF2 units. Next to that is the name of the work unit itself.


Title: Re: HPF2 Begins
Post by: john galbraith on June 26, 2006, 01:09:19 PM
Some people are reporting inconclusive results and errors with the new work units. I've had 2 out of 3 not complete with a valid result. I'm going to wait on running any more until they get the bugs ironed out. For those running BOINC, keep an eye on your results and be sure you're returning valid ones.

Completed 7 so far and got credit for all.


Title: Re: HPF2 Begins
Post by: Teletraan on June 26, 2006, 01:16:02 PM
Good to hear you're getting all good results, John. Hopefully the problem work units are a small percentage.


Title: Re: HPF2 Begins
Post by: john galbraith on June 26, 2006, 01:19:03 PM
These units do act a little strange.  Just saw one go to 23 minutes with 0%, then suddenly jumped to 16%.  The completion comes in chunks.


Title: Re: HPF2 Begins
Post by: Teletraan on June 26, 2006, 01:26:16 PM
Yes, the save points seem to be infrequent as compared to hpf1 units. What version of BOINC are you running?


Title: Re: HPF2 Begins
Post by: john galbraith on June 26, 2006, 03:08:04 PM
I run the ud software.  Don't like the idea of waiting days for results to get credited.  Also don't entirely trust the boys at Berkeley.  SETI has been a mess since switching to that platform and I've heard of other dc projects that use BOINC have had some troubles too.


Title: Re: HPF2 Begins
Post by: Teletraan on June 26, 2006, 03:43:20 PM
Gotcha, John. I think the UD software credits for all work, though, so you can't be sure if they are valid results. Is that right?


Title: Re: HPF2 Begins
Post by: john galbraith on June 26, 2006, 11:12:12 PM
You're probably right but it makes no difference to me.  Whether the results are valid or not, I spent the time crunching them.  Someone over at WCG said even invalid results are important so even better. :)


Title: Re: HPF2 Begins
Post by: john galbraith on June 28, 2006, 08:10:52 AM
Just had my first uncredited workunit of HPF2.  Shades of Grid!!!  Apparently, the ud software works the same as BOINC as far as not giving credit for aborts goes.  Switched all back to AIDS.  This has been very stable and has never had a bogus result.

Looks like the guy who wrote HPF2 went to the same school as the idiot who created the train wreck Cancer project over at grid, Clown College. >:(


Title: Re: HPF2 Begins
Post by: Ghost Plane on June 28, 2006, 09:24:20 AM
Say whatcha think, why don'tcha? :rofl:


Title: Re: HPF2 Begins
Post by: phicks on June 28, 2006, 09:43:29 AM
I use boinc and the invalid ones I've got I got points for.  The others that messed up I didn't. 


Title: Re: HPF2 Begins
Post by: phicks on June 28, 2006, 09:49:37 AM
This should help.

Quote
We have released version 5.07 of the code for the Human Proteome Folding - Phase 2 project on BOINC. This version should resolve a number of the problems members have been experiencing. In particular it should significantly reduce the occurance of the exit code 1282 and and exit code -1073741819 errors.

BOINC users will automatically recieve the new version when the client connects to the server to download new workunits. If you wish to get the new version of the application immediately, then you can reset the project (open the BOINC Manager, go to the Projects tab, select 'World Community Grid', and then click on the 'Reset Project' button).

We apologize for these problems and appreciate your patience and support while we resolve them


Title: Re: HPF2 Begins
Post by: Teletraan on June 28, 2006, 12:00:27 PM
Thanks for the update, phicks. Error 1282 is the one that I got. Nice fast response from WCG.


Title: Re: HPF2 Begins
Post by: Dark Angel on June 28, 2006, 03:30:07 PM
I just had one lock up my system and force me to reboot before it would continue.  Yesterday I had one turn sour on me on one of my other (non-oc'd) systems. Had to abort it before I could get anything else.


Title: Re: HPF2 Begins ... sort of
Post by: retsof on June 28, 2006, 04:50:28 PM
I aborted all of my waiting HPF2 units. I hadn't had any problems so far, but I'll pick up the new program version.


Title: Re: HPF2 Begins
Post by: Stephenish on June 29, 2006, 09:58:51 PM
I'm getting all HPF2 via BOINC now. Haven't noticed any issues so far.


Title: Re: HPF2 Begins
Post by: Johnny Cool1 on June 29, 2006, 10:16:03 PM
Personally, I have 20 hours of crunching that have led to:
Inconclusive
Invalid
Other

LOL!

18 hours in "Pending Violation, er, Pending Validation" Ehhh, it will show soon.

Other????

How is this for more descriptions?

"Sorry, we were disenterested"
"It seems that you were ticketed for a parking violation".
"Really? You expect this to be validated?"
"Uh, we were watching NetFlix, sorry!"
"Your Results were given a Yellow card".

LOL! Ok, more "descriptions" needed here.  ;D


Title: Re: HPF2 Begins
Post by: fingerle on June 30, 2006, 06:01:42 AM
 :lol:


Title: Re: HPF2 Begins
Post by: Ghost Plane on June 30, 2006, 06:06:50 AM
We were in the can :hoghappy:


Title: Re: HPF2 Begins
Post by: TLD on June 30, 2006, 02:50:54 PM
I thought they had that fixed.


Title: Re: HPF2 Begins
Post by: john galbraith on July 01, 2006, 07:24:28 PM
Finally decided to give BOINC a try on HPF2.  D/l'd it to one rig for the test, an AMD Athlon 3200+ with 1 gig PC3200 RAM.
The first task they sent was a FAAH unit and my rig finished it in less than 4-1/2 hrs, and that is faster than the same pc did the average FAAH unit on ud software, by at least 1-1/2 hrs.  Reset preferences to get HPF2 only and the next one was.  So far, I'm kinda disappointed.  This pc usually goes through those on ud in around 3 hrs, but on BOINC it's running slower.  After 2 hrs, 20 min. it's completed only 38.235%.  The increases come in very small increments, maybe 2% at a time, quite unlike ud where it would jump in much bigger chunks.

So where is all this speed I've been hearing about?


Title: Re: HPF2 Begins
Post by: Dark Angel on July 01, 2006, 07:36:45 PM
From what I've seen and heard the HPF2 units are hugely variable.  It could be you just got a chewy one.  The HIV units were much more chewy than the HPF1 units.


Title: Re: HPF2 Begins
Post by: john galbraith on July 01, 2006, 08:30:48 PM
I dunno.

I've got 2 A64's that crash through HPF2's in little more than 2 hours consistently.  Even a lowly 2.53 P4 takes less than 4 hrs.   All on the ud software.  If this BOINC thing ever finishes, think I'll go back to ud.  The devil you know.....


Title: Re: HPF2 Begins
Post by: Teletraan on July 02, 2006, 11:45:23 AM
John, I'd run a few more work units and then compare.


Title: Re: HPF2 Begins
Post by: john galbraith on July 02, 2006, 01:25:16 PM
You're right, of course.  Shouldn't be quick to judge. 
That first HPF2 unit took 6-1/2+ hours to finish and the next was scheduled to take nearly 4.  I suspended that one, closed BOINC and restarted the ud agent.  When the current ud job finishes, I'll switch back and give BOINC another run.


Title: Re: HPF2 Begins
Post by: Stephenish on July 02, 2006, 02:12:48 PM
A wee bit of patience goes a long way. If the work is valuable, who cares how long it takes...  ???

edit: Oh, I see, apples to oranges, trying to guess which platform is faster?


Title: Re: HPF2 Begins
Post by: Dark Angel on July 03, 2006, 04:21:58 AM
I've had another unit go strange.  Boinc decided it was missing a file and just stopped.  Restarted system and it's away again.  :doh: Not supposed to have to reboot a linux box so often.


Title: Re: HPF2 Begins
Post by: john galbraith on July 03, 2006, 12:15:53 PM
Sorry TT.  I ran another HPF2 unit on BOINC and it took nearly 6 hrs again.  That's about 50% longer than ud takes.  More points from Boinc don't matter to me.  I want to do this work as fast and efficiently as possible and it just looks like Boinc isn't the best way to do it.  Only advantage is the ability to host multiple projects and I don't run more than one anyway.  So it's ud for me. :)


Title: Re: HPF2 Begins
Post by: TLD on July 03, 2006, 01:32:34 PM
Thats odd usually BOINIC is faster. :think:


Title: Re: HPF2 Begins
Post by: john galbraith on July 03, 2006, 02:27:12 PM
That's what everyone has said but, on the machine I tested it on, it hasn't proved out.
No big deal.  I'll just keep using what works for me. :)


Title: Re: HPF2 Begins
Post by: Scribe on July 03, 2006, 02:43:50 PM
.........  So it's ud for me. :)


But UD has just anounced that the HPF they are running is all old stuff and is just make-work...... ???


Title: Re: HPF2 Begins
Post by: Dark Angel on July 03, 2006, 03:11:00 PM
John means the World Community Grid "UD" client.  ie the windows native client, as opposed to the windows BOINC client.


Title: Re: HPF2 Begins
Post by: john galbraith on July 03, 2006, 04:08:06 PM
Right.  Sorry I didn't make that clearer, but I figured most of you know how I feel about that other place. :lol:


Title: Re: HPF2 Begins
Post by: Stephenish on July 03, 2006, 08:21:36 PM
Could the work coming through BOINC be different than work being loaded on the windows "UD" client?


Title: Re: HPF2 Begins
Post by: Teletraan on July 03, 2006, 09:11:19 PM
I think it was before, stephenish. John, I would guess each client is getting different sized work units, not that one is more efficient than the other with the same units. But either choice is fine. Keep crunching  :thumb:


Title: Re: HPF2 Begins
Post by: john galbraith on July 03, 2006, 09:33:31 PM
There's no denying that you can't beat BOINC for points awarded.  It gives twice as many per hour than the ud software.  This assumes, of course, that all the work one does gets validated.  It also supports multiproc setups which I think you have.
If the points had more than bragging value, I'd be more willing to use it.  Of course, so would everyone else! :lol:   I can't compete with anyone on points, and I've risen as high as I'm ever gonna in the standings at Crunchers.  Being practical, it just makes sense to complete units in the shortest possible time.  Doing so costs me less per unit in electricity.


Title: Re: HPF2 Begins
Post by: PiNkY on July 03, 2006, 10:56:09 PM
something i do kinda wonder (it's been a while since i've run wcg at all, never tried the boinc client....so i may be completely off on this).....the ud app in the past has been explained as more of a wrapper that would trigger the rosetta app to do the processing.  would i be correct in assuming boinc is a similar wrapper type program, which also triggers the same rosetta app used in the ud version....and if so, then would they not theoretically be close to the same in processing time since they have the same code for the actual processing. :think:  which would make me think perhaps they do draw from different batches of work, causing slight differences in processing time....perhaps at one point in time, ud apps were getting a batch of slow units and boinc quick units, and vice versa now. :think:  or i could be completely wrong...just a guess.  which client you use shouldn't matter...just interesting to note the differences


Title: Re: HPF2 Begins
Post by: Stephenish on July 04, 2006, 08:27:30 AM
The only item I can think of which could slow processing time on the BOINC client is if you were running multiple projects under BOINC and had selected the "Keep in memory" option which maintains projects in memory even after the projects have switched. I have this option turned on to avoid a processing issue in [email protected] Not sure how Windows handles the "in memory" part, though, could be the retained projects get shunted to pagefile after a time.


Title: Re: HPF2 Begins
Post by: Scribe on July 04, 2006, 09:59:38 AM
The keepin memory problem with Rosie has been fixed so it is no longer required to stop the failures, however......if you do not have the keep in memory when doing multitask in Boinc you will drop back to the last saved checkpoint and lose work. Keeping in memory stoops this and as you guessed it is shunted into pagefile so does not impact anyway. 8)


Title: Re: HPF2 Begins
Post by: Teletraan on July 04, 2006, 11:01:23 AM
Pinky,
I think your analysis is right.


Title: Re: HPF2 Begins
Post by: john galbraith on July 04, 2006, 03:15:38 PM
The processing time differences I saw were upwards of 50%.  Also, I'm not so sure about one or both of the agents being just wrappers.  If they both crunched the work in the same way, they would both have the same hardware requirements.  But they don't.  The ud agent has higher reqs than BOINC, which would lead me to believe that the BOINC agent is crippled in comparison to ud, to allow slower machines to do the work.  Either that or the work units themselves are 'dumbed down' by or for BOINC to work on slow rigs, therefore running slower on ALL machines.  I call this the 'No Child Left Behind' theory, or building for the lowest common denominator. :lol:
WCG has apparently taken the attitude that old, slow computers were holding back the research and are no longer welcome.  'Course they put it nicer than that, but that's what they meant.  I can't say I disagree.  There are many good arguments for NOT using old computers for any purpose, let alone dc computing.  The only reason I can think of TO use them is that you already own them and want to put them to some use.  That is a purely personal decision. 


Title: Re: HPF2 Begins
Post by: Stephenish on July 04, 2006, 04:25:25 PM
So there IS likely a difference in the type of work units being given to WCG "UD" client vs. WCG BOINC client. Meaning apples to oranges comparison?


Title: Re: HPF2 Begins
Post by: PiNkY on July 05, 2006, 12:53:27 AM
interesting observation, john.  i didn't know there were differences in hardware req's, i wonder why.

actually theoretically - the rosetta code would probably have been provided by the people running the rosetta project, and wcg would likely take that and put it into the client...i doubt wcg would modify the code for the actual processing as that would probably defeat the purpose a bit, so one would assume the part that actually does the processing is the same....else it makes you wonder at the consistency of the final data, if one would receive the same results as the other - would seem silly if the two clients processed differently, to take the final data and look at them the same.  perhaps there's some unlooked at reason behind it all, like one client compiled to utilize optimizations that the other client doesn't.  does seem strange that there are mixed reports as to which is quicker, though...the mixed reports would lead me to think that either it's a difference in workunits themselves with one working on a tougher batch then the other....or, perhaps since i think john is reporting based on findings with hpf2, and most people probably did their testing with hpf1, perhaps both are right as to which client is quicker - ud client for hpf2 and boinc for hpf1, for whatever reason. :think:


Title: Re: HPF2 Begins
Post by: john galbraith on July 05, 2006, 07:54:09 AM
There've been a lot of people complaining in the WCG boards about the more demanding hardware requirements of the ud agent, particularly for HPF2.  In order for the results of both agents to be compatible the basic code would have to be the same, you'd think.  The difference must lie in the execution as the size of the downloads is about the same.  It's also curious about this validation business that BOINC uses.  The ud agent accepts results as is and credits them immediately but Boinc has to compare your result to at least two of the same, and as many as ten, before it will validate the result and give credit.  This process can take as long as a week. Seems like one of these agents is traveling much further to get to the same destination.
Many people are getting invalid results for which they get no credit at all, and this is p*ssing them off as well.
Boinc is the only way to go if you have Linux or Unix systems and/or you like to work on multiple projects.  It also supports multiprocessor systems.  But for people like me, who work on single projects with standard Windows computers and don't care about point totals, you have to give the edge to the ud agent for its set and forget simplicity.


Title: Re: HPF2 Begins
Post by: Teletraan on July 05, 2006, 12:13:15 PM
john,
Your observation has me curious now. Perhaps you could post the question at WCG and see if we can get an answer?


Title: Re: HPF2 Begins
Post by: Stephenish on July 05, 2006, 07:54:21 PM
For me, BOINC has posed fewer issues. The install of BOINC was easy, and hooking up with projects just takes a few seconds. The only complexity issue comes with the preference setting capability, that, to my mind, is beneficial. You could simply leave your preferences on default settings, with no additional time spent. The BOINC manager also has lovely logging capabilities that let you know what the system is doing. You won't find any logging on the "UD" client. I like to know what's going on. Plus, the added benefit of knowing my computer's time won't get wasted if one project goes down, BOINC wins hands down.