Hi All,
I have a design protocol implemented in PyRosetta that has a very large memory footprint ~ 4Gb. I ran a python memory profiler on the design script and discovered that the sharp memory consumption increase occurs upon the initialization of PackRotamerMover (I presume this is where the rotamer library is instantiated). I have taken the following measures to reduce the memory consumption with no success:
1. Importing just the needed rosetta modules (as opposed to "from rosetta import *)
2. Removing -ex1 -ex2 , etc from the command line args on the init func
3. using the -linmem_ig 10 / 20 flag
4. Removing unnecessary patches from the database
Does anyone have an idea what do in order to reduce the memory consumption?
Lior Z
Using -linmem_ig 10 is the standard suggestion to reduce memory usage and speed up design runs. As you're using PyRosetta, I'd recommend using the or_linmem_ig() method on the PackerTask directly to enable it. The commandline option *should* enable it by default, but it wouldn't hurt to enable it directly. You could also try or_double_lazy_ig() and see if that improves anything.
What I might do is use the memory profiler for different PackerTask settings (e.g. different resfiles). That is, start with all NATRO and see what the memory consumption is. Then turn on NATAA for a few residues, then gradually up to all residues. Then add PIKAA for a few positions, and gradually work your way up to your full design. Doing this gradually should tell you at what point your memory usage explodes. If it's at NATRO, your problem isn't the rotamer library. If it's somewhere in the middle, you should get a sense of how many and which residues you can pack/design with your protocol.
Also, do you have an non-protein ligands or non-canonicals in your protein? If so, I'd check the rotamer libraries for them, and see if that's an issue. (Try turning off packing just for them). Expecially if you have a large number of hydroxyls on them, or many rotatable bonds, you can get *huge* rotamer libraries if you're not careful.
Are you using monolith or namespace build? Monolith builds can ideed allocate that amount of memory, i would recommend to try to use namespace build and see if that helps.
Yea, namespace build did reduce the memory consumption dramatically. (from 4 Gb to 1.1)
Thanks!