You are here

std::bad_alloc error after changing .xml script?

11 posts / 0 new
Last post
std::bad_alloc error after changing .xml script?
#1

Hello,

Rosetta was working fine for me with the following .xml script:

<ROSETTASCRIPTS>
 
 <TASKOPERATIONS> You can control which parts you want to change from here
     
    ProteinInterfaceDesign name="pido" repack_chain1="1" repack_chain2="1" design_chain1="1" design_chain2="0" interface_distance_cutoff="10"/> task operation that designates which residues are designable and repackable at the interface
 </TASKOPERATIONS>

 <FILTERS> This part enables to output useful values for post-processing
    <Ddg name="ddG" scorefxn="talaris2014" threshold="-15" repeats="2"/> binding energy calculation; an average of two repeats is computed for better numerical accuracy
    <Sasa name="sasa" threshold="100"/> Buried surface area upon complex formation
    <Rmsd name="rmsd" confidence="0"/> confidence=0 means that the filter will be evaluated but not used as an acceptance criterion
    <TotalSasa name="tsasa" threshold="200"/>
    <CompoundStatement name="ddg_sasa"> combine filters into a single logical statement
      <AND filter_name="ddG"/>
      <AND filter_name="sasa"/>
      <AND filter_name="tsasa"/>
    </CompoundStatement>
 </FILTERS>


 <MOVERS>
    <Docking name="docking" score_high="soft_rep" fullatom="1" local_refine="1"/> Invokes RosettaDock local-refinement (in full-atom) with a soft potential
    <BackrubDD name="backrub" partner1="0" partner2="1" interface_distance_cutoff="8.0" moves="1000" sc_move_probability="0.25" scorefxn="talaris2014" small_move_probability="0.15" bbg_move_probability="0.25"/> perturb the backbone of chain2. Change moves to 1000 for real application
    <MutateResidue name="D63A" target="63A" new_res="ALA"/>
    <MutateResidue name="E64A" target="64A" new_res="ALA"/>
    <MutateResidue name="E67A" target="67A" new_res="ALA"/>
    <RepackMinimize name="des1" scorefxn_repack="soft_rep" scorefxn_minimize="soft_rep" minimize_bb="0" minimize_rb="1"/>
    <RepackMinimize name="des2" scorefxn_repack="talaris2014" scorefxn_minimize="talaris2014" minimize_bb="0" minimize_rb="1"/> Design and minimization at the interface
    <RepackMinimize name="des3" minimize_bb="1"/>
  
     <ParsedProtocol name="design">
      <Add mover_name="des1"/>
      <Add mover_name="des2"/>
      <Add mover_name="des3"/>
      <Add mover_name="backrub"/>
      <Add mover_name="des3" filter_name="ddg_sasa"/>
    </ParsedProtocol>
    <GenericMonteCarlo name="iterate" scorefxn_name="talaris2014" mover_name="design" trials="3"/> Iterate through design to find best solution. change trial numbers to higher for real application.
 </MOVERS>
  
 <PROTOCOLS>
    <Add mover="docking"/>
    <Add mover="D63A"/>
    <Add mover="E64A"/>
    <Add mover="E67A"/>
    <Add mover="iterate"/>
    <Add filter="ddG"/>
    <Add filter="sasa"/>
    <Add filter="tsasa"/>
    <Add filter="rmsd"/>
  </PROTOCOLS>
</ROSETTASCRIPTS>

 

 

However, I modified this script to this, and now it crashes as soon as it begins with a std::bad_alloc error:

 

<ROSETTASCRIPTS>
 
 <TASKOPERATIONS> You can control which parts you want to change from here
     
    <ProteinInterfaceDesign name="pido" repack_chain1="1" repack_chain2="1" design_chain1="0" design_chain2="1" interface_distance_cutoff="7"/> task operation that designates which residues are designable and repackable at the interface
 </TASKOPERATIONS>

 <RESIDUE_SELECTORS>
    <Chain name="chB" chains="B"/>
 </RESIDUE_SELECTORS>

 <SIMPLE_METRICS>
    InteractionEnergyMetric name="interface_energy" scoretypes_only="talaris2014"	 
    <RMSDMetric name="rmsd" rmsd_type="rmsd_all" residue_selector="chB" use_native="true"/>
    <TotalEnergyMetric name="total" residue_selector="chB"/>
    <TimingProfileMetric name="clock"/>
 </SIMPLE_METRICS>

 <MOVERS>
    <MutateResidue name="D63A" target="63A" new_res="ALA"/>
    <MutateResidue name="E64A" target="64A" new_res="ALA"/>
    <MutateResidue name="E67A" target="67A" new_res="ALA"/>
    <AtomTree name="tree" fold_tree_file="properFT"/> This works 
    <RepackMinimize name="des1" scorefxn_repack="soft_rep" scorefxn_minimize="soft_rep" minimize_bb_ch1="0" minimize_bb_ch2="1" minimize_rb="0" task_operations="pido"/> This works
    <RepackMinimize name="des2" scorefxn_repack="talaris2014" scorefxn_minimize="talaris2014" minimize_bb_ch1="0" minimize_bb_ch2="1" minimize_rb="0" task_operations="pido"/> This works
    <RunSimpleMetrics name="metric1" metrics="clock,rmsd,total" prefix="m1_"/>
    <RunSimpleMetrics name="metric2" metrics="clock,rmsd,total" prefix="m2_"/>
     <ParsedProtocol name="design">
      <Add mover_name="tree"/>
      <Add mover_name="des1"/>
      <Add mover_name="des2"/>
    </ParsedProtocol>
    <GenericMonteCarlo name="iterate" scorefxn_name="talaris2014" mover_name="design" trials="3"/> Iterate through design to find best solution. change trial numbers to higher for real application.
 </MOVERS>
  
 <PROTOCOLS>
    <Add mover="D63A"/>
    <Add mover="E64A"/>
    <Add mover="E67A"/>
    <Add mover="metric1"/>
    <Add mover="iterate"/>
    <Add mover="metric2"/>
 </PROTOCOLS>
</ROSETTASCRIPTS>

 

I understand that std::bad_alloc is an error associated with memory, but I am trying to figure out what part of my new .xml script is responsible for this behavior.

I ran both scripts on a CPU that was assigned 12G of memory, and again, nothing was changed except for the script, but before it was working fine and now I am getting this error.

Thanks for any help.

Category: 
Post Situation: 
Wed, 2020-03-04 11:44
chrisHKL

Can you post the full log?  It's pretty difficult to find exactly what is different between the two scripts.  I see you added some SimpleMetrics, which should work fine, but can't see anything past that.

Wed, 2020-03-04 12:00
jadolfbr

Hello,

Posting this in 2 separate comments due to file attachment limits:

Attached to this comment (singleTEST.txt) is the rosetta log using the original script. I cancelled the job after seeing it run for a few minutes.

File attachments: 
Wed, 2020-03-04 12:53
chrisHKL

And this is the rosetta log when using the new script where it crashes after about 30 seconds with std::bad_alloc.

 

File attachments: 
Wed, 2020-03-04 12:54
chrisHKL

Also, could you post the contents of the ROSETTA_CRASH.log file that was generated when the crash occurred?  That has the backtrace, which indicates exactly what was calling what when the crash occurred.

Wed, 2020-03-04 12:46
vmulligan

I don't have any other log files other than the ones I've attached to the other comment. I don't know if there is an input flag to generate a ROSETTA_CRASH file, but if there is, could you let me know and I can generate one.

The only other ouptut I have is from slurm (I am using a supercomputer cluster to run these jobs) which just says:

“Starting Rosetta run at: Wed Mar 4 14:02:58 EST 2020”
terminate called after throwing an instance of 'std::bad_alloc'
 what(): std::bad_alloc
srun: error: gra821: task 0: Exited with exit code 1

Wed, 2020-03-04 13:01
chrisHKL

I would start with updating your Rosetta version. The one you are using is almost a year old.  I would attempt to run it locally or on an interactive node to better get an output on a single core.  I would also think about pre-refining your input structure - this is a general thing to do when working pdb files in Rosetta.  
 

As it sits, the log doesn't even have the XML parsing in it, so I'm assuming that this is in MPI and this isn't the head node? If you can consolidate this to a single processor and update Rosetta, we should have a better idea.  I've looked over your script over and over and I can't see anything generally wrong. But I would certainly update and try on a single processor. Perhaps a bug fix has happened since then or a better error message for the problem you are seeing.

Wed, 2020-03-04 14:13
jadolfbr

Well when I am accessing the computer cluster to run this job, I AM accessing the head node, although I am running the executable rosetta_scripts.mpi.linuxiccrelease if that's what you mean by being in MPI. 

In fact, all the rosetta executables in the bin folder on this computer cluster are of the form XXX.mpi.linuxiccrelease

So when you say "consolidate this to a single processor", do you mean just run the  rosetta_scripts.mpi.linuxiccrelease executable with 1 CPU (in which case, I get the output log attached)? Or do you mean I should try using a rosetta_scripts.static.linuxiccrelease executable?

File attachments: 
Wed, 2020-03-04 15:27
chrisHKL

Please see comment #10 for an update.

Wed, 2020-03-04 17:57
chrisHKL

So I've figured out what the problem was:

When I ran the 2nd script on my local machine, it showed the following error messages and it was complaining about the AtomTree mover that I had added:

protocols.rosetta_scripts.ParsedProtocol: =======================BEGIN MOVER AtomTree - tree=======================
protocols.protein_interface_design.movers.SetAtomTree: Applying fold_tree: FOLD_TREE  EDGE 1 242 -1  EDGE 242 243 1 
core.conformation.Conformation: [ ERROR ] Error in assigning a FoldTree to a Conformation - size mismatch.
core.conformation.Conformation: [ ERROR ] Conformation of length 441: A[ALA:NtermProteinFull]PMAEGGGQNHHEVVKFMDVYQRSYCHPIETLVDIFQEYPDEIEYIFKPSCVPLMRCGGCCNAAGLACVPTEESNITMQIMRIKPHQGQH[HIS_D]IGEMSFLQHNKCECRPKKDRARQENCDKPRRAPMAEGGGQNHHEVVKFMDVYQRSYCHPIETLVDIFQEYPDEIEYIFKPSCVPLMRCGGCCNDEGLECVPTEESNITMQIMRIKPHQGQHIGEMSFLQHNKCECRPKKDRARQENCDKPRR[ARG:CtermProteinFull]T[THR:NtermProteinFull]GRPFVEMYSEIPEIIH[HIS_D]MTEGRELVIPCRVTSPNITVTLKKFPLDTLIPDGKRIIWDSRKGFIISNATYKEIGLLTCEATVNGHLYKTNYLTHRQTNTIIDVQISTPRPVKLLRGHTLVLNCTATTPLNTRVQMTWSYPDEKNKRASVRRRIDQSNSHANIFYSVLTIDKMQNKDKGLYTCRVRSGPSFKSVNTSVHIY[TYR:CtermProteinFull]
core.conformation.Conformation: [ ERROR ] FoldTree of length 243: FOLD_TREE  EDGE 1 242 -1  EDGE 242 243 1 

ERROR: Conformation: fold_tree nres should match conformation nres. conformation nres: 441 fold_tree nres: 243
ERROR:: Exit from: src/core/conformation/Conformation.cc line: 828
protocols.rosetta_scripts.ParsedProtocol: [ ERROR ] Exception while processing procotol: 

File: src/core/conformation/Conformation.cc:828
[ ERROR ] UtilityExitException
ERROR: Conformation: fold_tree nres should match conformation nres. conformation nres: 441 fold_tree nres: 243


protocols.rosetta_scripts.ParsedProtocol: [ ERROR ] Exception while processing procotol: 

File: src/core/conformation/Conformation.cc:828
[ ERROR ] UtilityExitException
ERROR: Conformation: fold_tree nres should match conformation nres. conformation nres: 441 fold_tree nres: 243


protocols.jd2.JobDistributor: [ ERROR ] 

[ERROR] Exception caught by JobDistributor for job 200ns1-101_AB_0012

[ ERROR ]: Caught exception:


File: src/core/conformation/Conformation.cc:828
[ ERROR ] UtilityExitException
ERROR: Conformation: fold_tree nres should match conformation nres. conformation nres: 441 fold_tree nres: 243

AN INTERNAL ERROR HAS OCCURED. PLEASE SEE THE CONTENTS OF ROSETTA_CRASH.log FOR DETAILS.

 

And this is the ROSETTA_CRASH.log:

##############################################################################################################
#
# Rosetta crash log. 
#

[START_CRASH_REPORT]
[ROSETTA_VERSION]: 2019.35+release.767c1ea25c5
[COMMIT_DATE]: 2019-08-31T05:00:04.827710
[APPLICATION]: /Applications/rosetta_bin_mac_2019.35.60890_bundle/main/source/bin/rosetta_scripts.static.macosclangrelease
[MODE]: Release
[EXTRAS]: default
[OS]: Apple Mac OS X
[COMPILER]: Clang version "9.0.0 (clang-900.0.39.2)"
[STDLIB]: libc++ version 4000
[START_OPTIONS]
 -in:ignore_unrecognized_res -in:file:s=200ns1-101_AB.pdb -in:file:native=200ns1-101_AB.pdb -in:file:fullatom -out:nstruct=1000 -out:file:silent=bin_struct.o -parser:protocol=updated2.xml -jd2:ntrials=3 -packing:extrachi_cutoff=5 -packing:linmem_ig=10 -packing:ex1 -packing:ex2 -corrections:restore_talaris_behavior -docking:no_filters

[END_OPTIONS]

[START_BACKTRACE]: RAW_LIBC
0   rosetta_scripts.static.macosclangrelease 0x0000000107ac308f backtrace_string(int) + 79
1   rosetta_scripts.static.macosclangrelease 0x0000000107ac2fcc utility::excn::Exception::Exception(char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 204
2   rosetta_scripts.static.macosclangrelease 0x00000001060a9a18 protocols::jd2::JobDistributor::go_main(std::__1::shared_ptr<protocols::moves::Mover>) + 1224
3   rosetta_scripts.static.macosclangrelease 0x00000001060a4e42 protocols::jd2::FileSystemJobDistributor::go(std::__1::shared_ptr<protocols::moves::Mover>) + 66
4   rosetta_scripts.static.macosclangrelease 0x0000000103fa2bbd main + 1149
5   libdyld.dylib                       0x00007fff7e9a73d5 start + 1
6   ???                                 0x0000000000000002 0x0 + 2

[END_BACKTRACE]

[FILE]: src/protocols/jd2/JobDistributor.cc
[LINE]: 327
[START_MESSAGE]
1000 jobs failed; check output for error messages

[END_MESSAGE]
[END_CRASH_REPORT]

 

Both message are repeated 1000 times respectively, because I had set my -nstruct flag to 1000.

For some reason I don't see these error messages or the crash log when I run the script on the computer cluster with  "> log.txt", and that's why I didn't realize this was the issue initially, but I suppose I really should have tested everything on my local machine first before moving to the computer cluster anyways.

 

More about the AtomTree mover:

My input structure is a ligand (chain A, residues 1-242), bound to a receptor (chain B, residues 243-441). When I initially wrote the script without the AtomTree, Rosetta displayed the default FOLD_TREE in the output as:

FOLD_TREE  EDGE 1 242 -1  EDGE 1 243 1  EDGE 243 441 -1 

 

I'm not sure why the 2nd edge is connecting residue 1 to 243 (could it be because of a lack of a TER line in my pdb file between residues 242 and 243? This line was originally there but was apparently removed when I ran the clean_pdb.py script found at $ROSETTAPATH/tools/protein_tools/scripts/, as was recommended when I first tried to use rosetta)

but I thought it should be between 242 and 243, so the "properFT" file that I am calling in the AtomTree mover is just:

FOLD_TREE EDGE 1 242 -1 EDGE 242 243 1 EDGE 243-441 -1 

 

I'm not sure what exactly Rosetta is complaining about my AtomTree, but I've removed the AtomTree mover and am just happy it's working again for now.

 

Wed, 2020-03-04 17:55
chrisHKL

Good to know. The crash log is a newer addition to Rosetta - so the older version on your cluster won't have this. By running on your laptop, you consolidated the output - when running the MPI version in a job, the head node probably failed with this - but also - sometimes some output is lost.  

The Foltree of -1 is a peptide edge.  The foldtree of 1 is a jump.  The input FT looks correct. The jump is between the FT root and the start of your chain B.  This should be correct as it will allow rigid body movement of chain B.  THis is usually how Rosetta does it. 

 

For your FT, it also might work as you want a jump between the last residue in chain A and the start of chain B.  Your FT says 243-441, you would need to remove the dash. 
 

Thu, 2020-03-05 07:02
jadolfbr