# Denovo density corrupted silent file

5 posts / 0 new
Denovo density corrupted silent file
#1

I'm running a denovo density routine on a server. Oddly enough, the silent files I genrate there are "corrupted" somehow. When I try to score them, I get the following error:

core.init: Rosetta version rosetta.source.release-152:d6547f6858282e67c41356255dfdd63b0e7a75aa 2017-09-08 02:01:37.594540 from http://www.rosettacommons.org
core.init: command: /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease -mode score -in::file::silent /home/labusr/Desktop/local_run/standalone/pacrg_fap20*silent -scorefile /home/labusr/Desktop/local_run/standalone/scores321 -n_matches 50
core.init: 'RNG device' seed mode, using '/dev/urandom', seed=-337716636 seed_offset=0 real_seed=-337716636
core.init.random: RandomGenerator:init: Normal mode, seed=-337716636 RG_type=mt19937
core.init: Resolved executable path: /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/denovo_density.default.linuxgccrelease
core.init: Looking for database based on location of executable: /home/labusr/rosetta/main/database/

ERROR: ERROR: Unable to open silent_input file: '/home/labusr/Desktop/local_run/standalone/pacrg_fap20*silent'
ERROR:: Exit from: src/core/io/silent/SilentFileData.cc line: 211
BACKTRACE:
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libutility.so(utility::exit(std::string const&, int, std::string const&, int)+0x27f) [0x7fbca95f048f]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::read_tags_fast(std::string const&, utility::vector1<std::string, std::allocator<std::string> >&) const+0x37b) [0x7fbcac3e68db]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::read_tags_fast(std::string const&) const+0x3d) [0x7fbcac3e7c3d]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::_read_file(std::string const&, bool)+0x74) [0x7fbcac3ed964]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::read_file(std::string const&)+0x2e) [0x7fbcac3ee78e]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::SilentFileJobInputter::fill_jobs(protocols::jd2::JobsContainer&)+0x5d3) [0x7fbcae4e0893]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_job_list_from_job_inputter()+0xb7) [0x7fbcae4acba7]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::init_jd()+0x12e) [0x7fbcae4ad4ae]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::JobDistributor()+0xc3) [0x7fbcae4b1523]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::FileSystemJobDistributor::FileSystemJobDistributor()+0x14) [0x7fbcae489434]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributorFactory::create_job_distributor()+0x1b3) [0x7fbcae4b27e3]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_instance()+0x128) [0x7fbcae4b1a18]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JD2ResourceManager::has_resource_with_description(std::string const&)+0x24) [0x7fbcae48e4c4]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::scoring::electron_density::getDensityMap(std::string, bool)+0x48) [0x7fbcabc12488]
/home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x43347c]
/home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x4297f7]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fbca75e5c05]
/home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x429f59]
Error: [ ERROR ] ERROR: Exception caught by JobDistributor while trying to fill the input jobs with JobInputter of type type 'SilentFile'
Error: [ ERROR ]

[ ERROR ] EXCN_utility_exit has been thrown from: src/core/io/silent/SilentFileData.cc line: 211
ERROR: ERROR: Unable to open silent_input file: '/home/labusr/Desktop/local_run/standalone/pacrg_fap20*silent'

ERROR:: Exit from: src/protocols/jd2/JobDistributor.cc line: 1154
BACKTRACE:
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libutility.so(utility::exit(std::string const&, int, std::string const&, int)+0x27f) [0x7fbca95f048f]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.6.so(utility::exit(std::string const&, int, int)+0x24) [0x7fbcb5849d04]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_job_list_from_job_inputter()+0x19f) [0x7fbcae4acc8f]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::init_jd()+0x12e) [0x7fbcae4ad4ae]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::JobDistributor()+0xc3) [0x7fbcae4b1523]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::FileSystemJobDistributor::FileSystemJobDistributor()+0x14) [0x7fbcae489434]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributorFactory::create_job_distributor()+0x1b3) [0x7fbcae4b27e3]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_instance()+0x128) [0x7fbcae4b1a18]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JD2ResourceManager::has_resource_with_description(std::string const&)+0x24) [0x7fbcae48e4c4]
/home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::scoring::electron_density::getDensityMap(std::string, bool)+0x48) [0x7fbcabc12488]
/home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x43347c]
/home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x4297f7]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fbca75e5c05]
/home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x429f59]
caught exception

[ ERROR ] EXCN_utility_exit has been thrown from: src/protocols/jd2/JobDistributor.cc line: 1154


Running the routine on my machine, produces files that are fine to score on my machine. I also get the same error when I try to score the .silent files generated on the server on my machine. I reinstalled Rosetta on the server, but the issue still remains. How to debug this error? Could it be version realted? I'm pretty sure I have a slightly older version on my machine than the recently installed one on the server.

Post Situation:
Mon, 2018-04-23 07:30
ahmadkhalifa

You're getting wildcards in the output:

ERROR: Unable to open silent_input file: /home/labusr/Desktop/local_run/standalone/pacrg_fap20*silent

Note the star in that output. This indicates to me that the files you're expecting to use aren't actually located at /home/labusr/Desktop/local_run/standalone/pacrg_fap20*silent -- Double check your path. I'd do an ls on the files to list the files and make sure they're present. If ls can't find the files, Rosetta won't be able to, either.

Mon, 2018-04-23 15:42
rmoretti

I'm a 100% sure the files are there. The wild card is for the .NUMBER. in between pacrg_fap20 and silent. To confirm, I just ls'ed this directoy. The files are there.

I just did the following test. I tried to score just one silent file from each run, the corrupted run on the server, and the other file that I generated on my machine. This is what comes out (same script, path, everything):

[labusr@luxor local_run]$./tubulin_fragement_score.sh core.init: Rosetta version rosetta.source.release-152:d6547f6858282e67c41356255dfdd63b0e7a75aa 2017-09-08 02:01:37.594540 from http://www.rosettacommons.org core.init: command: /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease -mode score -in::file::silent /home/labusr/Desktop/local_run/pacrg_fap20*silent -scorefile /home/labusr/Desktop/local_run/scores321 -n_matches 50 core.init: 'RNG device' seed mode, using '/dev/urandom', seed=-295085568 seed_offset=0 real_seed=-295085568 core.init.random: RandomGenerator:init: Normal mode, seed=-295085568 RG_type=mt19937 core.init: Resolved executable path: /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/denovo_density.default.linuxgccrelease core.init: Looking for database based on location of executable: /home/labusr/rosetta/main/database/ ERROR: ERROR: Unable to open silent_input file: '/home/labusr/Desktop/local_run/pacrg_fap20*silent' ERROR:: Exit from: src/core/io/silent/SilentFileData.cc line: 211 BACKTRACE: /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libutility.so(utility::exit(std::string const&, int, std::string const&, int)+0x27f) [0x7f9439aa448f] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::read_tags_fast(std::string const&, utility::vector1<std::string, std::allocator<std::string> >&) const+0x37b) [0x7f943c89a8db] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::read_tags_fast(std::string const&) const+0x3d) [0x7f943c89bc3d] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::_read_file(std::string const&, bool)+0x74) [0x7f943c8a1964] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::io::silent::SilentFileData::read_file(std::string const&)+0x2e) [0x7f943c8a278e] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::SilentFileJobInputter::fill_jobs(protocols::jd2::JobsContainer&)+0x5d3) [0x7f943e994893] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_job_list_from_job_inputter()+0xb7) [0x7f943e960ba7] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::init_jd()+0x12e) [0x7f943e9614ae] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::JobDistributor()+0xc3) [0x7f943e965523] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::FileSystemJobDistributor::FileSystemJobDistributor()+0x14) [0x7f943e93d434] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributorFactory::create_job_distributor()+0x1b3) [0x7f943e9667e3] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_instance()+0x128) [0x7f943e965a18] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JD2ResourceManager::has_resource_with_description(std::string const&)+0x24) [0x7f943e9424c4] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::scoring::electron_density::getDensityMap(std::string, bool)+0x48) [0x7f943c0c6488] /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x43347c] /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x4297f7] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f9437a99c05] /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x429f59] Error: [ ERROR ] ERROR: Exception caught by JobDistributor while trying to fill the input jobs with JobInputter of type type 'SilentFile' Error: [ ERROR ] [ ERROR ] EXCN_utility_exit has been thrown from: src/core/io/silent/SilentFileData.cc line: 211 ERROR: ERROR: Unable to open silent_input file: '/home/labusr/Desktop/local_run/pacrg_fap20*silent' ERROR:: Exit from: src/protocols/jd2/JobDistributor.cc line: 1154 BACKTRACE: /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libutility.so(utility::exit(std::string const&, int, std::string const&, int)+0x27f) [0x7f9439aa448f] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.6.so(utility::exit(std::string const&, int, int)+0x24) [0x7f9445cfdd04] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_job_list_from_job_inputter()+0x19f) [0x7f943e960c8f] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::init_jd()+0x12e) [0x7f943e9614ae] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::JobDistributor()+0xc3) [0x7f943e965523] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::FileSystemJobDistributor::FileSystemJobDistributor()+0x14) [0x7f943e93d434] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributorFactory::create_job_distributor()+0x1b3) [0x7f943e9667e3] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JobDistributor::get_instance()+0x128) [0x7f943e965a18] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libprotocols.1.so(protocols::jd2::JD2ResourceManager::has_resource_with_description(std::string const&)+0x24) [0x7f943e9424c4] /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/libcore.3.so(core::scoring::electron_density::getDensityMap(std::string, bool)+0x48) [0x7f943c0c6488] /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x43347c] /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x4297f7] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f9437a99c05] /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease() [0x429f59] caught exception [ ERROR ] EXCN_utility_exit has been thrown from: src/protocols/jd2/JobDistributor.cc line: 1154 [labusr@luxor local_run]$ ./tubulin_fragement_score.sh
core.init: Rosetta version rosetta.source.release-152:d6547f6858282e67c41356255dfdd63b0e7a75aa 2017-09-08 02:01:37.594540 from http://www.rosettacommons.org
core.init: command: /home/labusr/rosetta/main//source/bin/denovo_density.linuxgccrelease -mode score -in::file::silent /home/labusr/Desktop/local_run/pacrg_fap20.1.silent -scorefile /home/labusr/Desktop/local_run/scores321 -n_matches 50
core.init: 'RNG device' seed mode, using '/dev/urandom', seed=765044547 seed_offset=0 real_seed=765044547
core.init.random: RandomGenerator:init: Normal mode, seed=765044547 RG_type=mt19937
core.init: Resolved executable path: /home/labusr/rosetta/main/source/build/src/release/linux/3.10/64/x86/gcc/4.8/default/denovo_density.default.linuxgccrelease
core.init: Looking for database based on location of executable: /home/labusr/rosetta/main/database/
core.io.silent.SilentFileData: Reading all structures from /home/labusr/Desktop/local_run/pacrg_fap20.1.silent
core.io.silent.SilentFileData: Finished reading 50 structures from /home/labusr/Desktop/local_run/pacrg_fap20.1.silent
protocols.evaluation.ChiWellRmsdEvaluatorCreator: Evaluation Creator active ...
core.scoring.electron_density.ElectronDensity: Loading Density Map
core.scoring.electron_density.ElectronDensity: [ WARNING ] No density map specified
core.io.silent.SilentFileData: Reading all structures from /home/labusr/Desktop/local_run/pacrg_fap20.1.silent
core.io.silent.SilentFileData: Finished reading 50 structures from /home/labusr/Desktop/local_run/pacrg_fap20.1.silent
core.chemical.GlobalResidueTypeSet: Finished initializing fa_standard residue type set.  Created 557 residue types
core.chemical.GlobalResidueTypeSet: Total time to initialize 0.37 seconds.
denovo_density: 1b: [1/1]
denovo_density: 2b: [1/1]
[labusr@luxor local_run]\$


File attachments:
Tue, 2018-04-24 08:31
ahmadkhalifa

On Unix systems, wildcard expansion is handled by the shell, not the individual programs. This means that if Rosetta gets '/home/labusr/Desktop/local_run/pacrg_fap20*silent' as a filename, it will look for the file '/home/labusr/Desktop/local_run/pacrg_fap20*silent' -- with the literal asterisk in the name.

If you want wildcard expansion, you need to have the shell expand the wildcards. Normally we see this issue with Rosetta options files, where people put tildes or wildcards in an option file. This never works, as the shell never parses the options file, and thus can't do any tilde or wildcard expansion.

In your case, though, it doesn't seem to be an option file issue, as the log indicates you passes that value on the commandline. That's why I suggested that the files might not be present - if you're using bash as a shell and the wildcard expansion doen't match any files, the wildcard pattern (including the literal asterisk) will be passed to Rosetta as-is.

Given that that's not the case, we have to look for other explanations. It's a little hard to say without seeing the contents of tubulin_fragement_score.sh, but I suspect that you may be quoting the wildcard-containing string. Quotes (eitgher single or dobule) will suppress wildcard expansion, leading to the literal asterisk being passed to Rosetta.

But regardless of why this is happening, the issue is not that the silent files are corrupt, and it's not Rosetta per se which is having the issue -- the issue is definitely with how you're making the call to Rosetta, specifically with how the wildcard is being expanded (or not) with your shell.

Tue, 2018-04-24 08:59
rmoretti

I think I had multiple bugs with my code. I took care of it, and now I realize, it's the wildcard expansion that's causing my problem now. I want to understand what " -in::file::silent /directory/pacrg_fap20*silent \" does exactly. Does it open al lthe files at once? Does it open it one by one? Would you recommend a certain way to do the same task in a python script?

Thu, 2018-04-26 09:11
ahmadkhalifa