Rosetta
|
The StandardJobQueen is meant to handle the most common form of Rosetta jobs where a protocol is applied to a single input structure to generate a single output structure. Most JobQueens in Rosetta will to derive from this JobQueen. To make the process of deriving new JobQueens easy, this class's API has been carefully designed to be easy to work with and the class itself has been designed to do as much heavy lifting as possible and to leave only the responsibilities that the StandardJobQueen could not perform to the derived classes. More...
#include <StandardJobQueen.hh>
Public Member Functions | |
StandardJobQueen () | |
The StandardJobQueen constructor asks the PoseInputterFactory for a PoseInputter and creates a ResourceManager. More... | |
~StandardJobQueen () override | |
void | update_job_dag (JobDigraphUpdater &updater) override |
Starting the Job ///. More... | |
LarvalJobs | determine_job_list (JobDAGNodeID job_dag_node_index, core::Size max_njobs) override |
Create a list of LarvalJobs for each Node Index. Only can create max_njobs to save memory. More... | |
protocols::jd3::JobOP | mature_larval_job (protocols::jd3::LarvalJobCOP job, utility::vector1< JobResultCOP > const &input_job_results) override |
Mature the LarvalJob into a full Job that will be run on a processor. Calls complete_larval_job_maturation, which you will need to override. More... | |
void | note_job_completed (LarvalJobCOP job, JobStatus status, core::Size nresults) override |
The JD calls this function on completion of a LarvalJob, after updating the JobTracker. More... | |
void | completed_job_summary (LarvalJobCOP job, ResultIndex result_index, JobSummaryOP summary) override |
As each job completes, this function is called for each result coming from a LarvalJob. More... | |
std::list< JobResultID > | job_results_that_should_be_discarded () override |
The IDs of jobs that should be discarded, IE not kept in memory for the next set of job nodes. More... | |
std::list< output::OutputSpecificationOP > | jobs_that_should_be_output () override |
By default outputs all recently finished jobs for JD output (stored in recent_successes_). Override this method if you want to cull these lists. More... | |
output::ResultOutputterOP | result_outputter (output::OutputSpecification const &spec) override |
Return the bag of of PoseOutputters (in the form of a MultipleOutputter) for the Pose that has been requested and specified by a particular OutputSpecification;. More... | |
bool | has_job_previously_been_output (protocols::jd3::LarvalJobCOP job) override |
Checks the outputter to see if the job already has been output. Used for JD override behavior. More... | |
std::list< deallocation::DeallocationMessageOP > | deallocation_messages () override |
Resource Management ///. More... | |
void | process_deallocation_message (deallocation::DeallocationMessageOP message) override |
A deallocation message first sent to the JobDistributor on this host originating from a remote JobQueen. If a derived JobQueen has deallocation messages she needs to recieve, she should override derived_process_deallocation_message. More... | |
void | flush () override |
Send all buffered output to disk – called by the JobDistributor right before it shuts down if it hits an error or catches an exception that it cannot ignore. More... | |
std::string | job_definition_xsd () const override |
XSD Functions ///. More... | |
std::string | resource_definition_xsd () const override |
JobQueens may optionally define an XSD for their resource definition file which is fed to their resource manager (if they control one). If the JobDistributor is awakened with the flag "jd3::output_resource_xsd <output file>" on the command line, and if the derived JobQueen defines an XSD for resources she uses, then the JobDistributor will write out the resource definition XSD to the output file. The derived queen may return an empty string to indicate that no resources are definable for the resource manager. More... | |
void | determine_preliminary_job_list_from_xml_file (std::string const &job_def_string) |
Read from an input string representing the contents of the job-definiton XML file and construct a set of PreliminaryLarvalJobs; this function is primarily useful for testing, but could be used to organize jobs by an enterprising job distributor or by another JobQueen. More... | |
![]() | |
JobQueen () | |
~JobQueen () override | |
JobDigraphOP | create_and_set_initial_job_dag () |
Creates the initial job dag using create_initial_job_dag function and sets it to the JQ. More... | |
void | note_job_completed_and_track (LarvalJobCOP job, JobStatus status, core::Size nresults) |
Call note_job_completed in derived classes and track the job using the JobTracker. More... | |
LarvalJobs | determine_job_list_and_track (JobDAGNodeID job_dag_node_index, core::Size max_njobs) |
Call determine_job_list in derived classes and track the jobs using the JobTracker. More... | |
Static Public Member Functions | |
static std::string | job_def_complex_type_name (std::string const &type) |
Naming function for the complexTypes in the job-definition XSD. More... | |
Protected Member Functions | |
JobDigraphOP | create_initial_job_dag () override |
The job dag encodes which job nodes are independent or dependant on each other. If you have a applicationwhere all inputs are independant and only a single run of a protocol is needed, Then do not override this method. More... | |
virtual JobOP | complete_larval_job_maturation (protocols::jd3::LarvalJobCOP larval_job, utility::options::OptionCollectionCOP job_options, utility::vector1< JobResultCOP > const &input_job_results)=0 |
The derived JobQueen must define the method that takes a larval job and the job-specific options and matures the larval job into a full job. More... | |
virtual LarvalJobs | next_batch_of_larval_jobs_for_job_node (JobDAGNodeID job_dag_node_index, core::Size max_njobs) |
Called during the SJQs determine_job_list function. Override this function to create LarvalJobs for nodes other than preliminary job nodes. More... | |
GlobalJobID | current_job_index () const |
Get the current larval job index from the JobTracker. More... | |
virtual void | append_job_tag_subelements (utility::tag::XMLSchemaDefinition &job_definition_xsd, utility::tag::XMLSchemaComplexTypeGenerator &ct_gen) const |
XML Schema ///. More... | |
virtual void | append_common_tag_subelements (utility::tag::XMLSchemaDefinition &xsd, utility::tag::XMLSchemaComplexTypeGenerator &ct_gen) const |
Here, you specify which tags that are defined in the <common> element. More... | |
virtual void | parse_job_definition_tags (utility::tag::TagCOP, utility::vector1< PreliminaryLarvalJob > const &) |
This gives the derived JobQueen a chance to read any relevant information from the job-definition file. The second argument holds a PreliminaryLarvalJob for every job block and the PreliminaryLarvalJob itself has all of the information given in its corresponding block. More... | |
virtual void | determine_preliminary_job_list () |
Preliminary Larval Jobs ///. More... | |
bool | get_preliminary_larval_jobs_determined () const |
Has determine_preliminary_job_list been called? More... | |
PreliminaryLarvalJobTracker & | get_prelim_larval_job_tracker () |
Get write access to the PreliminaryLarvalJobTracker. More... | |
PreliminaryLarvalJobTracker const & | get_prelim_larval_job_tracker () const |
Get read access to the PreliminaryLarvalJobTracker. More... | |
utility::vector1< PreliminaryLarvalJob > const & | get_preliminary_larval_jobs () const |
Read access to derived JobQueens to the preliminary job list. More... | |
virtual void | note_preliminary_job_node_is_complete (PrelimJobNodeID pjn_index) |
Respond to the completion of a preliminary job node in derived SJQs. More... | |
virtual InnerLarvalJobs | refine_preliminary_job (PreliminaryLarvalJob const &prelim_job) |
Optional Larval Job Creation ///. More... | |
virtual LarvalJobs | expand_job_list (InnerLarvalJobOP inner_job, core::Size max_larval_jobs_to_create) |
Expand an InnerLarvalJob into a full set of LarvalJobs, creating nstruct LarvalJob objects Increments larval_job_counter_. More... | |
virtual StandardInnerLarvalJobOP | create_inner_larval_job (core::Size nstruct, JobDAGNodeID job_node, PrelimJobNodeID preliminary_job_node) const |
Factory method for derived classes if they wish to rely on classes derived from StandardInnerLarvalJob. This is invoked by the StandardJobQueen in her determine_job_list method just as jobs are prepared. If the base StandardInnerLarvalJob class is desired, then do not override this method. More... | |
virtual LarvalJobOP | create_larval_job (InnerLarvalJobOP job, NStructIndex nstruct_index, core::Size larval_job_index) |
Factory method for derived classes if they wish to rely on classes derived from LarvalJob. This is invoked by the StandardJobQueen in the expand_job_list method. If the base LarvalJob class is desired, then do not override this method. More... | |
InnerLarvalJobOP | create_and_init_inner_larval_job_from_preliminary (core::Size nstruct, PrelimJobNodeID prelim_job_node, JobDAGNodeID job_node) const |
Adds information regarding the job definition (job tag, input source, jobdef tag, and outputter). Uses the PreliminaryInnerLarvalJob to do so. More... | |
void | create_and_store_output_specification_for_job_result (LarvalJobCOP job, ResultIndex result_index, core::Size nresults) |
Output Specifications and Ouputters ///. More... | |
void | create_and_store_output_specification_for_job_result (LarvalJobCOP job, utility::options::OptionCollection const &job_options, ResultIndex result_index, core::Size nresults) |
Create an output specification for a given job + result index and store it in the recent_successes_ list, which will be given to the JobDistributor upon request. More... | |
virtual output::OutputSpecificationOP | create_output_specification_for_job_result (LarvalJobCOP job, utility::options::OptionCollection const &job_options, ResultIndex result_index, core::Size nresults) |
Construct the OutputSpecification for a completed successful job. More... | |
JobOutputIndex | build_output_index (protocols::jd3::LarvalJobCOP larval_job, ResultIndex result_index_for_job, core::Size n_results_for_job) |
Construct a JobOutputIndex for a given job based on "the obvious", but giving derived classes the chance to assign their own indices via the assign_output_index function. More... | |
virtual void | assign_output_index (protocols::jd3::LarvalJobCOP larval_job, ResultIndex result_index_for_job, core::Size n_results_for_job, JobOutputIndex &output_index) |
The derived job queen may assign her own numbering to output Poses if she chooses. More... | |
virtual void | derived_process_deallocation_message (deallocation::DeallocationMessageOP message) |
If the derived class has deallocation messages that it needs to broadcast to remote nodes and then to process on those remote nodes, then the StanardJobQueen will pass them along to the derived job queen using this function. More... | |
pose_outputters::PoseOutputterOP | pose_outputter_for_job (InnerLarvalJob const &innerJob) |
Create or get the PoseOutputter from the job. Cache the Outputter in the SJQ. More... | |
pose_outputters::PoseOutputterOP | pose_outputter_for_job (InnerLarvalJob const &innerJob, utility::options::OptionCollection const &job_options) |
Create or get the PoseOutputter from the job. Cache the Outputter in the SJQ. More... | |
pose_outputters::PoseOutputterOP | pose_outputter_for_job (pose_outputters::PoseOutputSpecification const &spec) |
Find the PoseOutputter for a job specification, respecting the Job-distributor provided output file suffix. More... | |
utility::vector1< pose_outputters::SecondaryPoseOutputterOP > | secondary_outputters_for_job (InnerLarvalJob const &innerJob, utility::options::OptionCollection const &job_options) |
core::pose::PoseOP | pose_for_inner_job (InnerLarvalJobCOP inner_job) |
core::pose::PoseOP | pose_for_inner_job (InnerLarvalJobCOP inner_job, utility::options::OptionCollection const &options) |
Return a copy of the Pose to be used with the given inner job. More... | |
core::pose::PoseOP | pose_for_job (LarvalJobCOP job, utility::options::OptionCollection const &options) |
Return a copy of the Pose to be used with the given job. Wrapper for pose_for_inner_job() More... | |
utility::options::OptionCollectionOP | options_for_job (InnerLarvalJob const &inner_job) const |
Options Collection ///. More... | |
utility::options::OptionCollectionOP | options_from_tag (utility::tag::TagCOP job_options_tags) const |
Create an OptionCollection from tags and OptionKeys stored in the SJQ. More... | |
pose_inputters::PoseInputterOP | pose_inputter_for_job (InnerLarvalJob const &inner_job) const |
Access the pose inputter. More... | |
void | add_options (utility::options::OptionKeyList const &opts) |
void | add_option (utility::options::OptionKey const &key) |
void | remove_default_input_element () |
Override the basic specification of input elements as either PDBs, Silent Structures, or ResourceManager defined structures. This should be called during the derived class's constructor if it is to be used. More... | |
utility::tag::TagCOP | common_block_tags () const |
Return the set of subtags that are common to the whole set of jobs in the Job definition file, if any are given. This set of tags is read from disk at most once per execution. More... | |
void | set_common_block_precedes_job_blocks (bool setting) |
By default, the common block is given before the job blocks in the job definition file. Setting this value to false reverses the order and the common block goes at the end of the file. More... | |
void | do_not_accept_all_pose_inputters_from_factory () |
Expert Override Input/Output behavior ///. More... | |
void | allow_pose_inputter (pose_inputters::PoseInputterCreatorOP creator) |
void | do_not_accept_all_pose_outputters_from_factory () |
void | allow_pose_outputter (pose_outputters::PoseOutputterCreatorOP creator) |
void | set_default_outputter (pose_outputters::PoseOutputterCreatorOP creator) |
![]() | |
JobDigraph const & | get_job_graph () const |
Get read access to the current job graph. More... | |
JobTracker & | get_job_tracker () |
Get write access to the current job tracker. More... | |
JobTracker const & | get_job_tracker () const |
Get read access to the current job tracker. More... | |
Private Member Functions | |
void | determine_preliminary_job_list_from_xml_file (std::string const &job_def_string, std::string const &job_def_schema) |
After generating the job-definition XSD, construct the preliminary job list. More... | |
void | load_job_definition_file (std::string const &job_def_string, std::string const &job_def_schema) |
void | determine_preliminary_job_list_from_command_line () |
Instead of reading a JobDefinition file, construct the set of PreliminaryLarvalJobs reading from the command line. Invoked by determine_preliminary_job_list. More... | |
pose_inputters::PoseInputSourcesAndInputters | read_input_poses_from_command_line () |
LarvalJobs | next_batch_of_larval_jobs_from_prelim (PrelimJobNodeID job_node_index, core::Size max_njobs) |
This function is called in determine_job_list. It is responsible for creating the set of LarvalJobs for ALL PreliminaryLarvalJobs. Call this function if you derive determine_job_list. More... | |
pose_outputters::SecondaryPoseOutputterOP | secondary_outputter_for_job (InnerLarvalJob const &inner_job, utility::options::OptionCollection const &job_options, std::string const &secondary_outputter_type, utility::tag::TagCOP outputter_tag) |
Get the SecondardyPoseOutputter for a job. An example of an SPO is a scorefile. More... | |
pose_outputters::SecondaryPoseOutputterOP | secondary_outputter_for_job (pose_outputters::PoseOutputSpecification const &) |
Get the SecondardyPoseOutputter for a job. An example of an SPO is a scorefile. More... | |
void | note_job_result_discarded (LarvalJobCOP job, ResultIndex result_index) |
The SJQ will keep track of all discarded jobs in the discarded_jobs_ diet. More... | |
utility::options::OptionKeyList | concatenate_all_options () const |
pose_outputters::PoseOutputterOP | get_outputter_from_job_tag (utility::tag::TagCOP tag) const |
The StandardJobQueen is meant to handle the most common form of Rosetta jobs where a protocol is applied to a single input structure to generate a single output structure. Most JobQueens in Rosetta will to derive from this JobQueen. To make the process of deriving new JobQueens easy, this class's API has been carefully designed to be easy to work with and the class itself has been designed to do as much heavy lifting as possible and to leave only the responsibilities that the StandardJobQueen could not perform to the derived classes.
protocols::jd3::standard::StandardJobQueen::StandardJobQueen | ( | ) |
The StandardJobQueen constructor asks the PoseInputterFactory for a PoseInputter and creates a ResourceManager.
References inputter_options_, load_starting_poses_only_once_, outputter_options_, prelim_job_tracker_, and secondary_outputter_options_.
|
overridedefault |
|
protected |
References options_.
|
protected |
References options_.
|
protected |
If the derived job queen (DJQ) wants to allow a PoseInputter to be used either because it is not registered with the PoseInputterFactory, or because she wants to allow only a subset of the inputters that are registered with the Factory, then she may indicate that to the StandardJobQueen with this function. If the derived job queen does not want to use all of the pose inputters provided by the factory, then she must call "do_not_accept_all_pose_inputters_from_factory()" before she calls this function. This function must only be called in the DJQ's constructor. All DJQs must invoke this method in their constructors and not simply the DJQ that lives on the "head node" in order for distributed output to workk correctly.
References inputter_creator_list_, inputter_creators_, inputter_options_, and use_factory_provided_pose_inputters_.
|
protected |
If the derived job queen (DJQ) wants to allow a PoseOutputter to be used either because it is not registered with the PoseOutputterFactory, or because she wants to allow only a subset of the outputters that are registered with the Factory, then she may indicate that to the StandardJobQueen with this function. If the derived job queen does not want to use all of the pose outputters provided by the factory, then she must call "do_not_accept_all_pose_outputters_from_factory()" before she calls this function. If the DJQ has called the "do_not_accept_all_pose_outputters_from_factory" function, then she specifies the default outputter on the first time she calls "allow_pose_outputter." This function must only be called in the DJQ's constructor.
References default_outputter_creator_, outputter_creator_list_, outputter_creators_, outputter_options_, override_default_outputter_, and use_factory_provided_pose_outputters_.
|
protectedvirtual |
Here, you specify which tags that are defined in the <common> element.
The derived JobQueen must inform the StandardJobQueen of any additional tags that belong as sub elements of the <Common> tag. The StandardJobQueen will have already appended the <Options> subtag before this function executes, so the only functions that the derived class should invoke of the XMLSchemaComplexTypeGenerator are its add_ordered_subelement_set_* functions, if you have additional subelements to add. Adding additional subelements to the complexType generator may require writing additional complexTypes to the XML Schema for the job definition file, and so that job definition xsd is passed in as well. The Tags in the <Common> block should be parsed for each Job in the mature_larval_job step; these tags are available to the derived class through the common_block_tags method. This is where data that is not constant over the course of the job can be declared (e.g. ScoreFunctions, the weights of which are often modified during the course of protocol execution), but data that is constant and that can be shared between multiple jobs ought to be initialized and held by the ResourceManager. Options provided in the <Common> block will supercede the command line, but will be overriden by any options provided for a particular job. (This is logic the StandardJobQueen manages).
Derived classes may choose to not override this method as a way to indicate that they have no common-block data that needs to be defined.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
Referenced by job_definition_xsd().
|
protectedvirtual |
XML Schema ///.
Here you can specify Schema's to use when creating your XSD for a specific JOB. AKA - you can add elements such as ResidueSelectors, task operations, etc. to parse.
The derived JobQueen must inform the StandardJobQueen of any additional tags that belong as sub elements of the <Job> tag. The StandardJobQueen will have already appended the <Input>, <Output>, and <Options> subtags before this function executes, so the only functions that the derived class should invoke of the XMLSchemaComplexTypeGenerator are its add_ordered_subelement_set_* functions, if you have additional subelements to add. Adding additional subelements to the complexType generator may require writing additional complexTypes to the XML Schema for the job definition file, and so that job definition xsd is passed in as well.
Derived classes may choose to not override this method as a way to indicate that they have no additional subtags of the <Job> tag they wish to add. This is a no-op implementation.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
Referenced by job_definition_xsd().
|
protectedvirtual |
The derived job queen may assign her own numbering to output Poses if she chooses.
No-op implementation – leave the indices the way they were initialized.
Just before a job result is written to disk, the StandardJobQueen asks the derived job queen what indices should be used to identify the soon-to-be-output Pose. The default behavior is to use the nstruct index for the primary output index, and if there are multiple result Poses from a job, to use the result index (as is) for the secondary output index. These values will already have been loaded into the output_index before this function is called. This function will possibly be invoked more than a single time for a single job result, so it is important that the derived JobQueen not assume that it only happens once. This function will only be called on JQ0; it will not be called on any other JQs, so it is welcome to use information that would only be known on the "head node"
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
Referenced by build_output_index().
|
protected |
Construct a JobOutputIndex for a given job based on "the obvious", but giving derived classes the chance to assign their own indices via the assign_output_index function.
References assign_output_index(), protocols::mean_field::max(), protocols::jd3::JobOutputIndex::n_primary_outputs, protocols::jd3::JobOutputIndex::n_secondary_outputs, protocols::jd3::JobOutputIndex::primary_output_index, and protocols::jd3::JobOutputIndex::secondary_output_index.
Referenced by create_output_specification_for_job_result().
|
protected |
Return the set of subtags that are common to the whole set of jobs in the Job definition file, if any are given. This set of tags is read from disk at most once per execution.
References common_block_tags_.
Referenced by protocols::multistage_rosetta_scripts::MRSJobQueen::parse_job_definition_tags().
|
protectedpure virtual |
The derived JobQueen must define the method that takes a larval job and the job-specific options and matures the larval job into a full job.
Implemented in protocols::multistage_rosetta_scripts::MRSJobQueen, and protocols::rosetta_scripts::RosettaScriptsJobQueen.
Referenced by mature_larval_job().
|
overridevirtual |
As each job completes, this function is called for each result coming from a LarvalJob.
Override this method to store any results you need.
Implements protocols::jd3::JobQueen.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
|
private |
References inputter_options_, options_, outputter_options_, and secondary_outputter_options_.
Referenced by job_definition_xsd(), options_for_job(), and options_from_tag().
|
protected |
Adds information regarding the job definition (job tag, input source, jobdef tag, and outputter). Uses the PreliminaryInnerLarvalJob to do so.
References create_inner_larval_job(), protocols::jd3::standard::PreliminaryLarvalJob::inner_job, and preliminary_larval_jobs_.
|
protected |
Output Specifications and Ouputters ///.
Create an output specification for a given job + result index and store it in the recent_successes_ list, which will be given to the JobDistributor upon request.
This function invokes the factory method create_output_specification_for_job_result which derive job queens may override. This version of the function will create an OptionCollection and then invoke the version of the function that expects one.
References options_for_job().
Referenced by note_job_completed().
|
protected |
Create an output specification for a given job + result index and store it in the recent_successes_ list, which will be given to the JobDistributor upon request.
This function invokes the factory method create_output_specification_for_job_result which derive job queens may override. This version of the function accepts an OptionCollection if there are multiple results for a particular job, and you can reuse the OptionCollection between calls.
References create_output_specification_for_job_result(), and recent_successes_.
|
overrideprotectedvirtual |
The job dag encodes which job nodes are independent or dependant on each other. If you have a applicationwhere all inputs are independant and only a single run of a protocol is needed, Then do not override this method.
The StandardJobQueen provides an implementation of this function which returns the most straight-forward DAG representing a set of jobs that have no interdependencies: a DAG with a single node and no edges.
Override this method if you have a need for any more job nodes other than what is created from input structures and the Job Definition file.
The base class provides a digraph with a single node – that is, all the jobs are independent of each other. This is equivalent to the kind of jobs that could be run in JD2.
Implements protocols::jd3::JobQueen.
Reimplemented in protocols::rosetta_scripts::RosettaScriptsJobQueen, protocols::multistage_rosetta_scripts::MRSJobQueenChecker, and protocols::multistage_rosetta_scripts::MRSJobQueen.
References determine_preliminary_job_list(), and preliminary_larval_jobs_.
|
protectedvirtual |
Factory method for derived classes if they wish to rely on classes derived from StandardInnerLarvalJob. This is invoked by the StandardJobQueen in her determine_job_list method just as jobs are prepared. If the base StandardInnerLarvalJob class is desired, then do not override this method.
Referenced by create_and_init_inner_larval_job_from_preliminary(), and determine_preliminary_job_list_from_xml_file().
|
protectedvirtual |
Factory method for derived classes if they wish to rely on classes derived from LarvalJob. This is invoked by the StandardJobQueen in the expand_job_list method. If the base LarvalJob class is desired, then do not override this method.
Factory method instantiates the base-class LarvalJob to start. Non-const so that derived classes may do sometthing like keep track of each of the larval jobs they instantiate.
Referenced by expand_job_list().
|
protectedvirtual |
Construct the OutputSpecification for a completed successful job.
Factory method that may be overriden by the derived job queen. Invoked by create_and_store_output_specification_for_job_result which is invoked via note_job_completed by default, and may be invoked by the derived job queen as needed.
Creates a MultipleOutputSpecification and packs it with one OutputSpecification per PoseOutputter / SecondaryPoseOutputter. The order of the OutputSpecifications that are given here needs to be recapitulated for the PoseMultipleOutputter that will be created by the call to result_outputter, as the OutputSpecifiations are iterated across in the same order as the PoseOutputters in the PoseMultipleOutputter.
References build_output_index(), pose_outputter_for_job(), and secondary_outputters_for_job().
Referenced by create_and_store_output_specification_for_job_result(), and protocols::multistage_rosetta_scripts::MRSJobQueen::note_job_completed().
|
protected |
Get the current larval job index from the JobTracker.
References protocols::jd3::JobQueen::get_job_tracker().
Referenced by expand_job_list(), and next_batch_of_larval_jobs_from_prelim().
|
overridevirtual |
Resource Management ///.
Implements protocols::jd3::JobQueen.
Reimplemented in protocols::rosetta_scripts::RosettaScriptsJobQueen.
References input_pose_index_map_, and prelim_job_tracker_.
|
protectedvirtual |
If the derived class has deallocation messages that it needs to broadcast to remote nodes and then to process on those remote nodes, then the StanardJobQueen will pass them along to the derived job queen using this function.
Reimplemented in protocols::rosetta_scripts::RosettaScriptsJobQueen.
Referenced by process_deallocation_message().
|
overridevirtual |
Create a list of LarvalJobs for each Node Index. Only can create max_njobs to save memory.
The StandardJobQueen manages the process of creating the list of LarvalJobs that will later be matured into actual jobs and run by the JobDistributor. It is not recommended that derived job queens override this method; doing so will mean that some of the data the SJQ relies on will not be initialized – see comments on the SJQs data members below to understand the consequences of overriding this method.
It is recommended to override next_batch_of_larval_jobs_for_job_node instead of this method However, if you do override this - especially for more complicated protocols - be sure to call similar functions here
The process begins by first constructing the job definition and resource definition XSDs. With these schemas, the StandardJobQueen validates the input XML files (if present). The StandardJobQueen then populates preliminary versions of LarvalJob objects./ If the XSD includes "command line options" (which may be specified either from the command line or in the <options> sections of the Job XML file), the StandardJobQueen loads the preliminary LarvalJob objects with the options. These preliminary LarvalJob objects will not have been nstruct expanded (i.e. if there are 100 nstruct for each of 5 different jobs, then there will only be 5 preliminary larval jobs created). It then passes the preliminary LarvalJob list and the TagOP objects for each preliminary LarvalJob to the derived class through the refine_job_list method.
Implements protocols::jd3::JobQueen.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueenChecker, and protocols::multistage_rosetta_scripts::MRSJobQueen.
References determine_preliminary_job_list(), next_batch_of_larval_jobs_for_job_node(), next_batch_of_larval_jobs_from_prelim(), preliminary_larval_jobs_, and preliminary_larval_jobs_determined_.
|
protectedvirtual |
Preliminary Larval Jobs ///.
Initialize the PrelimininaryLarvalJob list from input poses and the Job Definition file.
Allow the derived JobQueen to tell the StandardJobQueen to initialize the preliminary job list; this is perhaps necessary in the context of multi-round protocols when the job-definition file specifies the JobDAG. parse_job_definition_tags() is called at the end of this method.
References common_block_tags_, determine_preliminary_job_list_from_command_line(), determine_preliminary_job_list_from_xml_file(), get_preliminary_larval_jobs(), job_definition_xsd(), parse_job_definition_tags(), protocols::jd3::standard::perform_variable_substitution(), preliminary_larval_jobs_determined_, and required_initialization_performed_.
Referenced by create_initial_job_dag(), protocols::multistage_rosetta_scripts::MRSJobQueen::create_initial_job_dag(), determine_job_list(), and mature_larval_job().
|
private |
Instead of reading a JobDefinition file, construct the set of PreliminaryLarvalJobs reading from the command line. Invoked by determine_preliminary_job_list.
References get_outputter_from_job_tag(), protocols::jd3::standard::PreliminaryLarvalJob::inner_job, protocols::jd3::standard::PreliminaryLarvalJob::job_options, protocols::jd3::standard::PreliminaryLarvalJob::job_tag, protocols::jd3::nstruct_for_job(), options_from_tag(), protocols::jd3::standard::PreliminaryLarvalJob::pose_inputter, prelim_job_tracker_, preliminary_larval_jobs_, read_input_poses_from_command_line(), and representative_pose_outputter_map_.
Referenced by determine_preliminary_job_list().
void protocols::jd3::standard::StandardJobQueen::determine_preliminary_job_list_from_xml_file | ( | std::string const & | job_def_string | ) |
Read from an input string representing the contents of the job-definiton XML file and construct a set of PreliminaryLarvalJobs; this function is primarily useful for testing, but could be used to organize jobs by an enterprising job distributor or by another JobQueen.
Construct the XSD and then invoke the (private) determine_preliminary_job_list_from_xml_file method, which is also invoked by determine_preliminary_job_list.
References job_definition_xsd().
Referenced by determine_preliminary_job_list().
|
private |
After generating the job-definition XSD, construct the preliminary job list.
This is invoked both from determine_job_list_from_xml_file and determine_job_list – the latter always constructs an XSD to ensure that the derived JobQueen has properly constructed an XSD, even if a job definition file has not been provided on the command line.
References create_inner_larval_job(), get_outputter_from_job_tag(), protocols::jd3::standard::PreliminaryLarvalJob::inner_job, input_pose_counter_, inputter_creators_, job_definition_file_tags_, protocols::jd3::standard::PreliminaryLarvalJob::job_options, protocols::jd3::standard::PreliminaryLarvalJob::job_tag, load_job_definition_file(), load_starting_poses_only_once_, protocols::jd3::nstruct_for_job(), options_from_tag(), protocols::jd3::standard::PreliminaryLarvalJob::pose_inputter, prelim_job_tracker_, preliminary_larval_jobs_, preliminary_larval_jobs_determined_, read_input_poses_from_command_line(), representative_pose_outputter_map_, protocols::jd3::standard::TR(), and use_factory_provided_pose_inputters_.
|
protected |
Expert Override Input/Output behavior ///.
///
Note: (Used for Complex I/O tsuch as Membrane poses) /// ///
The derived job queen (DJQ) may tell the StandardJobQueen to only accept a subset of PoseInputters. First the DJQ says "do_not_accept_all_pose_inputters_from_factory" and then she can call "allow_pose_inputter." This function must only be called the DJQ's constructor.
References inputter_options_, and use_factory_provided_pose_inputters_.
|
protected |
The derived job queen (DJQ) may tell the StandardJobQueen to only accept a subset of PoseOutputters. First the DJQ says "do_not_accept_all_pose_outputters_from_factory" and then she can call "allow_pose_outputter." This function must only be called the DJQ's constructor.
References outputter_options_, override_default_outputter_, and use_factory_provided_pose_outputters_.
|
protectedvirtual |
Expand an InnerLarvalJob into a full set of LarvalJobs, creating nstruct LarvalJob objects Increments larval_job_counter_.
References create_larval_job(), current_job_index(), protocols::jd3::JobQueen::get_job_tracker(), protocols::jd3::JobTracker::increment_current_job_index(), protocols::mean_field::min(), and njobs_made_for_curr_inner_larval_job_.
Referenced by next_batch_of_larval_jobs_from_prelim().
|
overridevirtual |
Send all buffered output to disk – called by the JobDistributor right before it shuts down if it hits an error or catches an exception that it cannot ignore.
Implements protocols::jd3::JobQueen.
References pose_outputter_map_, and secondary_outputter_map_.
|
private |
References default_outputter_creator_, protocols::jd3::pose_outputters::PDBPoseOutputter::keyname(), outputter_creator_list_, outputter_creators_, override_default_outputter_, and use_factory_provided_pose_outputters_.
Referenced by determine_preliminary_job_list_from_command_line(), and determine_preliminary_job_list_from_xml_file().
|
protected |
Get write access to the PreliminaryLarvalJobTracker.
References prelim_job_tracker_.
|
protected |
Get read access to the PreliminaryLarvalJobTracker.
References prelim_job_tracker_.
|
protected |
Read access to derived JobQueens to the preliminary job list.
This will return an empty list if determine_preliminary_jobs has not yet been called.
References preliminary_larval_jobs_.
Referenced by protocols::multistage_rosetta_scripts::MRSJobQueen::complete_larval_job_maturation(), determine_preliminary_job_list(), and protocols::multistage_rosetta_scripts::MRSJobQueenChecker::estimate_number_of_bytes_needed_for_archiving().
|
protected |
Has determine_preliminary_job_list been called?
References preliminary_larval_jobs_determined_.
|
overridevirtual |
Checks the outputter to see if the job already has been output. Used for JD override behavior.
Gets the outputter from the Job, which is cached in the SJQ.
You should not need to override this method
Implements protocols::jd3::JobQueen.
References options_for_job(), and pose_outputter_for_job().
|
static |
Naming function for the complexTypes in the job-definition XSD.
Referenced by job_definition_xsd().
|
overridevirtual |
XSD Functions ///.
The StandardJobQueen assembles the XSD from virtual functions she invokes on the derived JobQueen: append_job_tag_subelements, append_common_tag_subelements, and add_option/add_options.
Implements protocols::jd3::JobQueen.
References append_common_tag_subelements(), append_job_tag_subelements(), common_block_precedes_job_blocks_, protocols::jd3::pose_inputters::PoseInputterFactory::complex_type_name_for_pose_inputter(), protocols::jd3::pose_outputters::PoseOutputterFactory::complex_type_name_for_pose_outputter(), concatenate_all_options(), inputter_creators_, job_def_complex_type_name(), protocols::jd3::option_type_from_key(), outputter_creators_, protocols::jd3::pose_inputters::PoseInputterFactory::pose_inputter_xml_schema_group_name(), protocols::jd3::pose_outputters::PoseOutputterFactory::pose_outputter_xml_schema_group_name(), protocols::jd3::pose_outputters::PoseOutputterFactory::secondary_pose_outputter_xml_schema_group_name(), use_factory_provided_pose_inputters_, use_factory_provided_pose_outputters_, and protocols::jd3::value_attribute_type_for_option().
Referenced by protocols::multistage_rosetta_scripts::MRSJobQueen::append_job_tag_subelements(), determine_preliminary_job_list(), and determine_preliminary_job_list_from_xml_file().
|
overridevirtual |
The IDs of jobs that should be discarded, IE not kept in memory for the next set of job nodes.
Override this method to note which jobs we don't need for the next set of Job Nodes.
Default implementation does not discard any job results.
Implements protocols::jd3::JobQueen.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
|
overridevirtual |
By default outputs all recently finished jobs for JD output (stored in recent_successes_). Override this method if you want to cull these lists.
Implements protocols::jd3::JobQueen.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
References recent_successes_.
|
private |
References common_block_tags_, job_definition_file_tags_, and protocols::jd3::standard::TR().
Referenced by determine_preliminary_job_list_from_xml_file().
|
overridevirtual |
Mature the LarvalJob into a full Job that will be run on a processor. Calls complete_larval_job_maturation, which you will need to override.
Implements protocols::jd3::JobQueen.
References complete_larval_job_maturation(), determine_preliminary_job_list(), options_for_job(), and required_initialization_performed_.
|
protectedvirtual |
Called during the SJQs determine_job_list function. Override this function to create LarvalJobs for nodes other than preliminary job nodes.
Referenced by determine_job_list().
|
private |
This function is called in determine_job_list. It is responsible for creating the set of LarvalJobs for ALL PreliminaryLarvalJobs. Call this function if you derive determine_job_list.
Updates PreliminaryLarvalJobTracker
References curr_inner_larval_job_index_, current_job_index(), expand_job_list(), inner_larval_jobs_for_curr_prelim_job_, protocols::jd3::standard::PreliminaryLarvalJob::job_options, protocols::jd3::standard::PreliminaryLarvalJob::job_tag, njobs_made_for_curr_inner_larval_job_, pose_outputter_for_job(), prelim_job_tracker_, preliminary_larval_jobs_, refine_preliminary_job(), and core::id::to_string().
Referenced by determine_job_list().
|
overridevirtual |
The JD calls this function on completion of a LarvalJob, after updating the JobTracker.
If you override this method, call the SJQs version first for PJN tracking and output.
Prepare this job for output by building an OutputSpecification for it and storing this specification in the list of recent successes. Note that tracking of completion of jobs from preliminary job nodes can only occur if the DerivedJobQueen invoked the SJQ's version of next_batch_of_larval_jobs_from_prelim in its determine_job_list method.
Implements protocols::jd3::JobQueen.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
References create_and_store_output_specification_for_job_result(), protocols::jd3::jd3_job_status_success, note_preliminary_job_node_is_complete(), options_for_job(), and prelim_job_tracker_.
|
private |
The SJQ will keep track of all discarded jobs in the discarded_jobs_ diet.
|
protectedvirtual |
Respond to the completion of a preliminary job node in derived SJQs.
Called during note_job_completed
The SJQ tracks which preliminary-job-nodes have had all of their jobs complete and lets the derived job queens know. DerivedJobQueens wishing to track when, e.g., all of the jobs that might depend on a particular Resource held by the ResourceManager have completed can override this method and then be updated by the SJQ at the completion of every PJN.
Reimplemented in protocols::rosetta_scripts::RosettaScriptsJobQueen.
Referenced by note_job_completed().
|
protected |
Options Collection ///.
Create an OptionCollection from the inner_job and OptionKeys stored in the SJQ.
References common_block_tags_, concatenate_all_options(), and protocols::jd3::options_for_job().
Referenced by protocols::multistage_rosetta_scripts::MRSJobQueen::complete_larval_job_maturation(), create_and_store_output_specification_for_job_result(), protocols::multistage_rosetta_scripts::MRSJobQueenChecker::estimate_number_of_bytes_needed_for_archiving(), has_job_previously_been_output(), mature_larval_job(), protocols::multistage_rosetta_scripts::MRSJobQueen::note_job_completed(), note_job_completed(), pose_for_inner_job(), and pose_outputter_for_job().
|
protected |
Create an OptionCollection from tags and OptionKeys stored in the SJQ.
References common_block_tags_, concatenate_all_options(), and protocols::jd3::options_from_tag().
Referenced by determine_preliminary_job_list_from_command_line(), and determine_preliminary_job_list_from_xml_file().
|
inlineprotectedvirtual |
This gives the derived JobQueen a chance to read any relevant information from the job-definition file. The second argument holds a PreliminaryLarvalJob for every job block and the PreliminaryLarvalJob itself has all of the information given in its corresponding block.
Reimplemented in protocols::multistage_rosetta_scripts::MRSJobQueen.
Referenced by determine_preliminary_job_list().
|
protected |
|
protected |
Return a copy of the Pose to be used with the given inner job.
References input_pose_index_map_, pose_inputter_for_job(), and protocols::jd3::standard::TR().
|
protected |
Return a copy of the Pose to be used with the given job. Wrapper for pose_for_inner_job()
References pose_for_inner_job().
|
protected |
Access the pose inputter.
References protocols::jd3::InnerLarvalJob::input_source(), inputter_creators_, protocols::jd3::InnerLarvalJob::job_node(), preliminary_larval_jobs_, and use_factory_provided_pose_inputters_.
Referenced by pose_for_inner_job(), and protocols::multistage_rosetta_scripts::MRSJobQueen::pose_for_inner_job_derived().
|
protected |
Create or get the PoseOutputter from the job. Cache the Outputter in the SJQ.
Access the pose outputter for a particular job; perhaps this outputter has already been created, or perhaps it needs to be created and stored.
References options_for_job().
Referenced by create_output_specification_for_job_result(), has_job_previously_been_output(), next_batch_of_larval_jobs_from_prelim(), and result_outputter().
|
protected |
Create or get the PoseOutputter from the job. Cache the Outputter in the SJQ.
References protocols::jd3::InnerLarvalJob::jobdef_tag(), protocols::jd3::InnerLarvalJob::outputter(), outputter_creators_, pose_outputter_map_, representative_pose_outputter_map_, and use_factory_provided_pose_outputters_.
|
protected |
Find the PoseOutputter for a job specification, respecting the Job-distributor provided output file suffix.
The StandardJobQueen guarantees that a different Outputter is used for each unique name as long as that name is not the empty string. This allows multiple threads to write to separate files concurrently.
References outputter_creators_, protocols::jd3::pose_outputters::PoseOutputSpecification::outputter_type(), pose_outputter_map_, representative_pose_outputter_map_, and use_factory_provided_pose_outputters_.
|
overridevirtual |
A deallocation message first sent to the JobDistributor on this host originating from a remote JobQueen. If a derived JobQueen has deallocation messages she needs to recieve, she should override derived_process_deallocation_message.
Implements protocols::jd3::JobQueen.
References derived_process_deallocation_message(), protocols::jd3::deallocation::input_pose_deallocation_msg, input_pose_index_map_, and protocols::jd3::standard::TR().
|
private |
Requests that set of input structures that have been indicated on the command line from the set of allowed PoseInputters (either all inputters registered with the PoseInputterFactory, or a subset as indicated by the derived JobQueen). Then each pose is givn a separate pose_id. Note that if this is called multiple times, then each structure that is listed on the command line will be given multiple pose_ids. As a consequence, each structure may be loaded multiple times.
References input_pose_counter_, inputter_creator_list_, protocols::jd3::standard::TR(), and use_factory_provided_pose_inputters_.
Referenced by determine_preliminary_job_list_from_command_line(), and determine_preliminary_job_list_from_xml_file().
|
protectedvirtual |
Optional Larval Job Creation ///.
Ask the derived JobQueen to expand / refine a preliminary larval job, by possibly reading per-job data out of the Tag associated with each job.
If there is nothing that needs to be done by the derived class, it may elect to use the base-class implementation of this function which simply returns a list of the inner-job pointers in the input list. Why all the back and forth? So that the derived job queens can take one input structure and multiply it into several sepearate jobs; e.g. in a ddG protocol, you need to run a separate set of jobs for the WT sequence and also for the mutant sequence. The preliminary job will reflect the structure of the WT, and then that will get expanded out. This funciton is non-const so that the derived class can even track which WT sequences it has seen so that after the first WT sequence for a particular input structure, it only expands out the mutant sequences if many mutants on that input.
If you derive this function, you will want to call the SJQs determine_job_list or next_batch_of_prelim_jobs within your own determine_job_list.
This base class implementation merely returns a one-element list containing the input inner_job. Derived classes have the flexibility to create several preliminary jobs from this input job
References protocols::jd3::standard::PreliminaryLarvalJob::inner_job.
Referenced by next_batch_of_larval_jobs_from_prelim().
|
protected |
Override the basic specification of input elements as either PDBs, Silent Structures, or ResourceManager defined structures. This should be called during the derived class's constructor if it is to be used.
TO DO
|
overridevirtual |
JobQueens may optionally define an XSD for their resource definition file which is fed to their resource manager (if they control one). If the JobDistributor is awakened with the flag "jd3::output_resource_xsd <output file>" on the command line, and if the derived JobQueen defines an XSD for resources she uses, then the JobDistributor will write out the resource definition XSD to the output file. The derived queen may return an empty string to indicate that no resources are definable for the resource manager.
Implements protocols::jd3::JobQueen.
|
overridevirtual |
Return the bag of of PoseOutputters (in the form of a MultipleOutputter) for the Pose that has been requested and specified by a particular OutputSpecification;.
This function guarantees that for each individual outputter-name (respecting the JD-provided filename suffix) that a separate set of PoseOutputters are returned so that multiple threads can potentially output at the same time.
You should not need to override this method.
Implements protocols::jd3::JobQueen.
References pose_outputter_for_job(), and secondary_outputter_for_job().
|
private |
Get the SecondardyPoseOutputter for a job. An example of an SPO is a scorefile.
Retrieve a particular secondary outputter for a job, pointing to the possibly (likely) shared outputter that several jobs will write their output to. If a representative outputter has not been created for this job (as is sometimes the case), then this function will update the representative_secondary_outputter_map_ member variable.
References representative_secondary_outputter_map_, and secondary_outputter_map_.
Referenced by result_outputter(), and secondary_outputters_for_job().
|
private |
Get the SecondardyPoseOutputter for a job. An example of an SPO is a scorefile.
References protocols::jd3::pose_outputters::PoseOutputSpecification::outputter_type(), representative_secondary_outputter_map_, and secondary_outputter_map_.
|
protected |
|
inlineprotected |
By default, the common block is given before the job blocks in the job definition file. Setting this value to false reverses the order and the common block goes at the end of the file.
References common_block_precedes_job_blocks_.
Referenced by protocols::multistage_rosetta_scripts::MRSJobQueen::MRSJobQueen().
|
protected |
If the derived job queen wants the user to get a particular PoseOutputter by default (i.e. when the user hasn't specified which outputter to use in a job definition file and in the absence of a command-line flag that says which outputter to use) instead of the default PoseOutputter chosen by the PoseOutputterFactory, then the DJQ should call this function. If the DJQ has previously called "do_not_accept_all_pose_outputters_from_factory", then the first call to "allow_pose_outputter" specified the default outputter, and a separate call to this function is not absolutely necessary. This function must only be called in the DJQ's constructor.
References default_outputter_creator_, outputter_creator_list_, outputter_creators_, override_default_outputter_, and use_factory_provided_pose_outputters_.
|
overridevirtual |
Starting the Job ///.
The StandardJobQueen's implementation is to not update the JobDAG at all: the most basic protocol defines a job DAG with only a single node.
Override this method if you have a need for updating the job dag during your protocol.
Implements protocols::jd3::JobQueen.
|
private |
Referenced by secondary_outputters_for_job().
|
private |
Referenced by job_definition_xsd(), and set_common_block_precedes_job_blocks().
|
private |
|
private |
Referenced by next_batch_of_larval_jobs_from_prelim().
|
private |
Referenced by allow_pose_outputter(), get_outputter_from_job_tag(), and set_default_outputter().
|
private |
Referenced by next_batch_of_larval_jobs_from_prelim().
|
private |
Referenced by determine_preliminary_job_list_from_xml_file(), and read_input_poses_from_command_line().
|
private |
Referenced by deallocation_messages(), pose_for_inner_job(), and process_deallocation_message().
|
private |
Referenced by allow_pose_inputter(), and read_input_poses_from_command_line().
|
private |
|
private |
|
private |
Referenced by determine_preliminary_job_list_from_xml_file(), and load_job_definition_file().
|
private |
Referenced by determine_preliminary_job_list_from_xml_file(), and StandardJobQueen().
|
private |
Referenced by expand_job_list(), and next_batch_of_larval_jobs_from_prelim().
|
private |
Referenced by add_option(), add_options(), and concatenate_all_options().
|
private |
Referenced by allow_pose_outputter(), get_outputter_from_job_tag(), and set_default_outputter().
|
private |
|
private |
|
private |
|
private |
Referenced by flush(), and pose_outputter_for_job().
|
private |
|
private |
|
private |
Referenced by create_and_init_inner_larval_job_from_preliminary(), create_initial_job_dag(), determine_job_list(), determine_preliminary_job_list_from_command_line(), determine_preliminary_job_list_from_xml_file(), get_preliminary_larval_jobs(), next_batch_of_larval_jobs_from_prelim(), and pose_inputter_for_job().
|
private |
|
private |
Referenced by create_and_store_output_specification_for_job_result(), and jobs_that_should_be_output().
|
private |
|
private |
Referenced by secondary_outputter_for_job(), and secondary_outputters_for_job().
|
private |
Referenced by determine_preliminary_job_list(), and mature_larval_job().
|
private |
Referenced by flush(), and secondary_outputter_for_job().
|
private |
Referenced by concatenate_all_options(), and StandardJobQueen().
|
private |
|
private |