Rosetta  2018.48
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Public Member Functions | Static Public Member Functions | Protected Member Functions | Static Protected Member Functions | Private Types | Private Member Functions | Static Private Member Functions | Private Attributes | Static Private Attributes | Friends | List of all members
protocols::jd2::JobDistributor Class Referenceabstract

#include <JobDistributor.hh>

Inheritance diagram for protocols::jd2::JobDistributor:
Inheritance graph
[legend]

Public Member Functions

virtual ~JobDistributor ()
 
virtual void go (protocols::moves::MoverOP mover)
 This may be overridden by derived classes. Default implementation invokes go_main. More...
 
void go (protocols::moves::MoverOP mover, JobOutputterOP jo)
 invokes go, after setting JobOutputter More...
 
void go (protocols::moves::MoverOP mover, JobInputterOP ji)
 invokes go, after setting JobInputter More...
 
void go (protocols::moves::MoverOP mover, JobInputterOP ji, JobOutputterOP jo)
 invokes go, after setting JobInputter and JobOutputter More...
 
virtual JobOP current_job () const
 Movers may ask their controlling job distributor for information about the current job. They may also write information to this job for later output, though this use is now discouraged as the addition of the MultiplePoseMover now means that a single job may include several separate trajectories. More...
 
virtual std::string current_output_name () const
 Movers may ask their controlling job distributor for the output name as defined by the Job and JobOutputter. More...
 
JobOutputterOP job_outputter () const
 Movers (or derived classes) may ask for the JobOutputter. More...
 
void set_job_outputter (const JobOutputterOP &new_job_outputter)
 Movers (or derived classes) may ask for the JobOutputter. More...
 
JobInputterOP job_inputter () const
 JobInputter access. More...
 
void set_job_inputter (JobInputterOP new_job_inputter)
 Set the JobInputter and reset the Job list – this is not something you want to do after go() has been called, but before it has returned. More...
 
virtual void mpi_finalize (bool finalize)
 should the go() function call MPI_finalize()? It probably should, this is true by default. More...
 
JobInputterInputSource::Enum job_inputter_input_source () const
 The input source for the current JobInputter. More...
 
virtual void restart ()
 
core::Size total_nr_jobs () const
 
core::Size current_job_id () const
 integer access - which job are we on? More...
 
std::string get_current_batch () const
 what is the current batch ? — name refers to the flag-file used for this batch More...
 
virtual void add_batch (std::string const &, core::Size id=0)
 add a new batch ( name will be interpreted as flag_file ) More...
 
core::Size current_batch_id () const
 what is the current batch number ? — refers to position in batches_ More...
 

Static Public Member Functions

static bool has_been_instantiated ()
 Has the job distributor been instantiated? More...
 
static JobDistributorget_instance ()
 static function to get the instance of ( pointer to) this singleton class More...
 

Protected Member Functions

 JobDistributor ()
 Singleton instantiation pattern; Derived classes will call default ctor, but their ctors, too must be protected (and the JDFactory must be their friend.) More...
 
 JobDistributor (bool empty)
 MPIArchiveJobDistributor starts with an empty job-list... More...
 
void go_main (protocols::moves::MoverOP mover)
 Non-virtual get-job, run it, & output loop. This function is pretty generic and your subclass may be able to use it. It is NOT virtual - this implementation can be shared by (at least) the simple FileSystemJobDistributor, the MPIWorkPoolJobDistributor, and the MPIWorkPartitionJobDistributor. Do not feel that you need to use it as-is in your class - but DO plan on implementing all its functionality! More...
 
JobsContainer const & get_jobs () const
 Read access to private data for derived classes. More...
 
JobsContainerget_jobs_nonconst ()
 Jobs is the container of Job objects. More...
 
void mark_job_as_completed (core::Size job_id, core::Real run_time)
 Jobs is the container of Job objects need non-const to mark Jobs as completed on Master in MPI-JobDistributor. More...
 
void mark_job_as_bad (core::Size job_id)
 
RosettaScriptsParserOP parser () const
 Parser access. More...
 
void begin_critical_section ()
 
void end_critical_section ()
 
void set_current_job_by_index (core::Size curr_job_index)
 For derived classes that wish to invoke JobDistributor functions which use the current_job_ and current_job_id_ member variables. Note that until those functions complete, it would be a bad idea for another thread to change current_job_. More...
 
bool obtain_new_job (bool re_consider_current_job=false)
 this function updates the current_job_id_ and current_job_ fields. The boolean return states whether or not a new job was obtained (if false, quit distributing!) More...
 
virtual core::Size get_new_job_id ()=0
 Return 0 to signal that no available jobs remain. Otherwise return an index into the Jobs object. More...
 
virtual void job_succeeded (core::pose::Pose &pose, core::Real run_time, std::string const &tag)
 This function is called upon a successful job completion; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter. More...
 
virtual void job_succeeded_additional_output (core::pose::Pose &pose, std::string const &tag)
 This function is called upon a successful job completion if there are additional poses generated by the mover base implementation is just a call to the job outputter. More...
 
virtual void job_failed (core::pose::Pose &, bool will_retry)
 This function is called when we give up on the job; it has been virtualized so BOINC and MPI can delay/protect output. More...
 
virtual void mark_current_job_id_for_repetition ()=0
 this function is called whenever a job "soft-fails" and needs to be retried. Generally it should ensure that the subsequent call to obtain_new_job returns this job over again. More...
 
virtual void remove_bad_inputs_from_job_list ()
 this function is called inside go_main if a mover returns FAIL_BAD_INPUT. Its purpose is to remove other jobs with the same input (which will still be bad) from the available list of jobs. The default implementation is a no-op and not all distributors are expected/able to implement this functionality, only those that can guaruntee no other jobs of that input are currently running. More...
 
virtual void current_job_finished ()
 Derived classes are allowed to clean up any temporary files or data relating to the current job after the current job has completed. Called inside go_main loop. Default implementation is a no-op. More...
 
virtual void note_all_jobs_finished ()
 Derived classes are allowed to perform some kind of action when the job distributor runs out of jobs to execute. Called inside go_main. Default implementation is a no-op. More...
 
void clear_current_job_output ()
 
virtual void handle_interrupt ()=0
 This function got called when job is not yet finished and got termitated abnormaly (ctrl-c, kill etc). when implimenting it in subclasses make sure to delete all in-progress-data that your job spawns. More...
 
void check_for_parser_in_go_main ()
 Send a message to the screen indicating that the parser is in use and that the mover that's been input to go_main will not be used, but instead will be replaced by the Mover created by the parser. More...
 
bool using_parser () const
 Is the parser in use? More...
 
bool run_one_job (protocols::moves::MoverOP &mover, time_t allstarttime, std::string &last_inner_job_tag, std::string &last_output_tag, core::Size &last_batch_id, core::Size &retries_this_job, bool first_job)
 
void setup_pymol_observer (core::pose::Pose &pose)
 After the construction of the pose for this job, check the command line to determine if the pymol observer should be attached to it. More...
 
void write_output_from_job (core::pose::Pose &pose, protocols::moves::MoverOP mover_copy, protocols::moves::MoverStatus status, core::Size jobtime, core::Size &retries_this_job)
 After a job has finished running, figure out from the MoverStatus whether the pose should be written to disk (or wherever) along with any other poses that the mover might have generated along the way. More...
 
void increment_failed_jobs ()
 Increment the number of failed jobs. More...
 
core::Size get_job_time_estimate () const
 Get an estimate of the time to run an additional job. If it can't be estimated, return a time of zero. More...
 
void set_batch_id (core::Size setting)
 set current_batch_id — eg for slave nodes in MPI framework More...
 
virtual bool next_batch ()
 switch current_batch_id_ to next batch More...
 
virtual void batch_underflow ()
 if end of batches_ reached via next_batch or set_batch_id ... More...
 
virtual void load_new_batch ()
 called by next_batch() or set_batch_id() to switch-over and restart JobDistributor on new batch More...
 
core::Size nr_batches () const
 how many batches are in our list ... this can change dynamically More...
 
std::string const & batch (core::Size batch_id)
 give name of batch with given id More...
 

Static Protected Member Functions

static void setup_system_signal_handler (void(*prev_fn)(int)=jd2_signal_handler)
 Setting up callback function that will be call when our process is about to terminate. This will allow us to exit propely (clean up in_progress_files/tmp files if any). More...
 
static void remove_system_signal_handler ()
 Set signal handler back to default state. More...
 
static void jd2_signal_handler (int Signal)
 Default callback function for signal handling. More...
 

Private Types

typedef
rosetta_scripts::RosettaScriptsParserOP 
RosettaScriptsParserOP
 

Private Member Functions

void init_jd ()
 
void populate_batch_list_from_cmd ()
 read -run:batches More...
 
void reset_job_state ()
 
void get_job_list_from_job_inputter ()
 

Static Private Member Functions

static JobDistributorcreate_singleton_instance ()
 private singleton creation function to be used with utility::thread::threadsafe_singleton More...
 

Private Attributes

JobInputterOP job_inputter_
 
JobOutputterOP job_outputter_
 
RosettaScriptsParserOP parser_
 
JobsContainerOP jobs_
 Container for the array of owning pointers to the Job objects. More...
 
JobOP current_job_
 pointer to current job. Information is somewhat duplicated with current_job_id_. More...
 
core::Size current_job_id_
 access into jobs_ object indicating current job. Contains more information than current_job_ in that it can be incremented... More...
 
core::Size last_completed_job_
 access into jobs_ object indicating the previous job. Used with the -jd2:delete_old_poses option for deleting unnecessary poses More...
 
core::Size number_failed_jobs_
 Number of failed jobs - kept track of so we can exit properly at the end of execution. More...
 
core::Size current_batch_id_
 current_batch or 0 More...
 
utility::vector1< std::string > batches_
 all batches if present More...
 

Static Private Attributes

static JobDistributorinstance_
 

Friends

class JobDistributorFactory
 

Detailed Description

Enforced JobDistributor destruction turned out to cause problems - calls to Tracers and/or the Options system during destruction caused crashes if those systems had been destructed first. So this is deprecated.

Member Typedef Documentation

Constructor & Destructor Documentation

protocols::jd2::JobDistributor::JobDistributor ( )
protected

Singleton instantiation pattern; Derived classes will call default ctor, but their ctors, too must be protected (and the JDFactory must be their friend.)

References init_jd().

protocols::jd2::JobDistributor::JobDistributor ( bool  empty)
protected

MPIArchiveJobDistributor starts with an empty job-list...

References protocols::jd2::JobDistributorFactory::create_parser(), init_jd(), job_inputter_, job_outputter_, jobs_, and parser_.

protocols::jd2::JobDistributor::~JobDistributor ( )
virtualdefault

WARNING WARNING! SINGLETONS' DESTRUCTORS ARE NEVER CALLED IN MINI! DO NOT TRY TO PUT THINGS IN THIS FUNCTION! here's a nice link explaining why: http://www.research.ibm.com/designpatterns/pubs/ph-jun96.txt

Member Function Documentation

void protocols::jd2::JobDistributor::add_batch ( std::string const &  batch,
core::Size  id = 0 
)
virtual

add a new batch ( name will be interpreted as flag_file )

add new batch to end of batch list... this might be called asynchronous... ie. while we are still in the middle of a current batch, or while we are in non-batch mode

References batch(), batches_, and protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID.

Referenced by protocols::jd2::archive::ArchiveManager::cancel_batch(), protocols::jd2::archive::ArchiveManager::queue_batch(), and protocols::jd2::archive::MPIArchiveJobDistributor::receive_batch().

std::string const& protocols::jd2::JobDistributor::batch ( core::Size  batch_id)
inlineprotected

give name of batch with given id

References batches_.

Referenced by add_batch(), and protocols::jd2::archive::MPIArchiveJobDistributor::sync_batches().

virtual void protocols::jd2::JobDistributor::batch_underflow ( )
inlineprotectedvirtual

if end of batches_ reached via next_batch or set_batch_id ...

Reimplemented in protocols::jd2::archive::MPIArchiveJobDistributor.

Referenced by next_batch(), and set_batch_id().

void protocols::jd2::JobDistributor::begin_critical_section ( )
protected

Referenced by write_output_from_job().

void protocols::jd2::JobDistributor::check_for_parser_in_go_main ( )
protected

Send a message to the screen indicating that the parser is in use and that the mover that's been input to go_main will not be used, but instead will be replaced by the Mover created by the parser.

void protocols::jd2::JobDistributor::clear_current_job_output ( )
protected
JobDistributor * protocols::jd2::JobDistributor::create_singleton_instance ( )
staticprivate

private singleton creation function to be used with utility::thread::threadsafe_singleton

References protocols::jd2::JobDistributorFactory::create_job_distributor().

Referenced by get_instance().

core::Size protocols::jd2::JobDistributor::current_batch_id ( ) const
inline
JobOP protocols::jd2::JobDistributor::current_job ( ) const
virtual

Movers may ask their controlling job distributor for information about the current job. They may also write information to this job for later output, though this use is now discouraged as the addition of the MultiplePoseMover now means that a single job may include several separate trajectories.

Reimplemented in protocols::jd2::MSDJobDistributor.

References current_job_.

Referenced by protocols::jd2::add_current_job_data_to_ss(), protocols::jd2::add_string_real_pair_to_current_job(), protocols::jd2::add_string_string_pair_to_current_job(), protocols::jd2::add_string_to_current_job(), protocols::jd2::add_strings_to_current_job(), protocols::chemically_conjugated_docking::UBQ_GTPaseMover::analyze_and_filter(), protocols::comparative_modeling::LoopRelaxThreadingMover::apply(), protocols::hybridization::MRMover::apply(), protocols::rosetta_scripts::ParsedProtocol::apply(), protocols::canonical_sampling::CanonicalSamplingMover::apply(), protocols::jd2::current_input_tag(), protocols::jd2::current_nstruct_index(), protocols::jd2::current_output_filename(), protocols::jd2::current_output_name(), current_output_name(), protocols::jd2::get_current_job(), protocols::jd2::get_current_jobs_starting_pose(), protocols::jd2::get_string_real_pairs_from_current_job(), protocols::jd2::get_string_string_pairs_from_current_job(), protocols::jd2::get_strings_from_current_job(), protocols::chemically_conjugated_docking::UBQ_GTPaseMover::initialize(), protocols::canonical_sampling::BiasEnergy::initialize_simulation(), protocols::jd2::jd2_used(), protocols::jd2::BOINCJobDistributor::job_failed(), protocols::jd2::max_nstruct_index(), protocols::topology_broker::RigidChunkClaimer::new_decoy(), protocols::jd2::output_intermediate_pose(), protocols::jd2::MPIFileBufJobDistributor::process_message(), protocols::jd2::MPIWorkPartitionJobDistributor::remove_bad_inputs_from_job_list(), protocols::features::RuntimeFeatures::report_features(), protocols::jd2::MPIFileBufJobDistributor::slave_job_succeeded(), protocols::jd2::MPIWorkPoolJobDistributor::slave_job_succeeded(), and protocols::canonical_sampling::BiasEnergy::~BiasEnergy().

void protocols::jd2::JobDistributor::current_job_finished ( )
protectedvirtual

Derived classes are allowed to clean up any temporary files or data relating to the current job after the current job has completed. Called inside go_main loop. Default implementation is a no-op.

Reimplemented in protocols::jd2::FileSystemJobDistributor.

Referenced by run_one_job().

core::Size protocols::jd2::JobDistributor::current_job_id ( ) const
std::string protocols::jd2::JobDistributor::current_output_name ( ) const
virtual

Movers may ask their controlling job distributor for the output name as defined by the Job and JobOutputter.

References current_job(), and job_outputter().

Referenced by protocols::simple_filters::MPSpanAngleFilter::compute(), and protocols::simple_filters::ResidueLipophilicityFilter::compute().

void protocols::jd2::JobDistributor::end_critical_section ( )
protected

Referenced by write_output_from_job().

std::string protocols::jd2::JobDistributor::get_current_batch ( ) const

what is the current batch ? — name refers to the flag-file used for this batch

BATCH interface:

the BATCH interface of the JobDistributor is used to allow consecutive execution of a set of jobs with different flags different batches behave like completely independent rosetta runs — but of course a number of processes can already work on a new batch, while others are still finishing the last jobs of the previous batch. run from command-line with -run:batches flag1 flag2 flag3 the flag1 flag2... point to files that are added to all other flags ( and removed at end of batch ) you can have all output in same output file or ( by redefining e.g. -out:file:silent in each batch-flag file ) in different output files

References batches_, protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID, and current_batch_id_.

Referenced by protocols::jd2::BatchJobInputter::check_batch(), protocols::jd2::current_batch(), next_batch(), obtain_new_job(), protocols::jd2::archive::MPIArchiveJobDistributor::process_message(), and protocols::jd2::MPIFileBufJobDistributor::slave_get_new_job_id().

JobDistributor * protocols::jd2::JobDistributor::get_instance ( )
static

static function to get the instance of ( pointer to) this singleton class

References create_singleton_instance(), and instance_.

Referenced by protocols::jd2::add_current_job_data_to_ss(), protocols::jd2::add_string_real_pair_to_current_job(), protocols::jd2::add_string_string_pair_to_current_job(), protocols::jd2::add_string_to_current_job(), protocols::jd2::add_strings_to_current_job(), protocols::hydrate::add_water_overcoordinated_hb_score(), protocols::jd2::JobOutputter::affixed_numbered_name(), protocols::chemically_conjugated_docking::UBQ_GTPaseMover::analyze_and_filter(), protocols::comparative_modeling::LoopRelaxThreadingMover::apply(), protocols::frag_picker::nonlocal::NonlocalFrags::apply(), protocols::hybridization::MRMover::apply(), protocols::make_rot_lib::MakeRotLibMover::apply(), protocols::rosetta_scripts::ParsedProtocol::apply(), protocols::canonical_sampling::CanonicalSamplingMover::apply(), protocols::ligand_docking::LigandDockMain::apply(), protocols::symmetric_docking::SymDockProtocol::apply(), protocols::abinitio::Broker_main(), protocols::make_rot_lib::MakeRotLibMover::calc_final_rotamers(), protocols::jd2::archive::ArchiveManager::cancel_batch(), protocols::canonical_sampling::canonical_sampling_main(), protocols::jd2::BatchJobInputter::check_batch(), protocols::comparative_modeling::cm_main(), protocols::simple_filters::MPSpanAngleFilter::compute(), protocols::simple_filters::ResidueLipophilicityFilter::compute(), protocols::frag_picker::FragmentScoreFilter::compute(), protocols::simple_filters::BuriedUnsatHbondFilter::compute(), protocols::jd2::current_batch(), protocols::jd2::current_batch_id(), protocols::jd2::current_input_tag(), protocols::jd2::current_nstruct_index(), protocols::jd2::current_output_filename(), protocols::jd2::current_output_name(), protocols::jd2::current_replica(), protocols::pose_creation::MergePDBMover::get_additional_output(), protocols::jd2::get_current_job(), protocols::jd2::get_current_jobs_starting_pose(), protocols::jd2::Job::get_pose(), protocols::jd2::get_string_real_pairs_from_current_job(), protocols::jd2::get_string_string_pairs_from_current_job(), protocols::jd2::get_strings_from_current_job(), protocols::chemically_conjugated_docking::UBQ_GTPaseMover::initialize(), protocols::canonical_sampling::BiasEnergy::initialize_simulation(), jd2_signal_handler(), protocols::jd2::jd2_used(), protocols::canonical_sampling::jobname_dirhash(), ligand_dock_main(), protocols::loop_build::LoopBuild_main(), protocols::loophash::loophash_main(), protocols::pose_creation::MakeJunctionsMover::make_pose_from_design(), protocols::jd2::max_nstruct_index(), protocols::make_rot_lib::MakeRotLibMover::minimize_rotamer(), protocols::topology_broker::RigidChunkClaimer::new_decoy(), protocols::jd2::output_intermediate_pose(), protocols::jd2::archive::ArchiveManager::queue_batch(), protocols::RBSegmentRelax_main(), protocols::relax::Relax_main(), protocols::features::RuntimeFeatures::report_features(), protocols::canonical_sampling::mc_convergence_checks::setup_convergence_checks_from_cmdline(), protocols::symmetric_docking::SymDock_main(), protocols::features::TopPercentOfAllInputs::TopPercentOfAllInputs(), protocols::hydrate::water_specific_hbond_energy(), protocols::jd2::write_score_tracer(), and protocols::canonical_sampling::BiasEnergy::~BiasEnergy().

void protocols::jd2::JobDistributor::get_job_list_from_job_inputter ( )
private
core::Size protocols::jd2::JobDistributor::get_job_time_estimate ( ) const
protected

Get an estimate of the time to run an additional job. If it can't be estimated, return a time of zero.

Right now this is just the average runtime for the completed jobs.

References jobs_, and protocols::jd2::tr().

Referenced by run_one_job().

JobsContainer const & protocols::jd2::JobDistributor::get_jobs ( ) const
protected
JobsContainer & protocols::jd2::JobDistributor::get_jobs_nonconst ( )
protected
virtual core::Size protocols::jd2::JobDistributor::get_new_job_id ( )
protectedpure virtual
void protocols::jd2::JobDistributor::go ( protocols::moves::MoverOP  mover)
virtual
void protocols::jd2::JobDistributor::go ( protocols::moves::MoverOP  mover,
JobOutputterOP  jo 
)

invokes go, after setting JobOutputter

References go(), and job_outputter_.

void protocols::jd2::JobDistributor::go ( protocols::moves::MoverOP  mover,
JobInputterOP  ji 
)

invokes go, after setting JobInputter

References go(), and set_job_inputter().

void protocols::jd2::JobDistributor::go ( protocols::moves::MoverOP  mover,
JobInputterOP  ji,
JobOutputterOP  jo 
)

invokes go, after setting JobInputter and JobOutputter

References go(), job_outputter_, and set_job_inputter().

void protocols::jd2::JobDistributor::go_main ( protocols::moves::MoverOP  mover)
protected

Non-virtual get-job, run it, & output loop. This function is pretty generic and your subclass may be able to use it. It is NOT virtual - this implementation can be shared by (at least) the simple FileSystemJobDistributor, the MPIWorkPoolJobDistributor, and the MPIWorkPartitionJobDistributor. Do not feel that you need to use it as-is in your class - but DO plan on implementing all its functionality!

References batches_, job_outputter_, jobs_, note_all_jobs_finished(), number_failed_jobs_, obtain_new_job(), run_one_job(), core::id::to_string(), and protocols::jd2::tr().

Referenced by protocols::unfolded_state_energy_calculator::UnfoldedStateEnergyCalculatorJobDistributor::go(), protocols::jd2::MPIFileBufJobDistributor::go(), protocols::jd2::MPIWorkPartitionJobDistributor::go(), go(), protocols::jd2::archive::MPIArchiveJobDistributor::go(), and protocols::jd2::MPIWorkPoolJobDistributor::slave_go().

virtual void protocols::jd2::JobDistributor::handle_interrupt ( )
protectedpure virtual

This function got called when job is not yet finished and got termitated abnormaly (ctrl-c, kill etc). when implimenting it in subclasses make sure to delete all in-progress-data that your job spawns.

Implemented in protocols::jd2::archive::MPIArchiveJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, protocols::jd2::MSDJobDistributor, and protocols::jd2::ShuffleFileSystemJobDistributor.

Referenced by jd2_signal_handler().

bool protocols::jd2::JobDistributor::has_been_instantiated ( )
static

Has the job distributor been instantiated?

Returns true for yes and false for no.

Author
Vikram K. Mulligan (vmull.nosp@m.ig@u.nosp@m.w.edu).

References instance_.

Referenced by protocols::jd2::jd2_used().

void protocols::jd2::JobDistributor::increment_failed_jobs ( )
inlineprotected

Increment the number of failed jobs.

References number_failed_jobs_.

Referenced by protocols::jd2::MPIWorkPoolJobDistributor::job_failed(), and job_failed().

void protocols::jd2::JobDistributor::init_jd ( )
private
void protocols::jd2::JobDistributor::jd2_signal_handler ( int  signal_)
staticprotected

Default callback function for signal handling.

Default callback function for signal handling

References get_instance(), handle_interrupt(), and job_outputter_.

void protocols::jd2::JobDistributor::job_failed ( core::pose::Pose ,
bool  will_retry 
)
protectedvirtual

This function is called when we give up on the job; it has been virtualized so BOINC and MPI can delay/protect output.

Mark job as bad, so at the end of execution we know definitively how many jobs failed for any reason This function (in this classes implementation) increments the number_failed_jobs_ class variable. If you write a child JobDistributor and do not want an exception to be thrown at the end of execution if some jobs failed, be sure to override this function so that it does not increment number_failed_jobs_.

Mark job as bad, so at the end of execution we know definitively how many jobs failed for any reason

Reimplemented in protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, protocols::jd2::MSDJobDistributor, and protocols::jd2::BOINCJobDistributor.

References current_job_id_, and increment_failed_jobs().

Referenced by protocols::jd2::FileSystemJobDistributor::job_failed(), run_one_job(), and write_output_from_job().

JobInputterOP protocols::jd2::JobDistributor::job_inputter ( ) const
JobInputterInputSource::Enum protocols::jd2::JobDistributor::job_inputter_input_source ( ) const

The input source for the current JobInputter.

References job_inputter_.

JobOutputterOP protocols::jd2::JobDistributor::job_outputter ( ) const
void protocols::jd2::JobDistributor::job_succeeded ( core::pose::Pose pose,
core::Real  run_time,
std::string const &  tag 
)
protectedvirtual

This function is called upon a successful job completion; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter.

Reimplemented in protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, and protocols::jd2::BOINCJobDistributor.

References current_job_, current_job_id_, job_outputter_, and mark_job_as_completed().

Referenced by protocols::jd2::FileSystemJobDistributor::job_succeeded(), and write_output_from_job().

void protocols::jd2::JobDistributor::job_succeeded_additional_output ( core::pose::Pose pose,
std::string const &  tag 
)
protectedvirtual

This function is called upon a successful job completion if there are additional poses generated by the mover base implementation is just a call to the job outputter.

References current_job_, and job_outputter_.

Referenced by write_output_from_job().

void protocols::jd2::JobDistributor::load_new_batch ( )
protectedvirtual
virtual void protocols::jd2::JobDistributor::mark_current_job_id_for_repetition ( )
protectedpure virtual

this function is called whenever a job "soft-fails" and needs to be retried. Generally it should ensure that the subsequent call to obtain_new_job returns this job over again.

Implemented in protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::ShuffleFileSystemJobDistributor, protocols::jd2::BOINCJobDistributor, and protocols::jd2::MSDJobDistributor.

Referenced by write_output_from_job().

void protocols::jd2::JobDistributor::mark_job_as_bad ( core::Size  job_id)
protected
void protocols::jd2::JobDistributor::mark_job_as_completed ( core::Size  job_id,
core::Real  run_time 
)
protected

Jobs is the container of Job objects need non-const to mark Jobs as completed on Master in MPI-JobDistributor.

References job_outputter_, jobs_, and protocols::jd2::tr().

Referenced by job_succeeded(), and protocols::jd2::MPIFileBufJobDistributor::mark_job_as_completed().

void protocols::jd2::JobDistributor::mpi_finalize ( bool  finalize)
virtual

should the go() function call MPI_finalize()? It probably should, this is true by default.

Reimplemented in protocols::jd2::MPIWorkPoolJobDistributor.

bool protocols::jd2::JobDistributor::next_batch ( )
protectedvirtual
void protocols::jd2::JobDistributor::note_all_jobs_finished ( )
protectedvirtual

Derived classes are allowed to perform some kind of action when the job distributor runs out of jobs to execute. Called inside go_main. Default implementation is a no-op.

Referenced by go_main().

core::Size protocols::jd2::JobDistributor::nr_batches ( ) const
inlineprotected

how many batches are in our list ... this can change dynamically

References batches_.

Referenced by protocols::jd2::archive::MPIArchiveJobDistributor::notify_archive(), and protocols::jd2::archive::MPIArchiveJobDistributor::sync_batches().

bool protocols::jd2::JobDistributor::obtain_new_job ( bool  re_consider_current_job = false)
protected
rosetta_scripts::RosettaScriptsParserOP protocols::jd2::JobDistributor::parser ( ) const
protected

Parser access.

References parser_.

Referenced by protocols::jd2::MSDJobDistributor::go().

void protocols::jd2::JobDistributor::populate_batch_list_from_cmd ( )
private

read -run:batches

read -run:batches and put it into batches_ vector.

References batches_.

Referenced by init_jd().

void protocols::jd2::JobDistributor::remove_bad_inputs_from_job_list ( )
protectedvirtual

this function is called inside go_main if a mover returns FAIL_BAD_INPUT. Its purpose is to remove other jobs with the same input (which will still be bad) from the available list of jobs. The default implementation is a no-op and not all distributors are expected/able to implement this functionality, only those that can guaruntee no other jobs of that input are currently running.

Reimplemented in protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, and protocols::jd2::FileSystemJobDistributor.

Referenced by run_one_job(), and write_output_from_job().

void protocols::jd2::JobDistributor::remove_system_signal_handler ( )
staticprotected

Set signal handler back to default state.

Set signal handler back to default state.

void protocols::jd2::JobDistributor::reset_job_state ( )
private
void protocols::jd2::JobDistributor::restart ( )
virtual

restart job-distribution from beginning – useful if you need a second pass over decoys...

Reimplemented in protocols::jd2::FileSystemJobDistributor.

References init_jd(), and reset_job_state().

Referenced by protocols::jd2::FileSystemJobDistributor::restart().

bool protocols::jd2::JobDistributor::run_one_job ( protocols::moves::MoverOP mover,
time_t  allstarttime,
std::string &  last_inner_job_tag,
std::string &  last_output_tag,
core::Size last_batch_id,
core::Size retries_this_job,
bool  first_job 
)
protected
void protocols::jd2::JobDistributor::set_batch_id ( core::Size  setting)
protected

set current_batch_id — eg for slave nodes in MPI framework

References batch_underflow(), batches_, current_batch_id_, load_new_batch(), and protocols::jd2::tr().

Referenced by protocols::jd2::MPIFileBufJobDistributor::slave_get_new_job_id().

void protocols::jd2::JobDistributor::set_current_job_by_index ( core::Size  curr_job_index)
protected

For derived classes that wish to invoke JobDistributor functions which use the current_job_ and current_job_id_ member variables. Note that until those functions complete, it would be a bad idea for another thread to change current_job_.

References current_job_, and current_job_id_.

void protocols::jd2::JobDistributor::set_job_inputter ( JobInputterOP  new_job_inputter)

Set the JobInputter and reset the Job list – this is not something you want to do after go() has been called, but before it has returned.

References get_job_list_from_job_inputter(), job_inputter_, and reset_job_state().

Referenced by go().

void protocols::jd2::JobDistributor::set_job_outputter ( const JobOutputterOP new_job_outputter)

Movers (or derived classes) may ask for the JobOutputter.

References job_outputter_.

void protocols::jd2::JobDistributor::setup_pymol_observer ( core::pose::Pose pose)
protected

After the construction of the pose for this job, check the command line to determine if the pymol observer should be attached to it.

References protocols::moves::AddPyMOLObserver(), protocols::moves::AddPyMOLObserver_to_conformation(), protocols::moves::AddPyMOLObserver_to_energies(), and protocols::jd2::tr().

Referenced by run_one_job().

void protocols::jd2::JobDistributor::setup_system_signal_handler ( void(*)(int)  signal_fn = jd2_signal_handler)
staticprotected

Setting up callback function that will be call when our process is about to terminate. This will allow us to exit propely (clean up in_progress_files/tmp files if any).

Setting up callback function that will be call when our process is about to terminate.

core::Size protocols::jd2::JobDistributor::total_nr_jobs ( ) const
inline
bool protocols::jd2::JobDistributor::using_parser ( ) const
protected

Is the parser in use?

Checks parser::protocol flag since this is JD2.

References parser_.

Referenced by run_one_job().

void protocols::jd2::JobDistributor::write_output_from_job ( core::pose::Pose pose,
protocols::moves::MoverOP  mover_copy,
protocols::moves::MoverStatus  status,
core::Size  jobtime,
core::Size retries_this_job 
)
protected

After a job has finished running, figure out from the MoverStatus whether the pose should be written to disk (or wherever) along with any other poses that the mover might have generated along the way.

References begin_critical_section(), current_job_, current_job_id_, end_critical_section(), protocols::moves::FAIL_BAD_INPUT, protocols::moves::FAIL_DO_NOT_RETRY, protocols::moves::FAIL_RETRY, job_failed(), job_outputter_, job_succeeded(), job_succeeded_additional_output(), last_completed_job_, mark_current_job_id_for_repetition(), protocols::moves::MS_SUCCESS, remove_bad_inputs_from_job_list(), and protocols::jd2::tr().

Referenced by run_one_job().

Friends And Related Function Documentation

friend class JobDistributorFactory
friend

Member Data Documentation

utility::vector1< std::string > protocols::jd2::JobDistributor::batches_
private
core::Size protocols::jd2::JobDistributor::current_batch_id_
private
JobOP protocols::jd2::JobDistributor::current_job_
private
core::Size protocols::jd2::JobDistributor::current_job_id_
private

access into jobs_ object indicating current job. Contains more information than current_job_ in that it can be incremented...

Referenced by clear_current_job_output(), current_job_id(), job_failed(), job_succeeded(), load_new_batch(), obtain_new_job(), reset_job_state(), set_current_job_by_index(), and write_output_from_job().

JobDistributor * protocols::jd2::JobDistributor::instance_
staticprivate
JobInputterOP protocols::jd2::JobDistributor::job_inputter_
private
JobOutputterOP protocols::jd2::JobDistributor::job_outputter_
private
JobsContainerOP protocols::jd2::JobDistributor::jobs_
private

Container for the array of owning pointers to the Job objects.

By making this a class, it makes it easier to permit easy updating of the jobs list (e.g. if it's too long to hold the whole thing in memory).

Referenced by get_job_list_from_job_inputter(), get_job_time_estimate(), get_jobs(), get_jobs_nonconst(), go_main(), JobDistributor(), load_new_batch(), mark_job_as_completed(), reset_job_state(), run_one_job(), and total_nr_jobs().

core::Size protocols::jd2::JobDistributor::last_completed_job_
private

access into jobs_ object indicating the previous job. Used with the -jd2:delete_old_poses option for deleting unnecessary poses

Referenced by reset_job_state(), run_one_job(), and write_output_from_job().

core::Size protocols::jd2::JobDistributor::number_failed_jobs_
private

Number of failed jobs - kept track of so we can exit properly at the end of execution.

Referenced by go_main(), and increment_failed_jobs().

RosettaScriptsParserOP protocols::jd2::JobDistributor::parser_
private

The documentation for this class was generated from the following files: