Rosetta  2021.16
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Public Member Functions | Protected Types | Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes | List of all members
protocols::jobdist::BaseJobDistributor Class Referenceabstract

Coordinates processing of jobs across multiple Rosetta processes. More...

#include <JobDistributors.hh>

Inheritance diagram for protocols::jobdist::BaseJobDistributor:
Inheritance graph
[legend]

Public Member Functions

 BaseJobDistributor (JobVector const &jobs)
 
 BaseJobDistributor (BaseJobDistributor const &)
 
 ~BaseJobDistributor () override
 
bool next_job (BasicJobOP &job, int &struct_n)
 If true, sets the next Job and nstruct number to be processed. Deliberately not virtual: should not be overriden. Uses the "find_available_job" method, which is common to both MPI and standard protocols, but used in slightly different manners. More...
 
virtual void startup ()
 Must be called by client before first call to next_job(). If overriden by a subclass, it MUST call the superclass implementation. More...
 
virtual void shutdown ()
 Must be called by client after last call to next_job(). If overriden by a subclass, it MUST call the superclass implementation. More...
 
virtual void begin_critical_section ()
 Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation. More...
 
virtual void end_critical_section ()
 Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation. More...
 
virtual void dump_pose_and_map (std::string const &, core::pose::Pose &)
 Virtual function for dump_pose that is needed for main_plain_mover. More...
 
virtual void temp_file (std::string const &)
 Virtual function for temp_file main_plain_mover. More...
 
virtual void score_map (std::map< std::string, core::Real > &)
 Virtual function for score_map that is needed for main_plain_mover sets the score_map. More...
 
void disable_ignorefinished ()
 
void enable_ignorefinished ()
 
void disable_output ()
 
void enable_output ()
 
void disable_inprogress ()
 
void enable_inprogress ()
 
bool ignorefinished () const
 
bool nooutput () const
 
bool inprogress () const
 
void set_proc_id (core::Size proc_id, core::Size nproc)
 
virtual std::string get_current_output_tag ()
 get output_tag for current job's current nstruct More...
 
virtual std::string get_output_filename ()
 

Protected Types

typedef utility::vector1
< BasicJobOP
JobVector
 

Protected Member Functions

virtual bool is_finished (BasicJobOP const &job, int struct_n)=0
 Is the given nstruct number of the given input job already finished by the local process? To be implemented by subclasses. More...
 
virtual void checkpoint_read ()
 Restore state from checkpoint file, if it exists. More...
 
virtual void checkpoint_write ()
 Save state to checkpoint file, overwriting previous. More...
 
virtual void checkpoint_clear ()
 Remove checkpoint file (at end of batch) More...
 
int current_nstruct ()
 accessor for current_nstruct_ More...
 
BasicJobOP current_job ()
 accessor for current_job owning pointer More...
 
int get_next_random_range (int low, int high)
 

Protected Attributes

bool ignorefinished_
 ignore already done jobs - redo everything ! More...
 
bool nooutput_
 do not write files - useful such things as statistics or rescoring ! More...
 
bool inprogress_
 write .in_progress files for multiple processor runs More...
 
int start_time_
 starttime More...
 
int random_counter_
 RandomStore - needed for. More...
 
utility::vector1< core::Realrandom_store_
 

Private Member Functions

bool find_available_job ()
 looks for a job that has not yet been started, and stores the index for the job, and the nstruct index in the member variables. Called by next_job() and by master_node_distribute_jobs(). More...
 

Private Attributes

bool const overwrite_
 
JobVector jobs_
 
core::Size current_job_
 
int current_nstruct_
 
bool is_started_
 
core::Size nproc_
 
core::Size proc_id_
 
core::Size curr_jobid_
 

Detailed Description

Coordinates processing of jobs across multiple Rosetta processes.

Job distributors need to be customized in three different ways:

Cluster architecture is a GLOBAL COMPILE-TIME decision: it's handled by scons, it's the same for all executables built at that time, and it should be implemented using ifdef's in this base class, by modifying the next_job() method directly.

Test for job completion is a PER-EXECUTABLE decision: it's handled by subclassing BaseJobDistributor and implementing the is_finished() method. BaseJobDistributor will consult is_finished() in whatever way is appropriate for the current cluster architecture to ensure that jobs are not repeated.

Type of input is handled by templating the job distributor on a Job object. BasicJob has been provided already, but you can subclass it if you need to carry around additional information about the inputs.

Member Typedef Documentation

typedef utility::vector1< BasicJobOP > protocols::jobdist::BaseJobDistributor::JobVector
protected

Constructor & Destructor Documentation

protocols::jobdist::BaseJobDistributor::BaseJobDistributor ( JobVector const &  jobs)

References start_time_.

protocols::jobdist::BaseJobDistributor::BaseJobDistributor ( BaseJobDistributor const &  src)
protocols::jobdist::BaseJobDistributor::~BaseJobDistributor ( )
override

References is_started_.

Member Function Documentation

void protocols::jobdist::BaseJobDistributor::begin_critical_section ( )
virtual

Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation.

If overriden by a subclass, it MUST call the superclass implementation.

Referenced by checkpoint_read(), checkpoint_write(), protocols::jobdist::AtomTreeDiffJobDistributor::dump_pose(), protocols::jobdist::PlainSilentFileJobDistributor::dump_pose(), protocols::jobdist::PlainPdbJobDistributor::dump_pose_and_map(), protocols::jobdist::PlainRawJobDistributor::dump_pose_and_map(), and protocols::jobdist::PlainSilentFileJobDistributor::dump_silent().

void protocols::jobdist::BaseJobDistributor::checkpoint_clear ( )
protectedvirtual

Remove checkpoint file (at end of batch)

Referenced by shutdown().

void protocols::jobdist::BaseJobDistributor::checkpoint_read ( )
protectedvirtual

Restore state from checkpoint file, if it exists.

References begin_critical_section(), end_critical_section(), and core::scoring::rg.

Referenced by startup().

void protocols::jobdist::BaseJobDistributor::checkpoint_write ( )
protectedvirtual

Save state to checkpoint file, overwriting previous.

Calls to this function are only a suggestion. If checkpointing is expensive, this function must track time between calls to avoid excessive disk activity, etc.

References begin_critical_section(), end_critical_section(), protocols::checkpoint::Timer::reset(), and core::scoring::rg.

Referenced by next_job().

BasicJobOP protocols::jobdist::BaseJobDistributor::current_job ( )
protected

accessor for current_job owning pointer

References current_job_, and jobs_.

Referenced by get_current_output_tag(), and protocols::jobdist::PlainSilentFileJobDistributor::get_current_output_tag().

int protocols::jobdist::BaseJobDistributor::current_nstruct ( )
inlineprotected
void protocols::jobdist::BaseJobDistributor::disable_ignorefinished ( )
inline

References ignorefinished_.

void protocols::jobdist::BaseJobDistributor::disable_inprogress ( )
inline

References inprogress_.

void protocols::jobdist::BaseJobDistributor::disable_output ( )
inline

References nooutput_.

void protocols::jobdist::BaseJobDistributor::dump_pose_and_map ( std::string const &  ,
core::pose::Pose  
)
virtual

Virtual function for dump_pose that is needed for main_plain_mover.

Needed if a score_map is to be output by the derived class

Reimplemented in protocols::jobdist::PlainRawJobDistributor, and protocols::jobdist::PlainPdbJobDistributor.

void protocols::jobdist::BaseJobDistributor::enable_ignorefinished ( )
inline

References ignorefinished_.

void protocols::jobdist::BaseJobDistributor::enable_inprogress ( )
inline
void protocols::jobdist::BaseJobDistributor::enable_output ( )
inline

References nooutput_.

void protocols::jobdist::BaseJobDistributor::end_critical_section ( )
virtual

Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation.

If overriden by a subclass, it MUST call the superclass implementation.

Referenced by checkpoint_read(), checkpoint_write(), protocols::jobdist::AtomTreeDiffJobDistributor::dump_pose(), protocols::jobdist::PlainSilentFileJobDistributor::dump_pose(), protocols::jobdist::PlainPdbJobDistributor::dump_pose_and_map(), protocols::jobdist::PlainRawJobDistributor::dump_pose_and_map(), and protocols::jobdist::PlainSilentFileJobDistributor::dump_silent().

bool protocols::jobdist::BaseJobDistributor::find_available_job ( )
private

looks for a job that has not yet been started, and stores the index for the job, and the nstruct index in the member variables. Called by next_job() and by master_node_distribute_jobs().

Deliberately not virtual: should not be overriden. Iterate over the jobs that exist, check that nothing has been started for each, and point the private data current_nstruct_ and current_job_ at whichever available job it finds first. Returns false if no job can be found.

References curr_jobid_, current_job_, current_nstruct_, get_next_random_range(), is_finished(), jobs_, nproc_, overwrite_, proc_id_, and protocols::boinc::Boinc::worker_is_finished().

Referenced by next_job().

std::string protocols::jobdist::BaseJobDistributor::get_current_output_tag ( )
virtual

get output_tag for current job's current nstruct

by default return current_job's current_nstruct' output_tag. Overridden in derived PlainSilentFileJobDistributor to return names with "S_" at the beginning

Reimplemented in protocols::jobdist::PlainSilentFileJobDistributor.

References current_job(), and current_nstruct().

int protocols::jobdist::BaseJobDistributor::get_next_random_range ( int  low,
int  high 
)
protected
std::string protocols::jobdist::BaseJobDistributor::get_output_filename ( )
virtual
bool protocols::jobdist::BaseJobDistributor::ignorefinished ( ) const
inline
bool protocols::jobdist::BaseJobDistributor::inprogress ( ) const
inline

References inprogress_.

virtual bool protocols::jobdist::BaseJobDistributor::is_finished ( BasicJobOP const &  job,
int  struct_n 
)
protectedpure virtual

Is the given nstruct number of the given input job already finished by the local process? To be implemented by subclasses.

Implemented in protocols::jobdist::PlainSilentFileJobDistributor, protocols::jobdist::PlainRawJobDistributor, protocols::jobdist::PlainPdbJobDistributor, and protocols::jobdist::AtomTreeDiffJobDistributor.

Referenced by find_available_job().

bool protocols::jobdist::BaseJobDistributor::next_job ( BasicJobOP job,
int &  struct_n 
)

If true, sets the next Job and nstruct number to be processed. Deliberately not virtual: should not be overriden. Uses the "find_available_job" method, which is common to both MPI and standard protocols, but used in slightly different manners.

Deliberately not virtual: should not be overriden. This is where to insert ifdefs and code for different cluster architectures!

References checkpoint_write(), current_job_, current_nstruct_, find_available_job(), get_output_filename(), is_started_, jobs_, protocols::wum::mpi_rank(), and start_time_.

Referenced by protocols::jobdist::main_atom_tree_diff_mover().

bool protocols::jobdist::BaseJobDistributor::nooutput ( ) const
inline
void protocols::jobdist::BaseJobDistributor::score_map ( std::map< std::string, core::Real > &  )
virtual

Virtual function for score_map that is needed for main_plain_mover sets the score_map.

Needed if a score_map is to be output by the derived class

Reimplemented in protocols::jobdist::PlainRawJobDistributor, and protocols::jobdist::PlainPdbJobDistributor.

void protocols::jobdist::BaseJobDistributor::set_proc_id ( core::Size  proc_id,
core::Size  nproc 
)
inline

References nproc_, and proc_id_.

void protocols::jobdist::BaseJobDistributor::shutdown ( )
virtual

Must be called by client after last call to next_job(). If overriden by a subclass, it MUST call the superclass implementation.

If overriden by a subclass, it MUST call the superclass implementation.

Reimplemented in protocols::jobdist::PlainSilentFileJobDistributor, and protocols::jobdist::AtomTreeDiffJobDistributor.

References checkpoint_clear(), curr_jobid_, current_nstruct_, is_started_, jobs_, protocols::boinc::Boinc::worker_finish_summary(), and protocols::boinc::Boinc::worker_shutdown().

Referenced by protocols::jobdist::AtomTreeDiffJobDistributor::shutdown(), and protocols::jobdist::PlainSilentFileJobDistributor::shutdown().

void protocols::jobdist::BaseJobDistributor::startup ( )
virtual

Must be called by client before first call to next_job(). If overriden by a subclass, it MUST call the superclass implementation.

If overriden by a subclass, it MUST call the superclass implementation.

Reimplemented in protocols::jobdist::PlainSilentFileJobDistributor, and protocols::jobdist::PlainPdbJobDistributor.

References checkpoint_read(), is_started_, and protocols::boinc::Boinc::worker_startup().

Referenced by protocols::jobdist::main_atom_tree_diff_mover(), protocols::jobdist::PlainPdbJobDistributor::startup(), and protocols::jobdist::PlainSilentFileJobDistributor::startup().

void protocols::jobdist::BaseJobDistributor::temp_file ( std::string const &  )
virtual

Virtual function for temp_file main_plain_mover.

Needed to output .in_progress files for multiple processor jobs

Reimplemented in protocols::jobdist::PlainPdbJobDistributor.

Member Data Documentation

core::Size protocols::jobdist::BaseJobDistributor::curr_jobid_
private

Referenced by find_available_job(), and shutdown().

core::Size protocols::jobdist::BaseJobDistributor::current_job_
private
int protocols::jobdist::BaseJobDistributor::current_nstruct_
private
bool protocols::jobdist::BaseJobDistributor::ignorefinished_
protected

ignore already done jobs - redo everything !

Referenced by disable_ignorefinished(), enable_ignorefinished(), and ignorefinished().

bool protocols::jobdist::BaseJobDistributor::inprogress_
protected

write .in_progress files for multiple processor runs

Referenced by disable_inprogress(), enable_inprogress(), and inprogress().

bool protocols::jobdist::BaseJobDistributor::is_started_
private
JobVector protocols::jobdist::BaseJobDistributor::jobs_
private
bool protocols::jobdist::BaseJobDistributor::nooutput_
protected

do not write files - useful such things as statistics or rescoring !

Referenced by disable_output(), enable_output(), and nooutput().

core::Size protocols::jobdist::BaseJobDistributor::nproc_
private

Referenced by find_available_job(), and set_proc_id().

bool const protocols::jobdist::BaseJobDistributor::overwrite_
private

Referenced by find_available_job().

core::Size protocols::jobdist::BaseJobDistributor::proc_id_
private

Referenced by find_available_job(), and set_proc_id().

int protocols::jobdist::BaseJobDistributor::random_counter_
protected

RandomStore - needed for.

Referenced by get_next_random_range().

utility::vector1< core::Real > protocols::jobdist::BaseJobDistributor::random_store_
protected

Referenced by get_next_random_range().

int protocols::jobdist::BaseJobDistributor::start_time_
protected

starttime

Referenced by BaseJobDistributor(), and next_job().


The documentation for this class was generated from the following files: