Coordinates processing of jobs across multiple Rosetta processes. More...

#include <JobDistributors.hh>

Public Member Functions
	BaseJobDistributor (JobVector jobs)

	BaseJobDistributor (BaseJobDistributor const &)

virtual	~BaseJobDistributor ()

bool	next_job (BasicJobOP &job, int &struct_n)
	If true, sets the next Job and nstruct number to be processed. Deliberately not virtual: should not be overriden. Uses the "find_available_job" method, which is common to both MPI and standard protocols, but used in slightly different manners. More...

virtual void	startup ()
	Must be called by client before first call to next_job(). If overriden by a subclass, it MUST call the superclass implementation. More...

virtual void	shutdown ()
	Must be called by client after last call to next_job(). If overriden by a subclass, it MUST call the superclass implementation. More...

virtual void	begin_critical_section ()
	Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation. More...

virtual void	end_critical_section ()
	Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation. More...

virtual void	dump_pose_and_map (std::string const &, core::pose::Pose &)
	Virtual function for dump_pose that is needed for main_plain_mover. More...

virtual void	temp_file (std::string const &)
	Virtual function for temp_file main_plain_mover. More...

virtual void	score_map (std::map< std::string, core::Real > &)
	Virtual function for score_map that is needed for main_plain_mover sets the score_map. More...

void	disable_ignorefinished ()

void	enable_ignorefinished ()

void	disable_output ()

void	enable_output ()

void	disable_inprogress ()

void	enable_inprogress ()

bool	ignorefinished () const

bool	nooutput () const

bool	inprogress () const

void	set_proc_id (core::Size proc_id, core::Size nproc)

virtual std::string	get_current_output_tag ()
	get output_tag for current job's current nstruct More...

virtual std::string	get_output_filename ()

Protected Types
typedef utility::vector1 < BasicJobOP >	JobVector

Protected Member Functions
virtual bool	is_finished (BasicJobOP const &job, int struct_n)=0
	Is the given nstruct number of the given input job already finished by the local process? To be implemented by subclasses. More...

virtual void	checkpoint_read ()
	Restore state from checkpoint file, if it exists. More...

virtual void	checkpoint_write ()
	Save state to checkpoint file, overwriting previous. More...

virtual void	checkpoint_clear ()
	Remove checkpoint file (at end of batch) More...

int	current_nstruct ()
	accessor for current_nstruct_ More...

BasicJobOP	current_job ()
	accessor for current_job owning pointer More...

int	get_next_random_range (int low, int high)

Protected Attributes
bool	ignorefinished_
	ignore already done jobs - redo everything ! More...

bool	nooutput_
	do not write files - useful such things as statistics or rescoring ! More...

bool	inprogress_
	write .in_progress files for multiple processor runs More...

int	start_time_
	starttime More...

int	random_counter_
	RandomStore - needed for. More...

utility::vector1< double >	random_store_

Private Member Functions
bool	find_available_job ()
	looks for a job that has not yet been started, and stores the index for the job, and the nstruct index in the member variables. Called by next_job() and by master_node_distribute_jobs(). More...

Private Attributes
bool const	overwrite_

JobVector	jobs_

core::Size	current_job_

int	current_nstruct_

bool	is_started_

Size	nproc_

Size	proc_id_

Size	curr_jobid_

Detailed Description

Coordinates processing of jobs across multiple Rosetta processes.

Job distributors need to be customized in three different ways:

by cluster architecture (none/Condor, MPI, BOINC, etc)
by local test for job completion (PDB exists, tag already in silent file, etc)
by type of input (single PDB file, pair of PDB files, list of tag IDs, etc)

Cluster architecture is a GLOBAL COMPILE-TIME decision: it's handled by scons, it's the same for all executables built at that time, and it should be implemented using ifdef's in this base class, by modifying the next_job() method directly.

Test for job completion is a PER-EXECUTABLE decision: it's handled by subclassing BaseJobDistributor and implementing the is_finished() method. BaseJobDistributor will consult is_finished() in whatever way is appropriate for the current cluster architecture to ensure that jobs are not repeated.

Type of input is handled by templating the job distributor on a Job object. BasicJob has been provided already, but you can subclass it if you need to carry around additional information about the inputs.

Member Typedef Documentation

typedef utility::vector1< BasicJobOP > protocols::jobdist::BaseJobDistributor::JobVector

protected

Constructor & Destructor Documentation

protocols::jobdist::BaseJobDistributor::BaseJobDistributor ( JobVector jobs )

References option, options, out, overwrite, and start_time_.

protocols::jobdist::BaseJobDistributor::BaseJobDistributor ( BaseJobDistributor const & src )

protocols::jobdist::BaseJobDistributor::~BaseJobDistributor ( )

virtual

References basic::Error(), and is_started_.

Member Function Documentation

void protocols::jobdist::BaseJobDistributor::begin_critical_section ( )

virtual

Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation.

If overriden by a subclass, it MUST call the superclass implementation.

Referenced by checkpoint_read(), checkpoint_write(), protocols::jobdist::AtomTreeDiffJobDistributor::dump_pose(), protocols::jobdist::PlainSilentFileJobDistributor::dump_pose(), protocols::jobdist::PlainPdbJobDistributor::dump_pose_and_map(), protocols::jobdist::PlainRawJobDistributor::dump_pose_and_map(), and protocols::jobdist::PlainSilentFileJobDistributor::dump_silent().

void protocols::jobdist::BaseJobDistributor::checkpoint_clear ( )

protectedvirtual

Remove checkpoint file (at end of batch)

References utility::file::file_delete(), and utility::file::file_exists().

Referenced by shutdown().

void protocols::jobdist::BaseJobDistributor::checkpoint_read ( )

protectedvirtual

Restore state from checkpoint file, if it exists.

References begin_critical_section(), utility::io::izstream::close(), end_critical_section(), utility::file::file_exists(), and numeric::random::RandomGenerator::restoreAllStates().

Referenced by startup().

void protocols::jobdist::BaseJobDistributor::checkpoint_write ( )

protectedvirtual

Save state to checkpoint file, overwriting previous.

Calls to this function are only a suggestion. If checkpointing is expensive, this function must track time between calls to avoid excessive disk activity, etc.

References begin_critical_section(), utility::io::ozstream::close(), end_critical_section(), protocols::checkpoint::Timer::reset(), and numeric::random::RandomGenerator::saveAllStates().

Referenced by next_job().

BasicJobOP protocols::jobdist::BaseJobDistributor::current_job ( )

protected

accessor for current_job owning pointer

References current_job_, and jobs_.

Referenced by get_current_output_tag(), and protocols::jobdist::PlainSilentFileJobDistributor::get_current_output_tag().

int protocols::jobdist::BaseJobDistributor::current_nstruct ( )

inlineprotected

accessor for current_nstruct_

References current_nstruct_.

Referenced by get_current_output_tag(), and protocols::jobdist::PlainSilentFileJobDistributor::get_current_output_tag().

void protocols::jobdist::BaseJobDistributor::disable_ignorefinished ( )

inline

References ignorefinished_.

void protocols::jobdist::BaseJobDistributor::disable_inprogress ( )

inline

References inprogress_.

void protocols::jobdist::BaseJobDistributor::disable_output ( )

inline

References nooutput_.

void protocols::jobdist::BaseJobDistributor::dump_pose_and_map	(	std::string const &	,
		core::pose::Pose &
	)

virtual

Virtual function for dump_pose that is needed for main_plain_mover.

Needed if a score_map is to be output by the derived class

Reimplemented in protocols::jobdist::PlainRawJobDistributor, and protocols::jobdist::PlainPdbJobDistributor.

void protocols::jobdist::BaseJobDistributor::enable_ignorefinished ( )

inline

References ignorefinished_.

void protocols::jobdist::BaseJobDistributor::enable_inprogress ( )

inline

References inprogress_.

Referenced by protocols::jobdist::PlainPdbJobDistributor::startup().

void protocols::jobdist::BaseJobDistributor::enable_output ( )

inline

References nooutput_.

void protocols::jobdist::BaseJobDistributor::end_critical_section ( )

virtual

Signal that if at all possible, we would like to not be killed while in the critical section. If overriden by a subclass, it MUST call the superclass implementation.

If overriden by a subclass, it MUST call the superclass implementation.

Referenced by checkpoint_read(), checkpoint_write(), protocols::jobdist::AtomTreeDiffJobDistributor::dump_pose(), protocols::jobdist::PlainSilentFileJobDistributor::dump_pose(), protocols::jobdist::PlainPdbJobDistributor::dump_pose_and_map(), protocols::jobdist::PlainRawJobDistributor::dump_pose_and_map(), and protocols::jobdist::PlainSilentFileJobDistributor::dump_silent().

bool protocols::jobdist::BaseJobDistributor::find_available_job ( )

private

looks for a job that has not yet been started, and stores the index for the job, and the nstruct index in the member variables. Called by next_job() and by master_node_distribute_jobs().

Deliberately not virtual: should not be overriden. Iterate over the jobs that exist, check that nothing has been started for each, and point the private data current_nstruct_ and current_job_ at whichever available job it finds first. Returns false if no job can be found.

References curr_jobid_, current_job_, current_nstruct_, get_next_random_range(), is_finished(), jobs_, numeric::mod(), nproc_, basic::options::OptionKeys::out::nstruct, nstruct, basic::options::option, overwrite_, proc_id_, basic::options::OptionKeys::run::shuffle, and protocols::boinc::Boinc::worker_is_finished().

Referenced by next_job().

std::string protocols::jobdist::BaseJobDistributor::get_current_output_tag ( )

virtual

get output_tag for current job's current nstruct

by default return current_job's current_nstruct' output_tag. Overridden in derived PlainSilentFileJobDistributor to return names with "S_" at the beginning

Reimplemented in protocols::jobdist::PlainSilentFileJobDistributor.

References current_job(), and current_nstruct().

int protocols::jobdist::BaseJobDistributor::get_next_random_range	(	int	low,
		int	high
	)

protected

References random_counter_, random_store_, protocols::sic_dock::range(), and numeric::random::uniform().

Referenced by find_available_job().

std::string protocols::jobdist::BaseJobDistributor::get_output_filename ( )

virtual

Reimplemented in protocols::jobdist::PlainSilentFileJobDistributor.

Referenced by protocols::jobdist::PlainPdbJobDistributor::dump_pose_and_map(), protocols::jobdist::PlainRawJobDistributor::dump_pose_and_map(), protocols::jobdist::PlainPdbJobDistributor::is_finished(), protocols::jobdist::PlainRawJobDistributor::is_finished(), next_job(), and protocols::jobdist::PlainPdbJobDistributor::temp_file().

bool protocols::jobdist::BaseJobDistributor::ignorefinished ( ) const

inline

References ignorefinished_.

Referenced by protocols::jobdist::PlainPdbJobDistributor::is_finished().

bool protocols::jobdist::BaseJobDistributor::inprogress ( ) const

inline

References inprogress_.

virtual bool protocols::jobdist::BaseJobDistributor::is_finished	(	BasicJobOP const &	job,
		int	struct_n
	)

protectedpure virtual

Is the given nstruct number of the given input job already finished by the local process? To be implemented by subclasses.

Implemented in protocols::jobdist::PlainSilentFileJobDistributor, protocols::jobdist::PlainRawJobDistributor, protocols::jobdist::PlainPdbJobDistributor, and protocols::jobdist::AtomTreeDiffJobDistributor.

Referenced by find_available_job().

bool protocols::jobdist::BaseJobDistributor::next_job	(	BasicJobOP &	job,
		int &	struct_n
	)

If true, sets the next Job and nstruct number to be processed. Deliberately not virtual: should not be overriden. Uses the "find_available_job" method, which is common to both MPI and standard protocols, but used in slightly different manners.

Deliberately not virtual: should not be overriden. This is where to insert ifdefs and code for different cluster architectures!

References checkpoint_write(), current_job_, current_nstruct_, basic::Error(), find_available_job(), get_output_filename(), is_started_, jobs_, basic::options::OptionKeys::run::maxruntime, mpi_rank(), basic::options::option, start_time_, and user.