#include <JobDistributor.hh>

Inheritance diagram for protocols::jd2::JobDistributor:

[legend]

Public Member Functions
virtual	~JobDistributor ()

virtual void	go (protocols::moves::MoverOP mover)
	This may be overridden by derived classes. Default implementation invokes go_main. More...

void	go (protocols::moves::MoverOP mover, JobOutputterOP jo)
	invokes go, after setting JobOutputter More...

void	go (protocols::moves::MoverOP mover, JobInputterOP ji)
	invokes go, after setting JobInputter More...

void	go (protocols::moves::MoverOP mover, JobInputterOP ji, JobOutputterOP jo)
	invokes go, after setting JobInputter and JobOutputter More...

virtual JobOP	current_job () const
	Movers may ask their controlling job distributor for information about the current job. They may also write information to this job for later output, though this use is now discouraged as the addition of the MultiplePoseMover now means that a single job may include several separate trajectories. More...

virtual std::string	current_output_name () const
	Movers may ask their controlling job distributor for the output name as defined by the Job and JobOutputter. More...

JobOutputterOP	job_outputter () const
	Movers (or derived classes) may ask for the JobOutputter. More...

void	set_job_outputter (const JobOutputterOP &new_job_outputter)
	Movers (or derived classes) may ask for the JobOutputter. More...

JobInputterOP	job_inputter () const
	JobInputter access. More...

void	set_job_inputter (JobInputterOP new_job_inputter)
	Set the JobInputter and reset the Job list – this is not something you want to do after go() has been called, but before it has returned. More...

virtual void	mpi_finalize (bool finalize)
	should the go() function call MPI_finalize()? It probably should, this is true by default. More...

JobInputterInputSource::Enum	job_inputter_input_source () const
	The input source for the current JobInputter. More...

virtual void	restart ()

core::Size	total_nr_jobs () const

core::Size	current_job_id () const
	integer access - which job are we on? More...

std::string	get_current_batch () const
	what is the current batch ? — name refers to the flag-file used for this batch More...

virtual void	add_batch (std::string const &, core::Size id=0)
	add a new batch ( name will be interpreted as flag_file ) More...

core::Size	current_batch_id () const
	what is the current batch number ? — refers to position in batches_ More...

Static Public Member Functions
static bool	has_been_instantiated ()
	Has the job distributor been instantiated? More...

static JobDistributor *	get_instance ()
	static function to get the instance of ( pointer to) this singleton class More...

Protected Member Functions
	JobDistributor ()
	Singleton instantiation pattern; Derived classes will call default ctor, but their ctors, too must be protected (and the JDFactory must be their friend.) More...

	JobDistributor (bool empty)
	MPIArchiveJobDistributor starts with an empty job-list... More...

void	go_main (protocols::moves::MoverOP mover)
	Non-virtual get-job, run it, & output loop. This function is pretty generic and your subclass may be able to use it. It is NOT virtual - this implementation can be shared by (at least) the simple FileSystemJobDistributor, the MPIWorkPoolJobDistributor, and the MPIWorkPartitionJobDistributor. Do not feel that you need to use it as-is in your class - but DO plan on implementing all its functionality! More...

JobsContainer const &	get_jobs () const
	Read access to private data for derived classes. More...

JobsContainer &	get_jobs_nonconst ()
	Jobs is the container of Job objects. More...

void	mark_job_as_completed (core::Size job_id, core::Real run_time)
	Jobs is the container of Job objects need non-const to mark Jobs as completed on Master in MPI-JobDistributor. More...

void	mark_job_as_bad (core::Size job_id)

RosettaScriptsParserOP	parser () const
	Parser access. More...

void	begin_critical_section ()

void	end_critical_section ()

void	set_current_job_by_index (core::Size curr_job_index)
	For derived classes that wish to invoke JobDistributor functions which use the current_job_ and current_job_id_ member variables. Note that until those functions complete, it would be a bad idea for another thread to change current_job_. More...

bool	obtain_new_job (bool re_consider_current_job=false)
	this function updates the current_job_id_ and current_job_ fields. The boolean return states whether or not a new job was obtained (if false, quit distributing!) More...

virtual core::Size	get_new_job_id ()=0
	Return 0 to signal that no available jobs remain. Otherwise return an index into the Jobs object. More...

virtual void	job_succeeded (core::pose::Pose &pose, core::Real run_time, std::string const &tag)
	This function is called upon a successful job completion; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter. More...

virtual void	job_succeeded_additional_output (core::pose::Pose &pose, std::string const &tag)
	This function is called upon a successful job completion if there are additional poses generated by the mover base implementation is just a call to the job outputter. More...

virtual void	job_failed (core::pose::Pose &, bool will_retry)
	This function is called when we give up on the job; it has been virtualized so BOINC and MPI can delay/protect output. More...

virtual void	mark_current_job_id_for_repetition ()=0
	this function is called whenever a job "soft-fails" and needs to be retried. Generally it should ensure that the subsequent call to obtain_new_job returns this job over again. More...

virtual void	remove_bad_inputs_from_job_list ()
	this function is called inside go_main if a mover returns FAIL_BAD_INPUT. Its purpose is to remove other jobs with the same input (which will still be bad) from the available list of jobs. The default implementation is a no-op and not all distributors are expected/able to implement this functionality, only those that can guaruntee no other jobs of that input are currently running. More...

virtual void	current_job_finished ()
	Derived classes are allowed to clean up any temporary files or data relating to the current job after the current job has completed. Called inside go_main loop. Default implementation is a no-op. More...

virtual void	note_all_jobs_finished ()
	Derived classes are allowed to perform some kind of action when the job distributor runs out of jobs to execute. Called inside go_main. Default implementation is a no-op. More...

void	clear_current_job_output ()

virtual void	handle_interrupt ()=0
	This function got called when job is not yet finished and got termitated abnormaly (ctrl-c, kill etc). when implimenting it in subclasses make sure to delete all in-progress-data that your job spawns. More...

void	check_for_parser_in_go_main ()
	Send a message to the screen indicating that the parser is in use and that the mover that's been input to go_main will not be used, but instead will be replaced by the Mover created by the parser. More...

bool	using_parser () const
	Is the parser in use? More...

bool	run_one_job (protocols::moves::MoverOP &mover, time_t allstarttime, std::string &last_inner_job_tag, std::string &last_output_tag, core::Size &last_batch_id, core::Size &retries_this_job, bool first_job)

void	setup_pymol_observer (core::pose::Pose &pose)
	After the construction of the pose for this job, check the command line to determine if the pymol observer should be attached to it. More...

void	write_output_from_job (core::pose::Pose &pose, protocols::moves::MoverOP mover_copy, protocols::moves::MoverStatus status, core::Size jobtime, core::Size &retries_this_job)
	After a job has finished running, figure out from the MoverStatus whether the pose should be written to disk (or wherever) along with any other poses that the mover might have generated along the way. More...

void	increment_failed_jobs ()
	Increment the number of failed jobs. More...

core::Size	get_job_time_estimate () const
	Get an estimate of the time to run an additional job. If it can't be estimated, return a time of zero. More...

void	write_citations_and_clear_citation_tracking () const
	Write out information about all modules that have been used that should be cited, then clear the list of citations from the CitationManager. More...

void	set_batch_id (core::Size setting)
	set current_batch_id — eg for slave nodes in MPI framework More...

virtual bool	next_batch ()
	switch current_batch_id_ to next batch More...

virtual void	batch_underflow ()
	if end of batches_ reached via next_batch or set_batch_id ... More...

virtual void	load_new_batch ()
	called by next_batch() or set_batch_id() to switch-over and restart JobDistributor on new batch More...

core::Size	nr_batches () const
	how many batches are in our list ... this can change dynamically More...

std::string const &	batch (core::Size batch_id)
	give name of batch with given id More...

Static Protected Member Functions
static void	setup_system_signal_handler (void(*prev_fn)(int)=jd2_signal_handler)
	Setting up callback function that will be call when our process is about to terminate. This will allow us to exit propely (clean up in_progress_files/tmp files if any). More...

static void	remove_system_signal_handler ()
	Set signal handler back to default state. More...

static void	jd2_signal_handler (int Signal)
	Default callback function for signal handling. More...

Private Types
typedef rosetta_scripts::RosettaScriptsParserOP	RosettaScriptsParserOP

Private Member Functions
void	init_jd ()

void	populate_batch_list_from_cmd ()
	read -run:batches More...

void	reset_job_state ()

void	get_job_list_from_job_inputter ()

Static Private Member Functions
static JobDistributor *	create_singleton_instance ()
	private singleton creation function to be used with utility::thread::threadsafe_singleton More...

Private Attributes
JobInputterOP	job_inputter_

JobOutputterOP	job_outputter_

RosettaScriptsParserOP	parser_

JobsContainerOP	jobs_
	Container for the array of owning pointers to the Job objects. More...

JobOP	current_job_
	pointer to current job. Information is somewhat duplicated with current_job_id_. More...

core::Size	current_job_id_
	access into jobs_ object indicating current job. Contains more information than current_job_ in that it can be incremented... More...

core::Size	last_completed_job_
	access into jobs_ object indicating the previous job. Used with the -jd2:delete_old_poses option for deleting unnecessary poses More...

core::Size	number_failed_jobs_
	Number of failed jobs - kept track of so we can exit properly at the end of execution. More...

core::Size	current_batch_id_
	current_batch or 0 More...

utility::vector1< std::string >	batches_
	all batches if present More...

Static Private Attributes
static JobDistributor *	instance_

Friends
class	JobDistributorFactory

Detailed Description

Enforced JobDistributor destruction turned out to cause problems - calls to Tracers and/or the Options system during destruction caused crashes if those systems had been destructed first. So this is deprecated.

Member Typedef Documentation

◆ RosettaScriptsParserOP

typedef rosetta_scripts::RosettaScriptsParserOP protocols::jd2::JobDistributor::RosettaScriptsParserOP

private

Constructor & Destructor Documentation

◆ JobDistributor() [1/2]

protocols::jd2::JobDistributor::JobDistributor ( )

protected

Singleton instantiation pattern; Derived classes will call default ctor, but their ctors, too must be protected (and the JDFactory must be their friend.)

References init_jd().

◆ JobDistributor() [2/2]

protocols::jd2::JobDistributor::JobDistributor ( bool empty )

protected

MPIArchiveJobDistributor starts with an empty job-list...

References protocols::jd2::JobDistributorFactory::create_parser(), init_jd(), job_inputter_, job_outputter_, jobs_, and parser_.

◆ ~JobDistributor()

protocols::jd2::JobDistributor::~JobDistributor ( )

virtualdefault

WARNING WARNING! SINGLETONS' DESTRUCTORS ARE NEVER CALLED IN MINI! DO NOT TRY TO PUT THINGS IN THIS FUNCTION! here's a nice link explaining why: http://www.research.ibm.com/designpatterns/pubs/ph-jun96.txt

Member Function Documentation

◆ add_batch()

void protocols::jd2::JobDistributor::add_batch	(	std::string const &	batch,
		core::Size	id = `0`
	)

virtual

add a new batch ( name will be interpreted as flag_file )

@detail add new batch to end of batch list... this might be called asynchronous... ie. while we are still in the middle of a current batch, or while we are in non-batch mode

References batch(), batches_, and protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID.

Referenced by protocols::jd2::archive::ArchiveManager::cancel_batch(), protocols::jd2::archive::ArchiveManager::queue_batch(), and protocols::jd2::archive::MPIArchiveJobDistributor::receive_batch().

◆ batch()

std::string const& protocols::jd2::JobDistributor::batch ( core::Size batch_id )

inlineprotected

give name of batch with given id

References batches_.

Referenced by add_batch(), and protocols::jd2::archive::MPIArchiveJobDistributor::sync_batches().

◆ batch_underflow()

virtual void protocols::jd2::JobDistributor::batch_underflow ( )

inlineprotectedvirtual

if end of batches_ reached via next_batch or set_batch_id ...

Reimplemented in protocols::jd2::archive::MPIArchiveJobDistributor.

Referenced by next_batch(), and set_batch_id().

◆ begin_critical_section()

void protocols::jd2::JobDistributor::begin_critical_section ( )

protected

Referenced by write_output_from_job().

◆ check_for_parser_in_go_main()

void protocols::jd2::JobDistributor::check_for_parser_in_go_main ( )

protected

Send a message to the screen indicating that the parser is in use and that the mover that's been input to go_main will not be used, but instead will be replaced by the Mover created by the parser.

◆ clear_current_job_output()

void protocols::jd2::JobDistributor::clear_current_job_output ( )

protected

References current_job_, and current_job_id_.

Referenced by protocols::jd2::MPIFileBufJobDistributor::mark_current_job_id_for_repetition(), protocols::jd2::MPIWorkPartitionJobDistributor::mark_current_job_id_for_repetition(), and protocols::jd2::MPIWorkPoolJobDistributor::mark_current_job_id_for_repetition().

◆ create_singleton_instance()

JobDistributor * protocols::jd2::JobDistributor::create_singleton_instance ( )

staticprivate

private singleton creation function to be used with utility::thread::threadsafe_singleton

References protocols::jd2::JobDistributorFactory::create_job_distributor().

Referenced by get_instance().

◆ current_batch_id()

core::Size protocols::jd2::JobDistributor::current_batch_id ( ) const

inline

what is the current batch number ? — refers to position in batches_

References current_batch_id_.

Referenced by protocols::jd2::archive::MPIArchiveJobDistributor::batch_underflow(), protocols::jd2::current_batch_id(), protocols::jd2::archive::MPIArchiveJobDistributor::load_new_batch(), protocols::jd2::MPIFileBufJobDistributor::mark_job_as_bad(), protocols::jd2::MPIFileBufJobDistributor::mark_job_as_completed(), protocols::jd2::MPIFileBufJobDistributor::master_get_new_job_id(), protocols::jd2::archive::MPIArchiveJobDistributor::master_to_archive(), run_one_job(), and protocols::jd2::MPIFileBufJobDistributor::send_job_to_slave().

◆ current_job()

JobOP protocols::jd2::JobDistributor::current_job ( ) const

virtual

Movers may ask their controlling job distributor for information about the current job. They may also write information to this job for later output, though this use is now discouraged as the addition of the MultiplePoseMover now means that a single job may include several separate trajectories.

Reimplemented in protocols::jd2::VectorPoseJobDistributor.

References current_job_.

◆ current_job_finished()

void protocols::jd2::JobDistributor::current_job_finished ( )

protectedvirtual

Derived classes are allowed to clean up any temporary files or data relating to the current job after the current job has completed. Called inside go_main loop. Default implementation is a no-op.

Reimplemented in protocols::jd2::FileSystemJobDistributor.

Referenced by run_one_job().

◆ current_job_id()

core::Size protocols::jd2::JobDistributor::current_job_id ( ) const

integer access - which job are we on?

References current_job_id_.

Referenced by protocols::canonical_sampling::jobname_dirhash(), protocols::jd2::MPIWorkPartitionJobDistributor::mark_current_job_id_for_repetition(), protocols::jd2::MPIFileBufJobDistributor::master_get_new_job_id(), protocols::jd2::MPIFileBufJobDistributor::master_go(), protocols::jd2::archive::MPIArchiveJobDistributor::notify_archive(), protocols::jd2::MPIFileBufJobDistributor::process_message(), protocols::jd2::MPIFileBufJobDistributor::send_job_to_slave(), and protocols::jd2::MPIFileBufJobDistributor::slave_mark_current_job_id_for_repetition().

◆ current_output_name()

std::string protocols::jd2::JobDistributor::current_output_name ( ) const

virtual

Movers may ask their controlling job distributor for the output name as defined by the Job and JobOutputter.

References current_job(), and job_outputter().

Referenced by protocols::simple_filters::MPSpanAngleFilter::compute(), and protocols::simple_filters::ResidueLipophilicityFilter::compute().

◆ end_critical_section()

void protocols::jd2::JobDistributor::end_critical_section ( )

protected

Referenced by write_output_from_job().

◆ get_current_batch()

std::string protocols::jd2::JobDistributor::get_current_batch ( ) const

what is the current batch ? — name refers to the flag-file used for this batch

BATCH interface:

the BATCH interface of the JobDistributor is used to allow consecutive execution of a set of jobs with different flags different batches behave like completely independent rosetta runs — but of course a number of processes can already work on a new batch, while others are still finishing the last jobs of the previous batch. run from command-line with -run:batches flag1 flag2 flag3 the flag1 flag2... point to @flag1 files that are added to all other flags ( and removed at end of batch ) you can have all output in same output file or ( by redefining e.g. -out:file:silent in each batch-flag file ) in different output files

References batches_, protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID, and current_batch_id_.

Referenced by protocols::jd2::BatchJobInputter::check_batch(), protocols::jd2::current_batch(), next_batch(), obtain_new_job(), protocols::jd2::archive::MPIArchiveJobDistributor::process_message(), and protocols::jd2::MPIFileBufJobDistributor::slave_get_new_job_id().

◆ get_instance()

JobDistributor * protocols::jd2::JobDistributor::get_instance ( )

static

static function to get the instance of ( pointer to) this singleton class

References create_singleton_instance(), and instance_.

◆ get_job_list_from_job_inputter()

void protocols::jd2::JobDistributor::get_job_list_from_job_inputter ( )

private

References job_inputter_, protocols::jd2::JobInputter::job_inputter_input_source_to_string(), and jobs_.

Referenced by init_jd(), and set_job_inputter().

◆ get_job_time_estimate()

core::Size protocols::jd2::JobDistributor::get_job_time_estimate ( ) const

protected

Get an estimate of the time to run an additional job. If it can't be estimated, return a time of zero.

Right now this is just the average runtime for the completed jobs.

References jobs_, and protocols::jd2::tr().

Referenced by run_one_job().

◆ get_jobs()

JobsContainer const & protocols::jd2::JobDistributor::get_jobs ( ) const

protected

Read access to private data for derived classes.

Jobs is the container of Job objects

References jobs_.

Referenced by protocols::jd2::MPIWorkPartitionJobDistributor::get_new_job_id(), protocols::jd2::ShuffleFileSystemJobDistributor::get_new_job_id(), protocols::jd2::BOINCJobDistributor::go(), protocols::jd2::VectorPoseJobDistributor::go_mpi(), protocols::jd2::VectorPoseJobDistributor::go_serial(), protocols::jd2::archive::MPIArchiveJobDistributor::load_new_batch(), protocols::jd2::MPIFileBufJobDistributor::master_get_new_job_id(), protocols::jd2::MPIFileBufJobDistributor::master_remove_bad_inputs_from_job_list(), protocols::jd2::MPIWorkPartitionJobDistributor::MPIWorkPartitionJobDistributor(), protocols::jd2::archive::MPIArchiveJobDistributor::notify_archive(), and protocols::jd2::MPIWorkPartitionJobDistributor::remove_bad_inputs_from_job_list().

◆ get_jobs_nonconst()

JobsContainer & protocols::jd2::JobDistributor::get_jobs_nonconst ( )

protected

Jobs is the container of Job objects.

This version provides nonconst access, for cases where the job list must be updated on the fly.

References jobs_.

Referenced by protocols::jd2::MPIWorkPoolJobDistributor::master_get_new_job_id(), protocols::jd2::MPIWorkPoolJobDistributor::master_mark_job_as_completed(), protocols::jd2::MPIWorkPoolJobDistributor::master_mark_job_as_failed(), protocols::jd2::MPIWorkPoolJobDistributor::master_remove_bad_inputs_from_job_list(), protocols::jd2::MPIWorkPoolJobDistributor::MPIWorkPoolJobDistributor(), and protocols::jd2::MPIWorkPoolJobDistributor::slave_get_new_job_id().

◆ get_new_job_id()

virtual core::Size protocols::jd2::JobDistributor::get_new_job_id ( )

protectedpure virtual

Return 0 to signal that no available jobs remain. Otherwise return an index into the Jobs object.

Implemented in protocols::jd2::VectorPoseJobDistributor, protocols::jd2::ShuffleFileSystemJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::FileSystemJobDistributor, and protocols::jd2::BOINCJobDistributor.

Referenced by obtain_new_job().

◆ go() [1/4]

void protocols::jd2::JobDistributor::go ( protocols::moves::MoverOP mover )

virtual

This may be overridden by derived classes. Default implementation invokes go_main.

Reimplemented in protocols::unfolded_state_energy_calculator::UnfoldedStateEnergyCalculatorJobDistributor, protocols::jd2::VectorPoseJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::BOINCJobDistributor, and protocols::jd2::archive::MPIArchiveJobDistributor.

References go_main().

Referenced by protocols::abinitio::Broker_main(), protocols::canonical_sampling::canonical_sampling_main(), protocols::comparative_modeling::cm_main(), go(), protocols::medal::graphics_main(), protocols::star::graphics_main(), ligand_dock_main(), protocols::loop_build::LoopBuild_main(), protocols::loophash::loophash_main(), protocols::RBSegmentRelax_main(), protocols::relax::Relax_main(), and protocols::symmetric_docking::SymDock_main().

◆ go() [2/4]

void protocols::jd2::JobDistributor::go	(	protocols::moves::MoverOP	mover,
		JobInputterOP	ji
	)

invokes go, after setting JobInputter

References go(), and set_job_inputter().

◆ go() [3/4]

void protocols::jd2::JobDistributor::go	(	protocols::moves::MoverOP	mover,
		JobInputterOP	ji,
		JobOutputterOP	jo
	)

invokes go, after setting JobInputter and JobOutputter

References go(), job_outputter_, and set_job_inputter().

◆ go() [4/4]

void protocols::jd2::JobDistributor::go	(	protocols::moves::MoverOP	mover,
		JobOutputterOP	jo
	)

invokes go, after setting JobOutputter

References go(), and job_outputter_.

◆ go_main()

void protocols::jd2::JobDistributor::go_main ( protocols::moves::MoverOP mover )

protected

Non-virtual get-job, run it, & output loop. This function is pretty generic and your subclass may be able to use it. It is NOT virtual - this implementation can be shared by (at least) the simple FileSystemJobDistributor, the MPIWorkPoolJobDistributor, and the MPIWorkPartitionJobDistributor. Do not feel that you need to use it as-is in your class - but DO plan on implementing all its functionality!

References batches_, job_outputter_, jobs_, note_all_jobs_finished(), number_failed_jobs_, obtain_new_job(), run_one_job(), core::id::to_string(), protocols::jd2::tr(), and write_citations_and_clear_citation_tracking().

Referenced by go(), protocols::jd2::archive::MPIArchiveJobDistributor::go(), protocols::jd2::MPIFileBufJobDistributor::go(), protocols::jd2::MPIWorkPartitionJobDistributor::go(), protocols::unfolded_state_energy_calculator::UnfoldedStateEnergyCalculatorJobDistributor::go(), and protocols::jd2::MPIWorkPoolJobDistributor::slave_go().

◆ handle_interrupt()

virtual void protocols::jd2::JobDistributor::handle_interrupt ( )

protectedpure virtual

This function got called when job is not yet finished and got termitated abnormaly (ctrl-c, kill etc). when implimenting it in subclasses make sure to delete all in-progress-data that your job spawns.

Implemented in protocols::jd2::VectorPoseJobDistributor, protocols::jd2::ShuffleFileSystemJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::FileSystemJobDistributor, and protocols::jd2::archive::MPIArchiveJobDistributor.

Referenced by jd2_signal_handler().

◆ has_been_instantiated()

bool protocols::jd2::JobDistributor::has_been_instantiated ( )

static

Has the job distributor been instantiated?

Returns true for yes and false for no.

Author: Vikram K. Mulligan (vmull.nosp@m.ig@u.nosp@m.w.edu).

References instance_.

Referenced by protocols::jd2::jd2_used().

◆ increment_failed_jobs()

void protocols::jd2::JobDistributor::increment_failed_jobs ( )

inlineprotected

Increment the number of failed jobs.

References number_failed_jobs_.

Referenced by job_failed(), and protocols::jd2::MPIWorkPoolJobDistributor::job_failed().

◆ init_jd()

void protocols::jd2::JobDistributor::init_jd ( )

private

References batches_, protocols::jd2::JobDistributorFactory::create_job_inputter(), protocols::jd2::JobDistributorFactory::create_job_outputter(), protocols::jd2::JobDistributorFactory::create_parser(), current_batch_id_, get_job_list_from_job_inputter(), instance_, job_inputter_, protocols::jd2::JobInputter::job_inputter_input_source_to_string(), job_outputter_, parser_, populate_batch_list_from_cmd(), and protocols::jd2::tr().

Referenced by JobDistributor(), and restart().

◆ jd2_signal_handler()

void protocols::jd2::JobDistributor::jd2_signal_handler ( int signal_ )

staticprotected

Default callback function for signal handling.

Default callback function for signal handling

References get_instance(), handle_interrupt(), and job_outputter_.

◆ job_failed()

void protocols::jd2::JobDistributor::job_failed	(	core::pose::Pose &	,
		bool	will_retry
	)

protectedvirtual

This function is called when we give up on the job; it has been virtualized so BOINC and MPI can delay/protect output.

Mark job as bad, so at the end of execution we know definitively how many jobs failed for any reason This function (in this classes implementation) increments the number_failed_jobs_ class variable. If you write a child JobDistributor and do not want an exception to be thrown at the end of execution if some jobs failed, be sure to override this function so that it does not increment number_failed_jobs_.

Mark job as bad, so at the end of execution we know definitively how many jobs failed for any reason

Reimplemented in protocols::jd2::MPIMultiCommJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::BOINCJobDistributor, and protocols::jd2::VectorPoseJobDistributor.

References current_job_id_, and increment_failed_jobs().

Referenced by protocols::jd2::FileSystemJobDistributor::job_failed(), run_one_job(), and write_output_from_job().

◆ job_inputter()

JobInputterOP protocols::jd2::JobDistributor::job_inputter ( ) const

JobInputter access.

References job_inputter_.

Referenced by protocols::jd2::get_current_job(), protocols::jd2::get_current_jobs_starting_pose(), protocols::jd2::Job::get_pose(), protocols::jd2::VectorPoseJobDistributor::go_mpi(), and protocols::jd2::VectorPoseJobDistributor::go_serial().

◆ job_inputter_input_source()

JobInputterInputSource::Enum protocols::jd2::JobDistributor::job_inputter_input_source ( ) const

The input source for the current JobInputter.

References job_inputter_.

◆ job_outputter()

JobOutputterOP protocols::jd2::JobDistributor::job_outputter ( ) const

Movers (or derived classes) may ask for the JobOutputter.

References job_outputter_.

◆ job_succeeded()

void protocols::jd2::JobDistributor::job_succeeded	(	core::pose::Pose &	pose,
		core::Real	run_time,
		std::string const &	tag
	)

protectedvirtual

This function is called upon a successful job completion; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter.

Reimplemented in protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::FileSystemJobDistributor, and protocols::jd2::BOINCJobDistributor.

References current_job_, current_job_id_, job_outputter_, and mark_job_as_completed().

Referenced by protocols::jd2::FileSystemJobDistributor::job_succeeded(), and write_output_from_job().

◆ job_succeeded_additional_output()

void protocols::jd2::JobDistributor::job_succeeded_additional_output	(	core::pose::Pose &	pose,
		std::string const &	tag
	)

protectedvirtual

This function is called upon a successful job completion if there are additional poses generated by the mover base implementation is just a call to the job outputter.

References current_job_, and job_outputter_.

Referenced by write_output_from_job().

◆ load_new_batch()

void protocols::jd2::JobDistributor::load_new_batch ( )

protectedvirtual

called by next_batch() or set_batch_id() to switch-over and restart JobDistributor on new batch

@detail restart JobDistributor with a new batch the BatchJobInputter loads new flags and sets global options after this we reload Factory dependent Objects (e.g., JobInputter and JobOutputter )

Reimplemented in protocols::jd2::archive::MPIArchiveJobDistributor.

References batches_, protocols::jd2::JobDistributorFactory::create_job_outputter(), protocols::jd2::JobDistributorFactory::create_parser(), current_batch_id_, current_job_, current_job_id_, protocols::jd2::JD2_BOGUS_JOB, job_inputter_, job_outputter_, jobs_, parser_, and protocols::jd2::tr().

Referenced by protocols::jd2::archive::MPIArchiveJobDistributor::load_new_batch(), next_batch(), and set_batch_id().

◆ mark_current_job_id_for_repetition()

virtual void protocols::jd2::JobDistributor::mark_current_job_id_for_repetition ( )

protectedpure virtual

this function is called whenever a job "soft-fails" and needs to be retried. Generally it should ensure that the subsequent call to obtain_new_job returns this job over again.

Implemented in protocols::jd2::VectorPoseJobDistributor, protocols::jd2::ShuffleFileSystemJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::FileSystemJobDistributor, and protocols::jd2::BOINCJobDistributor.

Referenced by write_output_from_job().

◆ mark_job_as_bad()

void protocols::jd2::JobDistributor::mark_job_as_bad ( core::Size job_id )

protected

Referenced by protocols::jd2::MPIFileBufJobDistributor::master_remove_bad_inputs_from_job_list().

◆ mark_job_as_completed()

void protocols::jd2::JobDistributor::mark_job_as_completed	(	core::Size	job_id,
		core::Real	run_time
	)

protected

Jobs is the container of Job objects need non-const to mark Jobs as completed on Master in MPI-JobDistributor.

References job_outputter_, jobs_, and protocols::jd2::tr().

Referenced by job_succeeded(), and protocols::jd2::MPIFileBufJobDistributor::mark_job_as_completed().

◆ mpi_finalize()

void protocols::jd2::JobDistributor::mpi_finalize ( bool finalize )

virtual

should the go() function call MPI_finalize()? It probably should, this is true by default.

Reimplemented in protocols::jd2::MPIWorkPoolJobDistributor.

◆ next_batch()

bool protocols::jd2::JobDistributor::next_batch ( )

protectedvirtual

switch current_batch_id_ to next batch

Reimplemented in protocols::jd2::MPIFileBufJobDistributor.

References batch_underflow(), batches_, protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID, current_batch_id_, get_current_batch(), load_new_batch(), and protocols::jd2::tr().

Referenced by protocols::jd2::MPIFileBufJobDistributor::next_batch(), and obtain_new_job().

◆ note_all_jobs_finished()

void protocols::jd2::JobDistributor::note_all_jobs_finished ( )

protectedvirtual

Derived classes are allowed to perform some kind of action when the job distributor runs out of jobs to execute. Called inside go_main. Default implementation is a no-op.

Referenced by go_main().

◆ nr_batches()

core::Size protocols::jd2::JobDistributor::nr_batches ( ) const

inlineprotected

how many batches are in our list ... this can change dynamically

References batches_.

Referenced by protocols::jd2::archive::MPIArchiveJobDistributor::notify_archive(), and protocols::jd2::archive::MPIArchiveJobDistributor::sync_batches().

◆ obtain_new_job()

bool protocols::jd2::JobDistributor::obtain_new_job ( bool re_consider_current_job = false )

protected

this function updates the current_job_id_ and current_job_ fields. The boolean return states whether or not a new job was obtained (if false, quit distributing!)

References batches_, protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID, current_job_, current_job_id_, get_current_batch(), get_new_job_id(), and next_batch().

Referenced by go_main(), protocols::jd2::MPIFileBufJobDistributor::master_go(), protocols::jd2::MPIFileBufJobDistributor::master_remove_bad_inputs_from_job_list(), and protocols::jd2::MPIFileBufJobDistributor::process_message().

◆ parser()

rosetta_scripts::RosettaScriptsParserOP protocols::jd2::JobDistributor::parser ( ) const

protected

Parser access.

References parser_.

Referenced by protocols::jd2::VectorPoseJobDistributor::go_mpi(), and protocols::jd2::VectorPoseJobDistributor::go_serial().

◆ populate_batch_list_from_cmd()

void protocols::jd2::JobDistributor::populate_batch_list_from_cmd ( )

private

read -run:batches

read -run:batches and put it into batches_ vector.

References batches_.

Referenced by init_jd().

◆ remove_bad_inputs_from_job_list()

void protocols::jd2::JobDistributor::remove_bad_inputs_from_job_list ( )

protectedvirtual

this function is called inside go_main if a mover returns FAIL_BAD_INPUT. Its purpose is to remove other jobs with the same input (which will still be bad) from the available list of jobs. The default implementation is a no-op and not all distributors are expected/able to implement this functionality, only those that can guaruntee no other jobs of that input are currently running.

Reimplemented in protocols::jd2::MPIWorkPoolJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIFileBufJobDistributor, and protocols::jd2::FileSystemJobDistributor.

Referenced by run_one_job(), and write_output_from_job().

◆ remove_system_signal_handler()

void protocols::jd2::JobDistributor::remove_system_signal_handler ( )

staticprotected

Set signal handler back to default state.

◆ reset_job_state()

void protocols::jd2::JobDistributor::reset_job_state ( )

private

References current_batch_id_, current_job_, current_job_id_, protocols::jd2::JD2_BOGUS_JOB, jobs_, and last_completed_job_.

Referenced by restart(), and set_job_inputter().

◆ restart()

void protocols::jd2::JobDistributor::restart ( )

virtual

restart job-distribution from beginning – useful if you need a second pass over decoys...

Reimplemented in protocols::jd2::FileSystemJobDistributor.

References init_jd(), and reset_job_state().

Referenced by protocols::jd2::FileSystemJobDistributor::restart().

◆ run_one_job()

bool protocols::jd2::JobDistributor::run_one_job	(	protocols::moves::MoverOP &	mover,
		time_t	allstarttime,
		std::string &	last_inner_job_tag,
		std::string &	last_output_tag,
		core::Size &	last_batch_id,
		core::Size &	retries_this_job,
		bool	first_job
	)

protected

◆ set_batch_id()

void protocols::jd2::JobDistributor::set_batch_id ( core::Size setting )

protected

set current_batch_id — eg for slave nodes in MPI framework

References batch_underflow(), batches_, current_batch_id_, load_new_batch(), and protocols::jd2::tr().

Referenced by protocols::jd2::MPIFileBufJobDistributor::slave_get_new_job_id().

◆ set_current_job_by_index()

void protocols::jd2::JobDistributor::set_current_job_by_index ( core::Size curr_job_index )

protected

For derived classes that wish to invoke JobDistributor functions which use the current_job_ and current_job_id_ member variables. Note that until those functions complete, it would be a bad idea for another thread to change current_job_.

References current_job_, and current_job_id_.

◆ set_job_inputter()

void protocols::jd2::JobDistributor::set_job_inputter ( JobInputterOP new_job_inputter )

Set the JobInputter and reset the Job list – this is not something you want to do after go() has been called, but before it has returned.

References get_job_list_from_job_inputter(), job_inputter_, and reset_job_state().

Referenced by go().

◆ set_job_outputter()

void protocols::jd2::JobDistributor::set_job_outputter ( const JobOutputterOP & new_job_outputter )

Movers (or derived classes) may ask for the JobOutputter.

References job_outputter_.

◆ setup_pymol_observer()

void protocols::jd2::JobDistributor::setup_pymol_observer ( core::pose::Pose & pose )

protected

After the construction of the pose for this job, check the command line to determine if the pymol observer should be attached to it.

References protocols::moves::AddPyMOLObserver(), protocols::moves::AddPyMOLObserver_to_conformation(), protocols::moves::AddPyMOLObserver_to_energies(), and protocols::jd2::tr().

Referenced by run_one_job().

◆ setup_system_signal_handler()

void protocols::jd2::JobDistributor::setup_system_signal_handler ( void(*)(int) signal_fn = jd2_signal_handler )

staticprotected

Setting up callback function that will be call when our process is about to terminate. This will allow us to exit propely (clean up in_progress_files/tmp files if any).

Setting up callback function that will be call when our process is about to terminate.

◆ total_nr_jobs()

core::Size protocols::jd2::JobDistributor::total_nr_jobs ( ) const

inline

References jobs_.

Referenced by protocols::features::TopPercentOfAllInputs::TopPercentOfAllInputs().

◆ using_parser()

bool protocols::jd2::JobDistributor::using_parser ( ) const

protected

Is the parser in use?

Checks parser::protocol flag since this is JD2.

References parser_.

Referenced by run_one_job().

◆ write_citations_and_clear_citation_tracking()

void protocols::jd2::JobDistributor::write_citations_and_clear_citation_tracking ( ) const

protected

Write out information about all modules that have been used that should be cited, then clear the list of citations from the CitationManager.

Author: Vikram K. Mulligan (vmull.nosp@m.igan.nosp@m.@flat.nosp@m.iron.nosp@m.insti.nosp@m.tute.nosp@m..org).

Referenced by go_main().

◆ write_output_from_job()

void protocols::jd2::JobDistributor::write_output_from_job	(	core::pose::Pose &	pose,
		protocols::moves::MoverOP	mover_copy,
		protocols::moves::MoverStatus	status,
		core::Size	jobtime,
		core::Size &	retries_this_job
	)

protected

After a job has finished running, figure out from the MoverStatus whether the pose should be written to disk (or wherever) along with any other poses that the mover might have generated along the way.

References begin_critical_section(), current_job_, current_job_id_, end_critical_section(), protocols::moves::FAIL_BAD_INPUT, protocols::moves::FAIL_DO_NOT_RETRY, protocols::moves::FAIL_RETRY, job_failed(), job_outputter_, job_succeeded(), job_succeeded_additional_output(), last_completed_job_, mark_current_job_id_for_repetition(), protocols::moves::MS_SUCCESS, remove_bad_inputs_from_job_list(), and protocols::jd2::tr().

Referenced by run_one_job().

Friends And Related Function Documentation

◆ JobDistributorFactory

friend class JobDistributorFactory

friend

Member Data Documentation

◆ batches_

utility::vector1< std::string > protocols::jd2::JobDistributor::batches_

private

all batches if present

Referenced by add_batch(), batch(), get_current_batch(), go_main(), init_jd(), load_new_batch(), next_batch(), nr_batches(), obtain_new_job(), populate_batch_list_from_cmd(), and set_batch_id().

◆ current_batch_id_

core::Size protocols::jd2::JobDistributor::current_batch_id_

private

current_batch or 0

Referenced by current_batch_id(), get_current_batch(), init_jd(), load_new_batch(), next_batch(), reset_job_state(), and set_batch_id().

◆ current_job_

JobOP protocols::jd2::JobDistributor::current_job_

private

pointer to current job. Information is somewhat duplicated with current_job_id_.

Referenced by clear_current_job_output(), current_job(), job_succeeded(), job_succeeded_additional_output(), load_new_batch(), obtain_new_job(), reset_job_state(), run_one_job(), set_current_job_by_index(), and write_output_from_job().

◆ current_job_id_

core::Size protocols::jd2::JobDistributor::current_job_id_

private

access into jobs_ object indicating current job. Contains more information than current_job_ in that it can be incremented...

Referenced by clear_current_job_output(), current_job_id(), job_failed(), job_succeeded(), load_new_batch(), obtain_new_job(), reset_job_state(), set_current_job_by_index(), and write_output_from_job().

◆ instance_

JobDistributor * protocols::jd2::JobDistributor::instance_

staticprivate

Referenced by get_instance(), has_been_instantiated(), and init_jd().

◆ job_inputter_

JobInputterOP protocols::jd2::JobDistributor::job_inputter_

private

Referenced by get_job_list_from_job_inputter(), init_jd(), job_inputter(), job_inputter_input_source(), JobDistributor(), load_new_batch(), run_one_job(), and set_job_inputter().

◆ job_outputter_

JobOutputterOP protocols::jd2::JobDistributor::job_outputter_

private

Referenced by go(), go_main(), init_jd(), jd2_signal_handler(), job_outputter(), job_succeeded(), job_succeeded_additional_output(), JobDistributor(), load_new_batch(), mark_job_as_completed(), run_one_job(), set_job_outputter(), and write_output_from_job().

◆ jobs_

JobsContainerOP protocols::jd2::JobDistributor::jobs_

private

Container for the array of owning pointers to the Job objects.

By making this a class, it makes it easier to permit easy updating of the jobs list (e.g. if it's too long to hold the whole thing in memory).

Referenced by get_job_list_from_job_inputter(), get_job_time_estimate(), get_jobs(), get_jobs_nonconst(), go_main(), JobDistributor(), load_new_batch(), mark_job_as_completed(), reset_job_state(), run_one_job(), and total_nr_jobs().

◆ last_completed_job_

core::Size protocols::jd2::JobDistributor::last_completed_job_

private

access into jobs_ object indicating the previous job. Used with the -jd2:delete_old_poses option for deleting unnecessary poses

Referenced by reset_job_state(), run_one_job(), and write_output_from_job().

◆ number_failed_jobs_

core::Size protocols::jd2::JobDistributor::number_failed_jobs_

private

Number of failed jobs - kept track of so we can exit properly at the end of execution.

Referenced by go_main(), and increment_failed_jobs().

◆ parser_

RosettaScriptsParserOP protocols::jd2::JobDistributor::parser_

private

Referenced by init_jd(), JobDistributor(), load_new_batch(), parser(), run_one_job(), and using_parser().

The documentation for this class was generated from the following files:

src/protocols/jd2/JobDistributor.hh
src/protocols/jd2/JobDistributor.cc

Public Member Functions

Static Public Member Functions

Protected Member Functions

Static Protected Member Functions

Private Types

Private Member Functions

Static Private Member Functions

Private Attributes

Static Private Attributes

Friends

Detailed Description

Member Typedef Documentation

◆ RosettaScriptsParserOP

Constructor & Destructor Documentation

◆ JobDistributor() [1/2]

◆ JobDistributor() [2/2]

◆ ~JobDistributor()

Member Function Documentation

◆ add_batch()

◆ batch()

◆ batch_underflow()

◆ begin_critical_section()

◆ check_for_parser_in_go_main()

◆ clear_current_job_output()

◆ create_singleton_instance()

◆ current_batch_id()

◆ current_job()

◆ current_job_finished()

◆ current_job_id()

◆ current_output_name()

◆ end_critical_section()

◆ get_current_batch()

◆ get_instance()

◆ get_job_list_from_job_inputter()

◆ get_job_time_estimate()

◆ get_jobs()

◆ get_jobs_nonconst()

◆ get_new_job_id()

◆ go() [1/4]

◆ go() [2/4]

◆ go() [3/4]

◆ go() [4/4]

◆ go_main()

◆ handle_interrupt()

◆ has_been_instantiated()

◆ increment_failed_jobs()

◆ init_jd()

◆ jd2_signal_handler()

◆ job_failed()

◆ job_inputter()

◆ job_inputter_input_source()

◆ job_outputter()

◆ job_succeeded()

◆ job_succeeded_additional_output()

◆ load_new_batch()

◆ mark_current_job_id_for_repetition()

◆ mark_job_as_bad()

◆ mark_job_as_completed()

◆ mpi_finalize()

◆ next_batch()

◆ note_all_jobs_finished()

◆ nr_batches()

◆ obtain_new_job()

◆ parser()

◆ populate_batch_list_from_cmd()

◆ remove_bad_inputs_from_job_list()

◆ remove_system_signal_handler()

◆ reset_job_state()

◆ restart()

◆ run_one_job()

◆ set_batch_id()

◆ set_current_job_by_index()

◆ set_job_inputter()

◆ set_job_outputter()

◆ setup_pymol_observer()

◆ setup_system_signal_handler()

◆ total_nr_jobs()

◆ using_parser()

◆ write_citations_and_clear_citation_tracking()

◆ write_output_from_job()