Rosetta 3.4
Public Member Functions | Static Public Member Functions | Protected Member Functions | Static Protected Member Functions | Friends
protocols::jd2::JobDistributor Class Reference

#include <JobDistributor.hh>

Inheritance diagram for protocols::jd2::JobDistributor:
Inheritance graph
[legend]
Collaboration diagram for protocols::jd2::JobDistributor:
Collaboration graph
[legend]

List of all members.

Public Member Functions

virtual ~JobDistributor ()
virtual void go (protocols::moves::MoverOP mover)
 This may be overridden by derived classes. Default implementation invokes go_main.
void go (protocols::moves::MoverOP mover, JobOutputterOP jo)
 invokes go, after setting JobOutputter
JobOP current_job () const
 Movers may ask their controlling job distributor for information about the current job. They may also load information into this job for later output.
std::string current_output_name () const
 Movers may ask their controlling job distributor for the output name as defined by the Job and JobOutputter.
JobOutputterOP job_outputter () const
 Movers (or derived classes) may ask for the JobOutputter.
void set_job_outputter (const JobOutputterOP &new_job_outputter)
 Movers (or derived classes) may ask for the JobOutputter.
JobInputterOP job_inputter () const
 JobInputter access.
virtual void mpi_finalize (bool finalize)
 should the go() function call MPI_finalize()? It probably should, this is true by default.
JobInputterInputSource::Enum job_inputter_input_source () const
 The input source for the current JobInputter.
virtual void restart ()
core::Size total_nr_jobs () const
core::Size current_job_id () const
 integer access - which job are we on?
std::string get_current_batch () const
 what is the current batch ? --- name refers to the flag-file used for this batch
virtual void add_batch (std::string const &, core::Size id=0)
 add a new batch ( name will be interpreted as flag_file )
core::Size current_batch_id () const
 what is the current batch number ? --- refers to position in batches_

Static Public Member Functions

static JobDistributorget_instance ()

Protected Member Functions

 JobDistributor ()
 Singleton instantiation pattern; Derived classes will call default ctor, but their ctors, too must be protected (and the JDFactory must be their friend.)
 JobDistributor (bool empty)
 MPIArchiveJobDistributor starts with an empty job-list...
void go_main (protocols::moves::MoverOP mover)
 Non-virtual get-job, run it, & output loop. This function is pretty generic and your subclass may be able to use it. It is NOT virtual - this implementation can be shared by (at least) the simple FileSystemJobDistributor, the MPIWorkPoolJobDistributor, and the MPIWorkPartitionJobDistributor. Do not feel that you need to use it as-is in your class - but DO plan on implementing all its functionality!
Jobs const & get_jobs () const
 Read access to private data for derived classes.
void mark_job_as_completed (core::Size job_id, core::Real run_time)
 Jobs is the container of Job objects need non-const to mark Jobs as completed on Master in MPI-JobDistributor.
void mark_job_as_bad (core::Size job_id)
ParserOP parser () const
 Parser access.
void begin_critical_section ()
void end_critical_section ()
bool obtain_new_job (bool re_consider_current_job=false)
 this function updates the current_job_id_ and current_job_ fields. The boolean return states whether or not a new job was obtained (if false, quit distributing!)
virtual core::Size get_new_job_id ()=0
 Return 0 to signal that no available jobs remain. Otherwise return an index into the Jobs object.
virtual void job_succeeded (core::pose::Pose &pose, core::Real run_time)
 This function is called upon a successful job completion; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter.
virtual void job_failed (core::pose::Pose &, bool)
 This function is called when we give up on the job; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter.
virtual void mark_current_job_id_for_repetition ()=0
 this function is called whenever a job "soft-fails" and needs to be retried. Generally it should ensure that the subsequent call to obtain_new_job returns this job over again.
virtual void remove_bad_inputs_from_job_list ()
 this function is called inside go_main if a mover returns FAIL_BAD_INPUT. Its purpose is to remove other jobs with the same input (which will still be bad) from the available list of jobs. The default implementation is a no-op and not all distributors are expected/able to implement this functionality, only those that can guaruntee no other jobs of that input are currently running.
virtual void current_job_finished ()
 Derived classes are allowed to clean up any temporary files or data relating to the current job after the current job has completed. Called inside go_main loop. Default implementation is a no-op.
virtual void note_all_jobs_finished ()
 Derived classes are allowed to perform some kind of action when the job distributor runs out of jobs to execute. Called inside go_main. Default implementation is a no-op.
void clear_current_job_output ()
virtual void handle_interrupt ()=0
 This function got called when job is not yet finished and got termitated abnormaly (ctrl-c, kill etc). when implimenting it in subclasses make sure to delete all in-progress-data that your job spawn.
void set_batch_id (core::Size setting)
 set current_batch_id --- eg for slave nodes in MPI framework
virtual bool next_batch ()
 switch current_batch_id_ to next batch
virtual void batch_underflow ()
 if end of batches_ reached via next_batch or set_batch_id ...
virtual void load_new_batch ()
 called by next_batch() or set_batch_id() to switch-over and restart JobDistributor on new batch
core::Size nr_batches () const
 how many batches are in our list ... this can change dynamically
std::string const & batch (core::Size batch_id)
 give name of batch with given id

Static Protected Member Functions

static void setup_system_signal_handler (void(*prev_fn)(int)=jd2_signal_handler)
 Setting up callback function that will be call when our process is about to terminate.
static void remove_system_signal_handler ()
 Set signal handler back to default state.
static void jd2_signal_handler (int Signal)
 Default callback function for signal handling.

Friends

class JobDistributorFactory

Detailed Description

Enforced JobDistributor destruction turned out to cause problems - calls to Tracers and/or the Options system during destruction caused crashes if those systems had been destructed first. So this is deprecated.


Constructor & Destructor Documentation

protocols::jd2::JobDistributor::JobDistributor ( ) [protected]

Singleton instantiation pattern; Derived classes will call default ctor, but their ctors, too must be protected (and the JDFactory must be their friend.)

protocols::jd2::JobDistributor::JobDistributor ( bool  empty) [protected]

MPIArchiveJobDistributor starts with an empty job-list...

References protocols::jd2::JobDistributorFactory::create_parser().

protocols::jd2::JobDistributor::~JobDistributor ( ) [virtual]

WARNING WARNING! SINGLETONS' DESTRUCTORS ARE NEVER CALLED IN MINI! DO NOT TRY TO PUT THINGS IN THIS FUNCTION! here's a nice link explaining why: http://www.research.ibm.com/designpatterns/pubs/ph-jun96.txt


Member Function Documentation

void protocols::jd2::JobDistributor::add_batch ( std::string const &  batch,
core::Size  id = 0 
) [virtual]

add a new batch ( name will be interpreted as flag_file )

add new batch to end of batch list... this might be called asynchronous... ie. while we are still in the middle of a current batch, or while we are in non-batch mode

References batch(), and protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID.

Referenced by protocols::jd2::archive::ArchiveManager::cancel_batch(), and protocols::jd2::archive::ArchiveManager::queue_batch().

std::string const& protocols::jd2::JobDistributor::batch ( core::Size  batch_id) [inline, protected]

give name of batch with given id

Referenced by add_batch().

virtual void protocols::jd2::JobDistributor::batch_underflow ( ) [inline, protected, virtual]

if end of batches_ reached via next_batch or set_batch_id ...

Reimplemented in protocols::jd2::archive::MPIArchiveJobDistributor.

Referenced by next_batch(), and set_batch_id().

void protocols::jd2::JobDistributor::begin_critical_section ( ) [protected]

Reimplemented in protocols::jd2::BOINCJobDistributor.

Referenced by go_main().

void protocols::jd2::JobDistributor::clear_current_job_output ( ) [protected]
core::Size protocols::jd2::JobDistributor::current_batch_id ( ) const [inline]
JobOP protocols::jd2::JobDistributor::current_job ( ) const
void protocols::jd2::JobDistributor::current_job_finished ( ) [protected, virtual]

Derived classes are allowed to clean up any temporary files or data relating to the current job after the current job has completed. Called inside go_main loop. Default implementation is a no-op.

Reimplemented in protocols::jd2::FileSystemJobDistributor.

Referenced by go_main().

core::Size protocols::jd2::JobDistributor::current_job_id ( ) const
std::string protocols::jd2::JobDistributor::current_output_name ( ) const

Movers may ask their controlling job distributor for the output name as defined by the Job and JobOutputter.

References current_job(), and job_outputter().

Referenced by protocols::features::FeaturesReporter::find_tag().

void protocols::jd2::JobDistributor::end_critical_section ( ) [protected]

Reimplemented in protocols::jd2::BOINCJobDistributor.

Referenced by go_main().

std::string protocols::jd2::JobDistributor::get_current_batch ( ) const

what is the current batch ? --- name refers to the flag-file used for this batch

BATCH interface:

the BATCH interface of the JobDistributor is used to allow consecutive execution of a set of jobs with different flags different batches behave like completely independent rosetta runs --- but of course a number of processes can already work on a new batch, while others are still finishing the last jobs of the previous batch. run from command-line with -run:batches flag1 flag2 flag3 the flag1 flag2... point to files that are added to all other flags ( and removed at end of batch ) you can have all output in same output file or ( by redefining e.g. -out:file:silent in each batch-flag file ) in different output files

References protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID.

Referenced by next_batch(), obtain_new_job(), protocols::jd2::archive::MPIArchiveJobDistributor::process_message(), and protocols::jd2::MPIFileBufJobDistributor::slave_get_new_job_id().

JobDistributor * protocols::jd2::JobDistributor::get_instance ( ) [static]

References protocols::jd2::JobDistributorFactory::create_job_distributor().

Referenced by protocols::jd2::JobOutputter::affixed_numbered_name(), protocols::toolbox::task_operations::ReadResfileFromDB::apply(), protocols::symmetric_docking::SymDockProtocol::apply(), protocols::surface_docking::SurfaceDockingProtocol::apply(), protocols::surface_docking::FullatomRelaxMover::apply(), protocols::simple_moves::ddG::apply(), protocols::RotamerDump::RotamerDumpMover::apply(), protocols::qsar::qsarMover::apply(), protocols::protein_interface_design::movers::InterfaceRecapitulationMover::apply(), protocols::protein_interface_design::filters::FilterScanFilter::apply(), protocols::ligand_docking::Transform::apply(), protocols::ligand_docking::StartFrom::apply(), protocols::ligand_docking::LigandDockMain::apply(), protocols::ligand_docking::InterfaceScoreCalculator::apply(), protocols::frag_picker::nonlocal::NonlocalFrags::apply(), protocols::docking::DockingProtocol::apply(), protocols::contact_map::ContactMap::apply(), protocols::canonical_sampling::CanonicalSamplingMover::apply(), protocols::antibody2::Ab_ModelCDRH3::apply(), protocols::antibody2::Ab_Assemble_Templates::apply(), protocols::anchored_design::AnchoredDesignMover::apply(), protocols::analysis::LoopAnalyzerMover::apply(), protocols::abinitio::Broker_main(), protocols::anchored_design::AnchoredDesignMover::calculate_rmsd(), protocols::jd2::archive::ArchiveManager::cancel_batch(), protocols::canonical_sampling::canonical_sampling_main(), protocols::comparative_modeling::cm_main(), protocols::jd2::current_output_filename(), protocols::jd2::current_output_name(), protocols::jd2::current_replica(), protocols::features::FeaturesReporter::find_tag(), protocols::jd2::get_current_job(), protocols::jd2::get_current_jobs_starting_pose(), protocols::mpi::ArchiveMasterBase::get_new_decoys(), protocols::jd2::Job::get_pose(), protocols::features::get_write_delete_pair(), jd2_signal_handler(), protocols::jd2::jd2_used(), protocols::canonical_sampling::jobname_dirhash(), ligand_dock_main(), protocols::loophash::loophash_main(), main(), protocols::features::TopCountOfEachInput::operator()(), protocols::canonical_sampling::mc_convergence_checks::Pool_ConvergenceCheck::operator()(), OPT_1GRP_KEY(), protocols::jd2::output_intermediate_pose(), protocols::simple_moves::SequenceProfileMover::parse_my_tag(), protocols::protein_interface_design::movers::SubroutineMover::parse_my_tag(), protocols::canonical_sampling::MetropolisHastingsMover::prepare_simulation(), protocols::jd2::archive::ArchiveManager::queue_batch(), protocols::relax::Relax_main(), protocols::analysis::InterfaceAnalyzerMover::report_data(), protocols::features::JobDataFeatures::report_features(), protocols::rotamer_recovery::RRReporterSQLite::report_rotamer_recovery_full(), protocols::docking::DockingPrepackProtocol::score_and_output(), protocols::analysis::InterfaceAnalyzerMover::set_pose_info(), protocols::canonical_sampling::mc_convergence_checks::setup_convergence_checks_from_cmdline(), protocols::symmetric_docking::SymDock_main(), protocols::features::TopPercentOfAllInputs::TopPercentOfAllInputs(), protocols::canonical_sampling::MetricRecorder::update_after_boltzmann(), and protocols::jd2::write_score_tracer().

Jobs const & protocols::jd2::JobDistributor::get_jobs ( ) const [protected]
virtual core::Size protocols::jd2::JobDistributor::get_new_job_id ( ) [protected, pure virtual]
void protocols::jd2::JobDistributor::go ( protocols::moves::MoverOP  mover) [virtual]
void protocols::jd2::JobDistributor::go ( protocols::moves::MoverOP  mover,
JobOutputterOP  jo 
)

invokes go, after setting JobOutputter

References go().

void protocols::jd2::JobDistributor::go_main ( protocols::moves::MoverOP  mover) [protected]
virtual void protocols::jd2::JobDistributor::handle_interrupt ( ) [protected, pure virtual]

This function got called when job is not yet finished and got termitated abnormaly (ctrl-c, kill etc). when implimenting it in subclasses make sure to delete all in-progress-data that your job spawn.

Implemented in protocols::jd2::archive::MPIArchiveJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, and protocols::jd2::ShuffleFileSystemJobDistributor.

Referenced by jd2_signal_handler().

void protocols::jd2::JobDistributor::jd2_signal_handler ( int  signal_) [static, protected]

Default callback function for signal handling.

Default callback function for signal handling

References get_instance(), and handle_interrupt().

void protocols::jd2::JobDistributor::job_failed ( core::pose::Pose ,
bool   
) [protected, virtual]

This function is called when we give up on the job; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter.

no-op implementation in the base class

Reimplemented in protocols::jd2::BOINCJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIFileBufJobDistributor, and protocols::jd2::MPIMultiCommJobDistributor.

Referenced by go_main().

JobInputterOP protocols::jd2::JobDistributor::job_inputter ( ) const
JobInputterInputSource::Enum protocols::jd2::JobDistributor::job_inputter_input_source ( ) const

The input source for the current JobInputter.

JobOutputterOP protocols::jd2::JobDistributor::job_outputter ( ) const
void protocols::jd2::JobDistributor::job_succeeded ( core::pose::Pose pose,
core::Real  run_time 
) [protected, virtual]

This function is called upon a successful job completion; it has been virtualized so BOINC and MPI can delay/protect output base implementation is just a call to the job outputter.

Reimplemented in protocols::jd2::BOINCJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIMultiCommJobDistributor, and protocols::jd2::MPIWorkPoolJobDistributor.

References mark_job_as_completed().

Referenced by go_main().

void protocols::jd2::JobDistributor::load_new_batch ( ) [protected, virtual]
virtual void protocols::jd2::JobDistributor::mark_current_job_id_for_repetition ( ) [protected, pure virtual]

this function is called whenever a job "soft-fails" and needs to be retried. Generally it should ensure that the subsequent call to obtain_new_job returns this job over again.

Implemented in protocols::jd2::BOINCJobDistributor, protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, protocols::jd2::MPIWorkPoolJobDistributor, and protocols::jd2::ShuffleFileSystemJobDistributor.

Referenced by go_main().

void protocols::jd2::JobDistributor::mark_job_as_bad ( core::Size  job_id) [protected]
void protocols::jd2::JobDistributor::mark_job_as_completed ( core::Size  job_id,
core::Real  run_time 
) [protected]

Jobs is the container of Job objects need non-const to mark Jobs as completed on Master in MPI-JobDistributor.

References protocols::jd2::tr().

Referenced by job_succeeded(), and protocols::jd2::MPIFileBufJobDistributor::mark_job_as_completed().

void protocols::jd2::JobDistributor::mpi_finalize ( bool  finalize) [virtual]

should the go() function call MPI_finalize()? It probably should, this is true by default.

Reimplemented in protocols::jd2::MPIWorkPoolJobDistributor.

bool protocols::jd2::JobDistributor::next_batch ( ) [protected, virtual]
void protocols::jd2::JobDistributor::note_all_jobs_finished ( ) [protected, virtual]

Derived classes are allowed to perform some kind of action when the job distributor runs out of jobs to execute. Called inside go_main. Default implementation is a no-op.

Referenced by go_main().

core::Size protocols::jd2::JobDistributor::nr_batches ( ) const [inline, protected]

how many batches are in our list ... this can change dynamically

bool protocols::jd2::JobDistributor::obtain_new_job ( bool  re_consider_current_job = false) [protected]

this function updates the current_job_id_ and current_job_ fields. The boolean return states whether or not a new job was obtained (if false, quit distributing!)

References protocols::jd2::BatchJobInputter::BOGUS_BATCH_ID, get_current_batch(), get_new_job_id(), and next_batch().

Referenced by go_main(), protocols::jd2::MPIFileBufJobDistributor::master_go(), protocols::jd2::MPIFileBufJobDistributor::master_remove_bad_inputs_from_job_list(), and protocols::jd2::MPIFileBufJobDistributor::process_message().

ParserOP protocols::jd2::JobDistributor::parser ( ) const [protected]

Parser access.

void protocols::jd2::JobDistributor::remove_bad_inputs_from_job_list ( ) [protected, virtual]

this function is called inside go_main if a mover returns FAIL_BAD_INPUT. Its purpose is to remove other jobs with the same input (which will still be bad) from the available list of jobs. The default implementation is a no-op and not all distributors are expected/able to implement this functionality, only those that can guaruntee no other jobs of that input are currently running.

Reimplemented in protocols::jd2::FileSystemJobDistributor, protocols::jd2::MPIFileBufJobDistributor, protocols::jd2::MPIWorkPartitionJobDistributor, and protocols::jd2::MPIWorkPoolJobDistributor.

Referenced by go_main().

void protocols::jd2::JobDistributor::remove_system_signal_handler ( ) [static, protected]

Set signal handler back to default state.

Referenced by protocols::jd2::FileSystemJobDistributor::go().

void protocols::jd2::JobDistributor::restart ( ) [virtual]

restart job-distribution from beginning -- useful if you need a second pass over decoys...

Reimplemented in protocols::jd2::FileSystemJobDistributor.

References protocols::jd2::JD2_BOGUS_JOB.

void protocols::jd2::JobDistributor::set_batch_id ( core::Size  setting) [protected]

set current_batch_id --- eg for slave nodes in MPI framework

References batch_underflow(), load_new_batch(), and protocols::jd2::tr().

Referenced by protocols::jd2::MPIFileBufJobDistributor::slave_get_new_job_id().

void protocols::jd2::JobDistributor::set_job_outputter ( const JobOutputterOP new_job_outputter)

Movers (or derived classes) may ask for the JobOutputter.

void protocols::jd2::JobDistributor::setup_system_signal_handler ( void(*)(int signal_fn = jd2_signal_handler) [static, protected]

Setting up callback function that will be call when our process is about to terminate.

This will allow us to exit propely (clean up in_progress_files/tmp files if any).

Referenced by protocols::jd2::FileSystemJobDistributor::go().

core::Size protocols::jd2::JobDistributor::total_nr_jobs ( ) const [inline]

Friends And Related Function Documentation

friend class JobDistributorFactory [friend]

The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines