![]() |
Rosetta Protocols
2014.35
|
The JD2ResourceManager is the ResourceManager that should be used when running protocols under the jd2 JobDistributor. More...
#include <JD2ResourceManager.hh>
Static Public Member Functions | |
static JD2ResourceManager * | get_jd2_resource_manager_instance () |
![]() | |
static ResourceManager * | get_instance () |
Protected Member Functions | |
JD2ResourceManager () | |
![]() | |
LazyResourceManager () | |
ResourceManager () | |
virtual void | add_resource (ResourceTag const &resource_tag, ResourceOP resource) |
virtual void | free_resource (ResourceTag const &resource_tag) |
Private Attributes | |
std::map< std::string, std::string > | fallback_resource_descriptions_created_ |
Friends | |
class | JD2ResourceManagerCreator |
The JD2ResourceManager is the ResourceManager that should be used when running protocols under the jd2 JobDistributor.
The purpose of the ResourceManager is to disentangle the process of feeding protocols with Resources they need from the job distribution system in which they run. When protocols request a resource under the ResourceManager, they do so without knowing what kind of ResourceManager they are communicating with. Protocols that are designed to run under JD2 could conceivably be run under any other job management scheme (e.g. a complicated MPI protocol). It's the job of the JD2ResourceManager to determine the right resource to deliver to a protocol, and the context that the JD2ResourceManager uses to make that decision is the job. E.g., when a protocol asks for the "native" (the Pose of the native structure), the JD2ResourceManager looks up what job is currently running and then delivers the (single) native Pose for that job.
The JD2ResourceManager is meant to work with the JD2ResourceManagerJobInputter. Jobs are defined in an input XML file and for each of the jobs that are defined, resources can be mapped to them. Resources are also declared in an XML file and may be declared in the same XML file as the Jobs or in a different XML file. The format of this XML file is specified in full between the documentation in the JD2ResourceManagerJobInputter class and three functions that are documented and implemented in this class.
The JD2ResourceManager keeps track of which resources are used by which jobs and keeps track of what jobs have completed. It allocates a Resource the first time that resource is requested by a protoocol, and it deallocates that Resource when all jobs that require that resource have finished. Thus a protocol that needs an expensive-to-create resource (e.g. a set of 9-mers) can load that resource once and then have that resource shared between all of the jobs that require that resource (e.g. 10K abinitio trajectories) without having to do any legwork itself. If you have 100K abinitio trajectories for 10 different targets, they could all be run in a single job. (Note: as of 9/2013, abinitio does not work with the jd2 or the ResourceManager but it should!)
The main consequence of relying on the ResourceManager to hand resources to jobs is that jobs which require different resources (usually specified on the command line, and therefore requiring that each job run in a separate process) now can be run together in a single process. For MPI jobs, this is a major improvement, especially in constructing benchmarks, where many separate, (possibly short) jobs need to be run.
(Because Resources are not constructed until they are requested, resources can be declared and never used; you can give a resource file that describes every resource you use in every job, and a job declaration file that only uses one of those resources.)
|
protected |
|
virtual |
|
private |
Check if the loader type is defined with the ResourceLoaderFactory
References basic::resource_manager::ResourceLoaderFactory::available_resource_loaders(), and basic::resource_manager::ResourceLoaderFactory::get_instance().
Referenced by read_resource_table_tag(), and read_resources_tags().
|
virtual |
Reimplemented from basic::resource_manager::LazyResourceManager.
References basic::resource_manager::LazyResourceManager::clear().
|
private |
|
static |
Referenced by protocols::jd2::JD2ResourceManagerJobInputter::check_each_job_has_startstruct(), protocols::jd2::JD2ResourceManagerJobInputter::cleanup_after_job_completion(), protocols::jd2::JD2ResourceManagerJobInputter::fill_jobs_from_stream(), protocols::jd2::JD2ResourceManagerJobInputter::parse_job_tag(), protocols::jd2::JD2ResourceManagerJobInputter::pose_from_job(), protocols::jd2::JD2ResourceManagerJobInputter::read_Data_for_subtag(), and protocols::jd2::JD2ResourceManagerJobInputter::record_job().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), protocols::jd2::Job::input_tag(), and basic::options::option.
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References basic::resource_manager::LazyResourceManager::add_resource(), basic::resource_manager::LazyResourceManager::add_resource_configuration(), create_resource_from_fallback(), protocols::jd2::JobDistributor::current_job(), fallback_resource_descriptions_created_, basic::resource_manager::LazyResourceManager::find_resource(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_resource_by_job_tag(), basic::resource_manager::LazyResourceManager::has_resource_configuration(), basic::resource_manager::LazyResourceManager::has_resource_tag_by_job_tag(), protocols::jd2::Job::input_tag(), numeric::random::random_range(), basic::resource_manager::ResourceConfiguration::resource_tag, and utility::to_string().
|
virtual |
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
Implements basic::resource_manager::LazyResourceManager.
References protocols::nonlocal::current_job(), protocols::jd2::JobDistributor::get_instance(), basic::resource_manager::LazyResourceManager::get_job_options(), basic::resource_manager::LazyResourceManager::has_job_options(), and protocols::jd2::Job::input_tag().
|
virtual |
|
private |
find the LocatorTag item from the resource tag. Based on the LocatorTag fill the LocatorID. locator item (&string): FileSystemResourceLocator (default)
Referenced by read_resources_tags().
*void protocols::jd2::JD2ResourceManager::read_resource_locators_tags | ( | utility::tag::TagCOP | tags | ) |
Read the portion of an XML file that declares ResourceLocator objects.
instantiate all the resource locators given in the input tags, and put them into the base class. Make sure no two resource locators share a common name.
The file format should be
/* (<--- please ignore this; it is merely an artifact of writing doxygen) The surrounding tag "ResourceLocators" will not be read by this function but instead wil be read the by function that will call this function. It is listed here simply for context. <ResourceLocators> declare one or more resource locators <LocatorType tag=name_for_this_instance > * where LocatorType is one of the several locator_types that have been implemented; the string must match the locator_type given by some ResourceLocatorCreator. E.g. "DatabaseResourceLocator". And, * where the "tag" (the name) for the declared instance must be unique among the set of all resource locators declared. <...\> provide any subtags that are required to instantiate a ResourceLocator of the indicated type <\LocatorType> <\ResourceLocators> /
Referenced by protocols::jd2::JD2ResourceManagerJobInputter::fill_jobs_from_stream().
|
private |
Referenced by read_resource_options_table_tag(), and read_resource_options_tags().
|
private |
References check_statement_sanity(), key, parse_database_connection(), read_resource_option_item(), safely_prepare_statement(), safely_read_from_database(), t, tags, protocols::jd2::TR, and type.
Referenced by read_resource_options_tags().
|
private |
Referenced by read_resources_tags().
*void protocols::jd2::JD2ResourceManager::read_resource_options_tags | ( | utility::tag::TagCOP | tags | ) |
For an example of a ResourceOptions object, see ImportPoseOptions.
instantiate all the resource options and put them in the base class. Make sure no two resource options are given the same name.
The file format should be
/* (<--- please ignore this; it is merely an artifact of writing doxygen) The surrounding tag "ResourceOptions" will not be read by this function but instead wil be read the by function that will call this function. It is listed here simply for context. <ResourceOptions> declare one or more resource options objects <OptionsType tag=name_for_this_instance > * where OptionsType is one of the several option_types that have been implemented; the string must match the option_type given by some ResourceOptionsCreator. E.g. "ImportPoseOptions". And, * where the "tag" (the name) for the declared instance must be unique among the set of all resource options declared. <...\> provide any subtags that are required to instantiate a ResourceOptions object of the indicated type <\OptionsType> or provide an SQL query to retrieve the ResourceOptions defined in a database <OptionsTable sql_command=command database_resource=dbresource/> * where command is a string representing an SQL query, and * where dbresource is the name for a previously-declared database-connection resource Each row of the resource options table should have the following format resource_options_tag, resource_options_type, resource_option_key, resource_option_value * The table should 'ORDER BY resource_options_tag', to have data for a specific resource-options object adjacent in the table * resource_option_{key, value}: Each resource options takes set of key value pairs (string -> string) for example, for PoseFromPDBOptions, has the key 'exit_if_missing_heavy_atoms' and takes '1' for true or '0' for false.; <\ResourceOptions>
References utility::tag::Tag::getName(), read_resource_option_item(), and read_resource_options_table_tag().
Referenced by protocols::jd2::JD2ResourceManagerJobInputter::fill_jobs_from_stream().
|
private |
References check_resource_loader_type(), check_statement_sanity(), basic::resource_manager::ResourceConfiguration::loader_type, basic::resource_manager::ResourceConfiguration::locator_id, basic::resource_manager::ResourceConfiguration::locator_tag, parse_database_connection(), basic::resource_manager::ResourceConfiguration::resource_options_tag, basic::resource_manager::ResourceConfiguration::resource_tag, safely_prepare_statement(), safely_read_from_database(), and protocols::jd2::TR.
Referenced by read_resources_tags().
|
private |
make sure the resource object has been given a tag and that no other resource object has been delecared with the same name.
Referenced by read_resources_tags().
*void protocols::jd2::JD2ResourceManager::read_resources_tags | ( | utility::tag::TagCOP | tags | ) |
read through all the resources, and put them into the base class for later instantiation. Make sure each resource is named, that it is the only resource that has been declared with that name.
The file format should be
/* (<--- please ignore this; it is merely an artifact of writing doxygen) The surrounding tag "Resources" will not be read by this function but instead wil be read the by function that will call this function. It is listed here simply for context. <Resources> declare one or more resource objects <ResourceLoaderType tag=name_for_this_instance locator=locatorname locatorID=locid options=optname \> * where ResourceLoaderType is one of the several loader_types that have been implemented; the string must match the loader_type given by some ResourceLoaderCreator. E.g. "LoopsFile" which is given by the LoopsFileLoaderCreator. And, * where the (required) "name_for_this_instance" (the tag) for the declared instance must be unique among the set of all resources declared, * where the (optional) "locatorname" must refer to a previously-declared resource locator, if a locator is not given, the FileSystemResourceLocator is used * where the (required) "locid" provides the locatorID that the resource locator uses to create the input stream needed to read in the resource (e.g. the file name), and * where the (optional) "optname" string refers to a previously-declared ResourceOptions object that will be used by the indicated ResourceLoader (if not given, then the ResourceLoader will use the default ResourceOptions object). <\ResourceLoaderType> or <ResourceLoaderType tag=name_for_this_instance file=filename locator=locatorname options=optname > * where ResourceLoaderType is the same as above, * where the (required) "name_for_this_instance" is the same as above, * where the (optional) "locatorname" must refer to a previously defined FileSystemResourceLocator * where the (required) "filename" is the path to the file used to create the resource * where the (optional) "optsname" is the same as above <\ResourceLoaderType> or provide an SQL query to retrieve the Resources defined in a database <ResourceTable sql_command=command database_resource=dbresource/> * where command is a string representing an SQL query, and * where dbresource is the name for a previously-declared database-connection resource The ResourceTable tag requires a 'sql_command' tag that is an SQL SELECT statement that returns the following columns: resource_tag locator_tag locator_id loader_type resource_options_tag <\Resources>
References check_resource_loader_type(), basic::resource_manager::ResourceConfiguration::loader_type, basic::resource_manager::ResourceConfiguration::locator_id, basic::resource_manager::ResourceConfiguration::locator_tag, read_resource_locator_items(), read_resource_options_tag_item(), read_resource_table_tag(), read_resource_tag_item(), basic::resource_manager::ResourceConfiguration::resource_options_tag, and basic::resource_manager::ResourceConfiguration::resource_tag.
Referenced by protocols::jd2::JD2ResourceManagerJobInputter::fill_jobs_from_stream().
|
friend |
|
private |
Referenced by get_resource(), and has_resource_with_description().