| org.griphyn.cPlanner.transfer.MultipleFTPerXFERJobRefiner
 org.griphyn.cPlanner.transfer.refiner.Default
All known Subclasses: org.griphyn.cPlanner.transfer.refiner.Bundle, org.griphyn.cPlanner.transfer.refiner.Chain,
| Default | | public class Default extends MultipleFTPerXFERJobRefiner (Code) | | The default transfer refiner, that implements the multiple refiner.
For each compute job if required it creates the following
- a single stagein transfer job
- a single stageout transfer job
- a single interpool transfer job
In addition this implementation prevents file clobbering while staging in data
to a remote site, that is shared amongst jobs.
author:
Karan Vahi
version:
$Revision: 258 $ |
| Field Summary | |
| final public static String | DESCRIPTION
A short description of the transfer refinement. | | protected Map | mFileTable
A Map containing information about which logical file has been
transferred to which site and the name of the stagein transfer node
that is transferring the file from the location returned from
the replica catalog. | | protected String | mLogMsg
| | protected PPS | mPPS
The handle to the provenance store implementation. |
| Method Summary | |
| public void | addInterSiteTXNodes(SubInfo job, Collection files)
Adds the inter pool transfer nodes that are required for transferring
the output files of the parents to the jobs execution site. | | public void | addJob(SubInfo job)
Add a new job to the workflow being refined. | | public void | addRelation(String parent, String child)
Adds a new relation to the workflow being refiner. | | public void | addRelation(String parent, String child, String site, boolean parentNew)
Adds a new relation to the workflow. | | public void | addStageInXFERNodes(SubInfo job, Collection files)
Adds the stage in transfer nodes which transfer the input files for a job,
from the location returned from the replica catalog to the job's execution
pool. | | public void | addStageOutXFERNodes(SubInfo job, Collection files, ReplicaCatalogBridge rcb)
Adds the stageout transfer nodes, that stage data to an output site
specified by the user.
Parameters:
job - SubInfo object corresponding to the node towhich the files are to be transferred to.
Parameters:
files - Collection of FileTransfer objects containing theinformation about source and destURL's.
Parameters:
rcb - bridge to the Replica Catalog. | | public void | addStageOutXFERNodes(SubInfo job, Collection files, ReplicaCatalogBridge rcb, boolean deletedLeaf)
Adds the stageout transfer nodes, that stage data to an output site
specified by the user.
Parameters:
job - SubInfo object corresponding to the node towhich the files are to be transferred to.
Parameters:
files - Collection of FileTransfer objects containing theinformation about source and destURL's.
Parameters:
rcb - bridge to the Replica Catalog. | | protected void | appendAttribute(StringBuffer xmlFeed, String key, String value)
Appends an xml attribute to the xml feed. | | protected String | constructFileKey(String lfn, String siteHandle)
Constructs the key for an entry to the file table. | | protected SubInfo | createRegistrationJob(String regJobName, SubInfo job, Collection files, ReplicaCatalogBridge rcb)
Creates the registration jobs, which registers the materialized files on
the output site in the Replica Catalog.
Parameters:
regJobName - The name of the job which registers the files in theReplica Mechanism.
Parameters:
job - The job whose output files are to be registered in theReplica Mechanism.
Parameters:
files - Collection of FileTransfer objects containingthe information about source and destURL's.
Parameters:
rcb - bridge to the Replica Catalog. | | public void | done()
Signals that the traversal of the workflow is done. | | public String | getDescription()
Returns a textual description of the transfer mode. | | protected void | logRefinerAction(SubInfo computeJob, SubInfo txJob, Collection files, String type)
Records the refiner action into the Provenace Store as a XML fragment. |
| DESCRIPTION | | final public static String DESCRIPTION(Code) | | A short description of the transfer refinement.
|
| mFileTable | | protected Map mFileTable(Code) | | A Map containing information about which logical file has been
transferred to which site and the name of the stagein transfer node
that is transferring the file from the location returned from
the replica catalog.
The key for the hashmap is logicalfilename:sitehandle and the value would be
the name of the transfer node.
|
| mLogMsg | | protected String mLogMsg(Code) | | The string holding the logging messages
|
| mPPS | | protected PPS mPPS(Code) | | The handle to the provenance store implementation.
|
| Default | | public Default(ADag dag, PegasusProperties properties, PlannerOptions options)(Code) | | The overloaded constructor.
Parameters:
dag - the workflow to which transfer nodes need to be added.
Parameters:
properties - the PegasusProperties object containing allthe properties required by Pegasus.
Parameters:
options - the options passed to the planner. |
| addInterSiteTXNodes | | public void addInterSiteTXNodes(SubInfo job, Collection files)(Code) | | Adds the inter pool transfer nodes that are required for transferring
the output files of the parents to the jobs execution site.
Parameters:
job - SubInfo object corresponding to the node towhich the files are to be transferred to.
Parameters:
files - Collection of FileTransfer objects containing theinformation about source and destURL's. |
| addJob | | public void addJob(SubInfo job)(Code) | | Add a new job to the workflow being refined.
Parameters:
job - the job to be added. |
| addRelation | | public void addRelation(String parent, String child)(Code) | | Adds a new relation to the workflow being refiner.
Parameters:
parent - the jobname of the parent node of the edge.
Parameters:
child - the jobname of the child node of the edge. |
| addRelation | | public void addRelation(String parent, String child, String site, boolean parentNew)(Code) | | Adds a new relation to the workflow. In the case when the parent is a
transfer job that is added, the parentNew should be set only the first
time a relation is added. For subsequent compute jobs that maybe
dependant on this, it needs to be set to false.
Parameters:
parent - the jobname of the parent node of the edge.
Parameters:
child - the jobname of the child node of the edge.
Parameters:
site - the execution pool where the transfer node is to be run.
Parameters:
parentNew - the parent node being added, is the new transfer joband is being called for the first time. |
| addStageInXFERNodes | | public void addStageInXFERNodes(SubInfo job, Collection files)(Code) | | Adds the stage in transfer nodes which transfer the input files for a job,
from the location returned from the replica catalog to the job's execution
pool.
Parameters:
job - SubInfo object corresponding to the node towhich the files are to be transferred to.
Parameters:
files - Collection of FileTransfer objects containing theinformation about source and destURL's. |
| addStageOutXFERNodes | | public void addStageOutXFERNodes(SubInfo job, Collection files, ReplicaCatalogBridge rcb)(Code) | | Adds the stageout transfer nodes, that stage data to an output site
specified by the user.
Parameters:
job - SubInfo object corresponding to the node towhich the files are to be transferred to.
Parameters:
files - Collection of FileTransfer objects containing theinformation about source and destURL's.
Parameters:
rcb - bridge to the Replica Catalog. Used for creating registrationnodes in the workflow. |
| addStageOutXFERNodes | | public void addStageOutXFERNodes(SubInfo job, Collection files, ReplicaCatalogBridge rcb, boolean deletedLeaf)(Code) | | Adds the stageout transfer nodes, that stage data to an output site
specified by the user.
Parameters:
job - SubInfo object corresponding to the node towhich the files are to be transferred to.
Parameters:
files - Collection of FileTransfer objects containing theinformation about source and destURL's.
Parameters:
rcb - bridge to the Replica Catalog. Used for creating registrationnodes in the workflow.
Parameters:
deletedLeaf - to specify whether the node is being added fora deleted node by the reduction engine or not.default: false |
| appendAttribute | | protected void appendAttribute(StringBuffer xmlFeed, String key, String value)(Code) | | Appends an xml attribute to the xml feed.
Parameters:
xmlFeed - the xmlFeed to which xml is being written
Parameters:
key - the attribute key
Parameters:
value - the attribute value |
| constructFileKey | | protected String constructFileKey(String lfn, String siteHandle)(Code) | | Constructs the key for an entry to the file table. The key returned
is lfn:siteHandle
Parameters:
lfn - the logical filename of the file that has to betransferred.
Parameters:
siteHandle - the name of the site to which the file is beingtransferred. the key for the entry to be made in the filetable. |
| createRegistrationJob | | protected SubInfo createRegistrationJob(String regJobName, SubInfo job, Collection files, ReplicaCatalogBridge rcb)(Code) | | Creates the registration jobs, which registers the materialized files on
the output site in the Replica Catalog.
Parameters:
regJobName - The name of the job which registers the files in theReplica Mechanism.
Parameters:
job - The job whose output files are to be registered in theReplica Mechanism.
Parameters:
files - Collection of FileTransfer objects containingthe information about source and destURL's.
Parameters:
rcb - bridge to the Replica Catalog. Used for creating registrationnodes in the workflow. the registration job. |
| done | | public void done()(Code) | | Signals that the traversal of the workflow is done. It signals to the
Provenace Store, that refinement is complete.
|
| getDescription | | public String getDescription()(Code) | | Returns a textual description of the transfer mode.
a short textual description |
| logRefinerAction | | protected void logRefinerAction(SubInfo computeJob, SubInfo txJob, Collection files, String type)(Code) | | Records the refiner action into the Provenace Store as a XML fragment.
Parameters:
computeJob - the compute job.
Parameters:
txJob - the associated transfer job.
Parameters:
files - list of FileTransfer objects containing file transfers.
Parameters:
type - the type of transfer job |
|
|