| java.lang.Object
 org.griphyn.cPlanner.transfer.implementation.Abstract
org.griphyn.cPlanner.transfer.implementation.AbstractMultipleFTPerXFERJob
org.griphyn.cPlanner.transfer.implementation.CRFT
| CRFT | | public class CRFT extends AbstractMultipleFTPerXFERJob (Code) | | The implementation that creates transfer jobs referring to the c rft client
executable distributed with the VDS.
The rft client is invoked on the submit host. Hence there should be an
entry in the transformation catalog for logical transformation
CRFT at site local. The transformation should point
to the client that is distributed with RFT in GT4.
The user can tweak the options to the globus-crft client by specifying the properties
in the properties files with the prefix pegasus.transfer.crft.
The following table lists all the properties with their prefixes stripped off,
that the user can specify in the properties files. The default value is used
if the user does not specify a particular property. If a value is not specified,
the particular option is not generated.
For the properties which have a default value of no default and the user not
providing a value for the property, the option is not propogated further to the
client underneath. In that case, it is upto the client to construct the
appropriate value for that property/option.
| property | default value | description |
|---|
| endpoint |
no default (required option) |
The endpoint to contact when creating a service. |
|---|
| concurrent |
no default |
The number of simultaneous transfers. |
|---|
| parallel |
no default |
The number of parallel sockets to use with each transfer. |
|---|
| tcp-bs |
no default |
specifies the size (in bytes) of the TCP buffer to be used by the
underlying ftp data channels |
|---|
| verbose |
true |
to generate more verbose output, helpful for debugging. |
|---|
It leads to the creation of the setup chmod jobs to the workflow, that appear
as parents to compute jobs in case the transfer implementation does not
preserve the X bit on the file being transferred. This is required for
staging of executables as part of the workflow. The setup jobs are only added
as children to the stage in jobs.
In order to use the transfer implementation implemented by this class,
- the property pegasus.transfer.*.impl must be set to value CRFT.
There should be an entry in the transformation catalog with the fully qualified
name as globus::crft for all the sites where workflow is run,
or on the local site in case of third party transfers.
author:
Karan Vahi
version:
$Revision: 145 $ |
| Method Summary | |
| public boolean | addSetXBitJobs(String computeJobName, String txJobName, Collection execFiles, int transferClass)
Adds the dirmanager job to the workflow, that do a chmod on the executable
files that are being staged. | | public TransferJob | createTransferJob(SubInfo job, Collection files, Collection execFiles, String txJobName, int jobClass)
Calls out to the super class method to create the main structure of the job.
In addition, for the CRFT adds the specific condor magic that allows for
the transfer of the input file correctly to working directory.
Parameters:
job - the SubInfo object for the job, in relation to whichthe transfer node is being added. | | public boolean | doesPreserveXBit()
Returns a boolean indicating whether the transfer protocol being used by
the implementation preserves the X Bit or not while staging. | | protected String | generateArgumentString(TransferJob job)
Constructs the arguments to the transfer executable that need to be
passed to the executable referred to in this transfer mode. | | protected String | getCompleteTCName()
Returns the complete name for the transformation. | | protected String | getDerivationName()
Returns the logical name of the derivation that this implementation
refers to. | | protected String | getDerivationNamespace()
Returns the namespace of the derivation that this implementation
refers to. | | protected String | getDerivationVersion()
Returns the version of the derivation that this implementation
refers to. | | public String | getDescription()
Returns a textual description of the transfer implementation. | | protected List | getEnvironmentVariables(String site)
Returns the environment profiles that are required for the default
entry to sensibly work. | | public TransformationCatalogEntry | getTransformationCatalogEntry(String siteHandle)
Retrieves the transformation catalog entry for the executable that is
being used to transfer the files in the implementation.
Parameters:
siteHandle - the handle of the site where the transformation isto be searched. | | public boolean | useThirdPartyTransferAlways()
Return a boolean indicating whether the transfers to be done always in
a third party transfer mode. | | protected void | writeJumboStdIn(FileWriter writer, Collection files)
Writes to a file on the submit host, that is passed to the rft-client
as input. |
| CONCURRENT_KEY | | final public static String CONCURRENT_KEY(Code) | | The key name that denotes the number of files to be transferred at any
given time.
|
| CREATE_KEY | | final public static String CREATE_KEY(Code) | | The key name that denotes to create a RFT service.
|
| DERIVATION_NAME | | final public static String DERIVATION_NAME(Code) | | The name of the underlying derivation.
|
| DERIVATION_NAMESPACE | | final public static String DERIVATION_NAMESPACE(Code) | | The derivation namespace for for the transfer job.
|
| DERIVATION_VERSION | | final public static String DERIVATION_VERSION(Code) | | The derivation version number for the transfer job.
|
| DESCRIPTION | | final public static String DESCRIPTION(Code) | | A short description of the transfer implementation.
|
| END_POINT_KEY | | final public static String END_POINT_KEY(Code) | | The key name that denotes the endpoint to contact when creating a
service.
|
| MONITOR_KEY | | final public static String MONITOR_KEY(Code) | | The key name that denotes to monitor the request. Makes the client block.
|
| PARALLEL_KEY | | final public static String PARALLEL_KEY(Code) | | The key name that denotes the number of parallel sockets to use for each
transfer.
|
| PROPERTIES_PREFIX | | final public static String PROPERTIES_PREFIX(Code) | | The prefix for all the properties this mode requires.
|
| SUBMIT_KEY | | final public static String SUBMIT_KEY(Code) | | The key name that denotes to start a RFT service.
|
| TCP_BUFFER_SIZE_KEY | | final public static String TCP_BUFFER_SIZE_KEY(Code) | | The key name that denotes the TCP buffer size in bytes.
|
| TRANSFER_FILE_KEY | | final public static String TRANSFER_FILE_KEY(Code) | | The key name that points to the transfer file that is containing the
source and destination urls.
|
| TRANSFORMATION_NAME | | final public static String TRANSFORMATION_NAME(Code) | | The name of the underlying transformation that is queried for in the
Transformation Catalog.
|
| TRANSFORMATION_NAMESPACE | | final public static String TRANSFORMATION_NAMESPACE(Code) | | The transformation namespace for the transfer job.
|
| TRANSFORMATION_VERSION | | final public static String TRANSFORMATION_VERSION(Code) | | The version number for the transfer job.
|
| VERBOSE_KEY | | final public static String VERBOSE_KEY(Code) | | The key name that denotes whether to do verbose or not.
|
| CRFT | | public CRFT(PegasusProperties properties, PlannerOptions options)(Code) | | The overloaded constructor, that is called by the Factory to load the
class.
Parameters:
properties - the properties object.
Parameters:
options - the options passed to the Planner. |
| addSetXBitJobs | | public boolean addSetXBitJobs(String computeJobName, String txJobName, Collection execFiles, int transferClass)(Code) | | Adds the dirmanager job to the workflow, that do a chmod on the executable
files that are being staged. It is empty as RFT preserves X bit permission
while staging files.
Parameters:
computeJobName - the name pf the computeJob for which the files arebeing staged.
Parameters:
txJobName - the name of the transfer job that is staging the files.
Parameters:
execFiles - the executable files that are being staged.
Parameters:
transferClass - the class of transfer job boolean indicating whether any XBitJobs were succesfully added ornot. |
| createTransferJob | | public TransferJob createTransferJob(SubInfo job, Collection files, Collection execFiles, String txJobName, int jobClass)(Code) | | Calls out to the super class method to create the main structure of the job.
In addition, for the CRFT adds the specific condor magic that allows for
the transfer of the input file correctly to working directory.
Parameters:
job - the SubInfo object for the job, in relation to whichthe transfer node is being added. Either the transfernode can be transferring this jobs input files tothe execution pool, or transferring this job's outputfiles to the output pool.
Parameters:
files - collection of FileTransfer objectsrepresenting the data files and staged executables to betransferred.
Parameters:
execFiles - subset collection of the files parameter, that identifiesthe executable files that are being transferred.
Parameters:
txJobName - the name of transfer node.
Parameters:
jobClass - the job Class for the newly added job. Can be one of thefollowing:stage-instage-outinter-pool transfer the created TransferJob. |
| doesPreserveXBit | | public boolean doesPreserveXBit()(Code) | | Returns a boolean indicating whether the transfer protocol being used by
the implementation preserves the X Bit or not while staging.
boolean |
| generateArgumentString | | protected String generateArgumentString(TransferJob job)(Code) | | Constructs the arguments to the transfer executable that need to be
passed to the executable referred to in this transfer mode. Since the
rft client is run on the submit host, the path to the input file
to the rft client is given, instead of passing it through condor
files.
In addition , it SETS THE STDIN of the transfer job to null, as the
input file is not being sent to the remote sides. There should be a
generic function prepareIPFile to do this.
Parameters:
job - the object containing the transfer node. the argument string |
| getCompleteTCName | | protected String getCompleteTCName()(Code) | | Returns the complete name for the transformation.
the complete name. |
| getDerivationName | | protected String getDerivationName()(Code) | | Returns the logical name of the derivation that this implementation
refers to.
the name of the derivation. |
| getDerivationNamespace | | protected String getDerivationNamespace()(Code) | | Returns the namespace of the derivation that this implementation
refers to.
the namespace of the derivation. |
| getDerivationVersion | | protected String getDerivationVersion()(Code) | | Returns the version of the derivation that this implementation
refers to.
the version of the derivation. |
| getDescription | | public String getDescription()(Code) | | Returns a textual description of the transfer implementation.
a short textual description |
| getEnvironmentVariables | | protected List getEnvironmentVariables(String site)(Code) | | Returns the environment profiles that are required for the default
entry to sensibly work. Returns an empty list.
Parameters:
site - the site where the job is going to run. List of environment variables, else null in case where therequired environment variables could not be found. |
| getTransformationCatalogEntry | | public TransformationCatalogEntry getTransformationCatalogEntry(String siteHandle)(Code) | | Retrieves the transformation catalog entry for the executable that is
being used to transfer the files in the implementation.
Parameters:
siteHandle - the handle of the site where the transformation isto be searched. the transformation catalog entry if found, else null. |
| useThirdPartyTransferAlways | | public boolean useThirdPartyTransferAlways()(Code) | | Return a boolean indicating whether the transfers to be done always in
a third party transfer mode. This always returns true, indicating
transfers can only be done in a third party transfer mode.
A value of false does not preclude third party transfers. They still can
be done, by setting the property "pegasus.transfer.*.thirdparty.sites".
false |
| writeJumboStdIn | | protected void writeJumboStdIn(FileWriter writer, Collection files) throws Exception(Code) | | Writes to a file on the submit host, that is passed to the rft-client
as input. The rft-client is always run on the submit host, and hence
can access the file.
Parameters:
writer - the writer to the stdin file.
Parameters:
files - Collection of FileTransfer objects containingthe information about sourceam fin and destURL's.
throws:
Exception - |
| Methods inherited from org.griphyn.cPlanner.transfer.implementation.Abstract | protected boolean addSetXBitJobs(SubInfo computeJob, SubInfo txJob, Collection execFiles)(Code)(Java Doc)
public boolean addSetXBitJobs(SubInfo computeJob, String txJobName, Collection execFiles, int transferClass)(Code)(Java Doc)
public boolean addSetXBitJobs(SubInfo computeJob, String txJobName, Collection execFiles, int transferClass, int xbitIndex)(Code)(Java Doc)
public void applyPriority(TransferJob job)(Code)(Java Doc)
public boolean checkAndTransferProxy(TransferJob job)(Code)(Java Doc)
protected void construct(SubInfo job, String key, String value)(Code)(Java Doc)
public SubInfo createNoOPJob(String name)(Code)(Java Doc)
protected SubInfo createSetXBitJob(FileTransfer file, String name)(Code)(Java Doc)
protected Set determineDisabledChmodSites(String sites)(Code)(Java Doc)
protected boolean disableChmodJobCreation(String site)(Code)(Java Doc)
public String getNOOPJobName(String name, int counter)(Code)(Java Doc)
protected String getPathToUserProxy()(Code)(Java Doc)
protected String getPriority(TransferJob job)(Code)(Java Doc)
public String getSetXBitJobName(String name, int counter)(Code)(Java Doc)
public void setRefiner(Refiner refiner)(Code)(Java Doc)
|
|
|