The GRID Data Transfer Service is called File Transfer Service (FTS) is a data movement service. FTS aims to reliably copy one Storage URL to another. It uses a 3rd party copy (e.g. gsiftp) to achieve this, but will retry if this fails. It also schedules these copys along network channels to ensure that bandwidth is properly used.
State in the FTS is held in a database, which ensures that the service can be restarted reliably.
FTS is used by the experiment frameworks (typically the end-users do not interact directly with it).
Experiment frameworks submit jobs to the FTS; a job is a set of pairs with source and destination file names.
Scheme of FTS service operation:
The development of research in high-energy physics, astrophysics, and biology requires the joint work of many organizations to process a large amount of data in a relatively short time.
This requires geographically distributed computing systems capable of transmitting and receiving data of the order of hundreds of terabytes per day, simultaneously processing hundreds of thousands of tasks and storing hundreds of petabytes of data for a long time.
The data management subsystem includes three services that support file access:
- resource of data storage (StorageElement, SE),
- service of catalogs (Catalog Services, CS,
- Data transfer scheduler (DataScheduler, DS).
To organize data storage in a grid environment, various
Systems and services The most common are the following:
Castor, dCache, DPM (Data Pool Management)
For interaction with which a special service is developed
SRM (Storage Resource Manager)
For the transfer of data at the physical level, the GridFTP protocol developed in the Globus project
To access files
DCache uses its own protocols (for example, DCAP), gridFTP.
To ensure the necessary reliability, performance and organization of interaction between the other data management services, a data transmission service was created:
FTS (File Transfer Service), whose main duties are:
- Providing reliable and convenient file transfer mechanisms of type
Point-to-point, monitoring and monitoring of programs, - Distribution of site resources between different organizations,
- Management of user requests.
Every day, with the help of FTS, hundreds of thousands of files are transferred between different sites, and the volumes are hundreds of terabytes per day.
FTS is designed to reliably transfer files between large data stores, primarily between centers of Tier0 and Tier1.