Publication

DataLight: data transfer and logging of large output applications in Grid environments

Bibliographic Details
Summary:	In this paper we present a powerful and lightweight library called DataLight that handles the transfer of large data file sets from the local storage of the Worker Node to an gLite Storage Element. The library is very easy to integrate into non-Grid/legacy code and it is flexible enough to allow for overlapping of several data transfer and computation. It has the ability to send messages to a Web Portal while the application is running, so that a user can login to the Portal and receive immediate updates. In particular the user can start post-processing the produced data while the application is still running. We present the application details both of the library and Portal and discuss some relevant security issues.
Main Authors:	Abreu, P.
Other Authors:	Fonseca, R.; Silva, L. O.
Subject:	Web portal Large data sets Grid migration Data transfer
Year:	2011
Country:	Portugal
Document type:	conference output
Access type:	open access
Associated institution:	ISCTE
Language:	English
Origin:	Repositório ISCTE

Description
Summary:	In this paper we present a powerful and lightweight library called DataLight that handles the transfer of large data file sets from the local storage of the Worker Node to an gLite Storage Element. The library is very easy to integrate into non-Grid/legacy code and it is flexible enough to allow for overlapping of several data transfer and computation. It has the ability to send messages to a Web Portal while the application is running, so that a user can login to the Portal and receive immediate updates. In particular the user can start post-processing the produced data while the application is still running. We present the application details both of the library and Portal and discuss some relevant security issues.