Globus components ----------------- * GRAM - job management (like condor_schedd, but it is not really resource scheduler, it is rather a broker to other schedulers, like PBS, Condor, LSF etc.), uses GridFTP for staging in/out * GridFTP - client/server file transfer, necessitates client to have an opened socket vs. its server (gsiftp-based) * RFT - web service file transfer (SOAP-based) * MDS4 - information aggregator (like condor_collector) * WebMDS - web tool to view indexed services * RLS - replica location web service, distributed registry of all grid resources (non-SOAP) * DRS - data replication web service, built on top of RLS * CAS - security service for files and directories, uses GSS authentication, GridFTP can be configured to use CAS to securely transfer files Installation ------------ * In order to install RLS service, during Globus configuration one must either set the environment variable GLOBUS_IODBC_PATH or to specify the following flag '--with-iodbc=$GLOBUS_LOCATION' * In order to install DRS one must not only specify the '--with-iodbc' flag, but also '--enable-drs' one Software requirements --------------------- * JDK 1.4 and higher * Ant - tool for building Java code (similar to make) Web services development ------------------------ * org.apache.axis.wsdl.Java2WSDL - to turn Java interfaces into WSDLs % java org.apache.axis.wsdl.Java2WSDL -o wp.wsdl -l"http://localhost:8080/axis/services/WidgetPrice" -n "urn:Example6" -p"samples.userguide.example6" "urn:Example6" samples.userguide.example6.WidgetPrice * org.apache.axis.wsdl.WSDL2Java - to turn WSDL to Java implementation classes and to make bindings between namespaces and packages % java org.apache.axis.wsdl.WSDL2Java -o . -d Session -s -S true -Nurn:Example6 samples.userguide.example6 wp.wsdl * org.apache.axis.client.AdminClient - client to administer SOAP server, deploy/undeploy services % java org.apache.axis.client.AdminClient -p 8443 widget/deploy.wsdd * log4j - implementation of Jakarta Commons Logging APIs for structured logging * ReflectionResourceProperty - this class has a limitation of creating getter/setter for such a property * Deploying WS - by means of 'ant deploy' or using 'globus-build-service.sh' script that creates a gar and then deploying it using 'globus-deploy-gar' * Implementation service-related files must be placed inside 'impl' directories and must match to the value attribute of service tag in deploy-server.wsdd. Besides, the wsdlFile attribute must end with '_service.wsdl' - this is the file that includes the WSDL bindings: % share/schema/ha/HA_service.wsdl ... * To run a client of some service, one must ensure to specify the 'build/stubs/classes' as part of the classpath while compiling * To make the developed service part of the container throughout its life, one must specify the following in deploy-server.wsdd: % ... * The ReflectionResourceProperty use requires including in Java classes getters/setters with names that were given to the resource properties in the WSDL file (but with the first letter in lowercase) Web services specifications (WSRF and related) ---------------------------------------------- * Resource - GT4 bean to store the state * ResourceKey - id, uniquely defining the resource in the container * ResourceHome - manage resources, add them and remove them * EPR - id, uniquely defining the resource in the network * Operation provider - provide ability to destroy resources * ResourceProperty - view on the resource state, appear as XML attributes of the resource * ResourceLifetime - supplies mechanisms to manage the lifecycle of resources * ServiceGroup - determines how we should go about grouping services or WS-Resources together, register sources for performing specific actions afterwards (such as MD4) * BaseFaults - standartized way to represent faults in WS * Notification - notifies clients when something occurs in the WS * Addressing - mechanism to address WS which are more versatile than a plain URI Globus utilities ---------------- * grid-cert-request - creation private/public keys pair * grid-ca-sign -in -out - issuing a certificate for some user * grid-proxy-init - creating session for transferring files using GridFTP. The proxy certificates are stored inside /tmp/x509* files % -verify % -debug * globus-gridftp-server - GridFTP server startup script * globus-url-copy gsiftp:// file:/// - transferring files using GridFTP * counter-client - client of CounterService web service * globusrun-ws - submission program % -submit - to submit a program % -validate - to ensure the validity of program arguments % -factory-type <> - what kind of scheduler it is % -factory | -F <> - where the scheduler is located % -job-command <> - what job with what parameters to run % -status - monitor the job's state (like condor_q) * wsrf-query - to query the web services that implement the WS-Resource interface % -s % Has to include "wsdlpp:extends="wsrpw:GetResourceProperty" in wsdl file in order to be able to retrieve the resource property value from the provider. Otherwise the provider is equal to null * wsrf-destroy - destroys the web service by its EPR GRAM client interface --------------------- * 4 port types: % Managed Job - job monitoring % Managed Job Factory - job creation % Delegation - % Delegation Factory - security and delegating jobs to other pools MDS4 client interface and architecture -------------------------------------- * 3 port types: % MDS-Index - monitoring the values, collected from registered resources (by means of XPath) % MDS-Trigger - performs user-specified actions (sending e-mails or generating logs) % MDS-Archive - monitoring historical information from a persistent database * WebMDS works as a servlet of standalone Tomcat container (it is possible though to run everything: both Globus container and Tomcat itself - in Tomcat container only; see GT 4.0 WS MDS WebMDS: System Administrator's Guide). The Tomcat container is installed by unzipping its tarball. In order to create the context file to link between Tomcat and MDS, one must issue the following: % $GLOBUS_LOCATION/lib/webmds/bin/webmds-create-context-file $CATALINA_HOME/conf/Catalina/localhost And then start the Tomcat by issuing $CATALINA_HOME/bin/startup.sh xalan.jar file must be present in $CATALINA_HOME/common/lib in order for the WebMDS to work properly RLS general architecture ------------------------ * Local replica catalog (LRC) stores mappings between physical and logical names for certain resources * Replica location index (RLI) collects information about the logical name mappings stored in LRCs * Client might query both LRC and RLI. In the first case it will only get the local mapping of the desired resource. In the latter it will get the LRIs, which contain all the replicas, which can be recursively queried about the resources Technicalities -------------- * Globus container listens on default port 8443 * GridFTP server listens on default port 2811 * $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml - specifies Postgre database connection database * $GLOBUS_LOCATION/share/globus_wsrf_rft/rft_schema.sql - setup SQL script for creating RFT schema in the Postgre database * $GLOBUS_LOCATION/log4j.properties - specifies logging properties of SOAP server % log4j.category.org.globus.wsrf.handlers.MessageLoggingHandler = DEBUG - debugging level * $GLOBUS_LOCATION/etc/globus-devel-env.sh - environment for Java developer of GT4 * $GLOBUS_LOCATION/etc/globus-env.sh - environment for launching GT4 utilities * $GLOBUS_LOCATION/etc/globus_wsrf_rft/security-config.xml - contains container-wide security definitions, all the security can be annuled by setting parameter * Postgre SQL commands must end with semi-colon * Runtime URI of the container: NotificationConsumerManager consumer; consumer = NotificationConsumerManager.getInstance(); logger.info("Running inside " + consumer.getURL()); or URL baseURL = ServiceHost.getBaseURL(); String instanceService = home.getInstanceServicePath(); * Looking the value of "instanceHome" in JNDI: ResourceHome home; ResourceContext ctx; ctx = ResourceContext.getResourceContext(); String homeLoc = Constants.JNDI_SERVICES_BASE_NAME + ctx.getService() + "/instanceHome"; try { Context initialContext = new InitialContext(); home = (ResourceHome) initialContext.lookup(homeLoc); ... * Trying to start up the Globus container results in the following Error : Unable to setup database driver with pooling.Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. Solution: specifying 'tcpip_socket = true' in postgresql.conf (on Linux in '/var/lib/psql/data/postgresql.conf'), which is equivalent to running the 'postmaster' process (Postgre listener) with -i flag * Trying to reliably transfer file by 'rft -h -f /tmp/rft.xfr' results in the following Error : Error authenticating user at source/dest host. Authentication failed [Caused by: Operation unauthorized (Mechanism level: Authorization failed. Expected "/CN=host/localhost.localdomain" target but received "/O=Grid/OU=GlobusTest/OU=simpleCA-pcait194/CN=host/pcait194")] [Caused by: Authentication failed Solution: lines 8 and 9 of .xfr file must be equal to GridFTP name of the source and destination servers * Trying to compile WSDL files into Java classes results in the following Error : verification failed at PC 517 Solution: use proper version of JDK * Trying to wsrf-query services of container results in the following Error : ERROR container.GSIServiceThread inside the container.log (NextedException on the stdout) Solution: run the container in -nosec mode * Trying to start up the WebMDS service results in the following Error : Servlet WebmdsServlet is not available Solution: configure the $GLOBUS_LOCATION/lib/webmds/conf/*_nosec properly * Trying to start up the WebMDS service results in the following Error : java.lang.NoClassDefFoundError: org/apache/xalan/transformer/TransformerIdentityImpl Solution: copy the appropriate .jar to $CATALINA_HOME/common/lib * Anything that results in the following Error : java.io.IOException: Token length [...] > [...] Solution: there is a mismatch between the protocols of client and server (HTTP and HTTPS) * While starting Globus container occurs the following Error : ERROR database.RFTDatabaseSetup [main,getDBConnection:205] Unable to connect to database.Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at org.postgresql.jdbc1.AbstractJdbc1Connection.openConnection Solution: verify if $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml is valid and if Postgre database is running (/etc/init.d/postgre start) * While starting Globus container occurs the following Error : ERROR database.RFTDatabaseSetup [main,getDBConnection:205] Unable to connect to database.A connection error has occurred: FATAL: No pg_hba.conf entry for host 132.68.37.112, user sharov, database rftDatabase Solution: add the following line host rftDatabase "sharov" "132.68.37.112" 255.255.255.255 md5 to pg_hba.conf in the Postgre server machine (/var/lib/pgsql/data/pg_hba.conf) * While starting Globus container occurs the following Error : InvocationTargetException Solution: seems like the deployment of last web services was unsuccessful. Remove all the files from $GLOBUS_LOCATION/share/schema/ and from $GLOBUS_LOCATION/etc/. Recompile and redeploy the services and restart the container * While querying resource properties and/or notifications occurs the following Error : java.lang.IllegalArgumentException or ClassCastException Solution: verify that you imported the wsdls of resource properties and notifications Formats ------- * XFR format (describing RFT details): % [true | false] (binary | ascii transfer) % 16000 (block size in bytes) % 16000 (TCP Buffer size in bytes) % [true | false] (third party transfer) % 1 (number of parallel streams) % [true | false] (data channel authentication (DCAU)) % 1 (concurrency of the request) % /O=Grid/OU=GlobusTest/OU=simpleCA-pcait194/CN=host/pcait194 (grid Subject name of the source gridftp server) % /O=Grid/OU=GlobusTest/OU=simpleCA-pcait194/CN=host/pcait194 (grid Subject name of the destination gridftp server) % [true | false] (transfer all or none of the transfers) % 10 (maximum number of retries) % gsiftp://gridftp.mcs.anl.gov/etc/group (source URL) % gsiftp://gridftp.mcs.anl.gov/tmp/deleteme (destination URL) * RSL format (describing submitted job details) % my_echo ${GLOBUS_USER_HOME} Hello World! ${GLOBUS_USER_HOME}/stdout ${GLOBUS_USER_HOME}/stderr gsiftp://cognito.mcs.anl.gov:2811/bin/echo file:///${GLOBUS_USER_HOME}/my_echo file:///${GLOBUS_USER_HOME}/my_echo