Open Grid Service Architecture (OGSA) Installation Guide for RedHat 9.0 Linux HOWTO by Hong Ong $Id: gt3_install.txt,v 1.1 2004/03/30 17:03:07 hong Exp hong $ This document explains how to install, configure and deploy the Globus GT3 toolkit. Although the explanation is aimed toward the RH 9.0 Linux distribution, other UNIX or UNIX-like distributions could potentially follow this guide. Copyright and License --------------------- This document, Open Grid Service Architecture (OGSA) Installation Guide for RedHat 9.0 Linux HOWTO, is copyrighted (c) 2003, 2004 by the Distributed Systems Group, Portsmouth, UK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html. Linux is a registered trademark of Linus Torvalds. Redhat is a registered trademark of Redhat Inc. Disclaimer ---------- No liability for the contents of this document can be accepted. Use the concepts, examples and information at your own risk. There may be errors and inaccuracies that could be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility. All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements. 0. Prologues ============ The document details the installation process for GT3.2. However, the installation process for GT3.0+ and GT3.2+ is similar, if not identical. Therefore, you could potentially use this guide for GT3.0 installation. We will point out the difference(s) where appropriate. We have produced scripts to automate (where possible) the installation steps described in this document. The scripts are now in alpha release and can be downloaded at http://dsg.port.ac.uk/projects/ogsa-testbed. 1. Introduction =============== 2 Software Requirements ========================= * GT3.2beta or GT3.0.2 Package: gt3 Version: 3.0+ Homepage: http://www-unix.globus.org/toolkit Description: Grid middleware * The following packages are required prior to GT3 installation. Please refer to "OGSA Prerequisite Software Installation and Configuration on Linux RedHat 9.0 HOWTO" on how to install and configure the following packages. * Java(TM) 2 Software Development Kit, Standard Edition Package: j2sdk Version: 1.4.2_03 Homepage: http://java.sun.com/linux Description: JVM, core class libraries and tools for Java programming. * Apache Ant Package: apache-ant Version: 1.6.1+ Homepage: http://ant.apache.org/ Description: apache build tool * Network Time Protocol Package: ntp Version: 4.1.2 Homepage: http://www.ntp.org Description: Network time protocol. * [Optional] Junit Package: junit Version: 3.8.1+ Homepage: http://www.junit.org Description: Unit testing. Junit is required if you want to do testing. * [Optional] Jakarta Servlet Container Tomcat 4.1+ Package: jakarta-tomcat Verion: 4.1+ Homepage: http://jakarta.apache.org/tomcat/index.html Description: Web service container. Tomcat is required if you play to deploy OGSA to tomcat container * [Optional] Relational Database and JDBC Driver Relational database (either mysql or postgresql) and its corresponding JDBC driver are required if you want to install MMJFS, and run the Reliable File Transfer (RFT) and Replica Location Service (RLS) services. Choose one: * Postgresql Package: postgresql Version: 7.4.0+ Homepage: http://www.postgresql.org Description: relational database Package: postgresql-jdbc Version: 7.4.0+ Homepage: http://www.postgresql.org Description: Postgresql JDBC Driver * MySQL Package: mysql Version: 4.0.18+ Homepage: http://www.mysql.com Description: relational database Package: mysql-connector-java Version: 3.0.11 Homepage: http://www.mysql.com/downloads Description: mysql jdbc driver 3. Globus GT3 Installation ========================== 3.1 Caveat ========== * The target platform is a RH 9.0 Linux. * You are familiar with system administration tasks. * You have correctly installed and tested all prerequisite software prior to GT3 installation. * If the aforementioned software (in Section 2) is either not installed or tested, or you are not sure, please refer to "OGSA Prerequisite Software Installation and Configuration on Linux RedHat 9.0 HOWTO" 3.2 GT3 Installation ==================== 3.2.1 Roadmap ============= * GT3 installation is performed in five stages: ** Stage 1 - Install GT3 toolkits ** Stage 2 - Set up the Grid Security Infrastructure (GSI) ** Stage 3 - Install Job Submission Manager (MMJFS) ** Stage 4 - Configure GT3 toolkits ** Stage 5 - Testing GT3 toolkits * Note that if you want to install GT3.0.2 rather than GT3.2beta, please substitute 'gt3.2b' with 'gt3.0.2', where appropriate, in the following installation description. 3.2.2 Annotation ================ OGSA_HOME - GT3 installation directory (/opt/globus/gt3) ANT_HOME - Apache Ant directory (/opt/apache-ant/ant) JAVA_HOME - Java SDK directory (/usr/java/java) CATALINA_HOME - jakarta tomcat directory (/opt/tomcat4/tomcat) 3.2.3 Stage 1 - Install GT3 toolkits ==================================== * Download GT3.x from http://www-unix.globus.org/toolkit A. Create Installation Directory ---------------------------------- * Make a $OGSA_HOME installation directory. This is where all the GT3 installation will reside. [root@l59 ]# mkdir -p /opt/globus/gt3.2b * Create a symbolic link to $OGSA_HOME [root@l59 ]# ln -s /opt/globus/gt3.2b /opt/globus/gt3 * GT3 needs to be installed as a non-root user. Create user and group ids. As root, [root@l59 ]# groupadd globus [root@l59 ]# useradd -g globus -c /opt/globus globus * [Optional] we could give globus a password. [root@l59 ]# passwd globus * Change $OGSA_HOME directory permission [root@l59 ]# chown -R globus.globus /opt/globus B. Unpacking the source --------------------------------- * Unpack the GT3 source [globus@l59 ]$ [globus@l59 ]$ tar zxvf gt3.2beta-all-installer.tar.gz gt3.2beta-all-installer/ gt3.2beta-all-installer/bundles/ gt3.2beta-all-installer/bundles/globus-data-management-client-3.2-src_bundle.tar.gz gt3.2beta-all-installer/bundles/globus-data-management-sdk-3.2-src_bundle.tar.gzgt3.2beta-all-installer/bundles/globus-data-management-server-3.2-src_bundle.tar.gz gt3.2beta-all-installer/bundles/globus-information-services-client-3.2-src_bundle.tar.gz < ... lines omitted > gt3.2beta-all-installer/contrib/ gt3.2beta-all-installer/contrib/pyGridWare-gt3-1.0.1.tar.gz gt3.2beta-all-installer/contrib/pyGlobus-gt-3.2.beta-1.0.3.tar.gz gt3.2beta-all-installer/contrib/gt3tutorial-0.2.2.tar.gz gt3.2beta-all-installer/contrib/ogsadai-3.1-src.tar.gz C. Check for Java, ANT, etc. ---------------------------- * Change directory to the gt3.2beta-all-installer directory: [globus@l59 ]$ cd gt3.2beta-all-installer/ * Check that JAVA_HOME and ANT_HOME, are set. [globus@l59 ]$ echo $JAVA_HOME $ANT_HOME /usr/java/java /opt/apache-ant/ant ** Export these global variables if not set, [globus@l59 ]$ export JAVA_HOME=/usr/java/java [globus@l59 ]$ export ANT_HOME=/opt/apache-ant/ant * Check that the path to java and ant executable are set. [globus@l59 ]$ which java ant /usr/java/java/bin/java /opt/apache-ant/ant/bin/ant ** Export the path if not set, [globus@l59 ]$ export PATH=/usr/java/java/bin:$PATH [globus@l59 ]$ export PATH=/opt/apache-ant/ant/bin:$PATH * For GT3.2, you may need to install the libxml2-devel and libxml2 packages. This is a reported bug. Please see, http://bugzilla.globus.org/globus/show_bug.cgi?id=1560 * To check, whether you have libxml2-devel and libxml2 installed: [root@l59 ]# rpm -q libxml2-devel libxml2 libxml2-2.5.4-2 libxml2-devel-2.5.4-2 * If you don't see this output, you need to install these packages. * You can download these packages from Redhat's download mirror site * To install libxml2 package, do [root@l59 ]# rpm -ivh libxml2-2.5.4-2.i386.rpm libxml2-devel-2.5.4-2.i386.rpm D. Install the GT3 source ------------------------- * Install the GT3 toolkits [globus@l59 ]$ (date; ./install-gt3 /opt/globus/gt3; date) | tee gt3.2b-install.log ** The installation output: Tue Mar 2 17:43:12 GMT 2004 Build environment: ant is /opt/apache-ant/ant/bin/ant java is /usr/java/java/bin/java gcc is /usr/bin/gcc Building GPT ... build_gpt ====> installing GPT into /opt/globus/gt3.2b build_gpt ====> building support/Compress-Zlib-1.16 build_gpt ====> building support/Archive-Tar-0.22 < ... lines omitted > Creating the job manager configuration file... Done running /opt/globus/gt3.2b/setup/globus/setup-globus-job-manager-fork... loading cache ./config.cache checking for mpirun... /opt/lam/bin/mpirun updating cache ./config.cache creating ./config.status creating fork.pm Tue Mar 2 20:37:57 GMT 2004 ** The date command is to keep a record on how long it takes to install GT3. You can omit this if you want. ** The installation may take hours depending on your hardware capability. E. Check for error. ------------------- * Check for installation errors. Use grep to check for installation errors. [globus@l59 ]$ grep Error gt3.2b-install.log |more ** If you received an error message like this: /usr/bin/ld: cannot find -lxml2 collect2: ld returned 1 exit status make[2]: *** [libglobus_gaa_simple_gcc32dbg.la] Error 1 make[2]: Leaving directory `/tmp/download/gt3.2beta-all-installer/BUILD/gaa_simple-1.0' make[1]: *** [all_filelists] Error 1 make[1]: Leaving directory `/tmp/download/gt3.2beta-all-installer/BUILD/gaa_simple-1.0/pkgdata' make: *** [all-recursive] Error 1 ** It means libxml2 package is not installed. This bug has been reported . A solution is to install the libxml2, libxml2-devel packages. ** If you received a warning message like this: Warning: Host cert file: /etc/grid-security/hostcert.pem not found. Re-run setup-globus-gram-job-manager after installing host cert file. ** It means you have not set-up the /etc/grid-security yet. This will be covered later. * If you are using j2sdk1.4.x, you need a newer version of xlan.jar. xalan.jar is used by GT3 security libraries. As root, [root@l59 ]# mkdir -p $JAVA_HOME/jre/lib/endorsed [root@l59 ]# cp /opt/globus/gt3/endorsed/xalan.jar $JAVA_HOME/jre/lib/endorsed F. Set up the OGSA Framework binary (generating the launcher scripts). ---------------------------------------------------------------------- * Goto the $OGSA_HOME directory. As globus, [globus@l59 ]$ cd /opt/globus/gt3/ [globus@l59 ]$ ant setup Buildfile: build.xml launchers: generateLaunchers: [echo] generating launcher scripts setAbsoluteGlobusLocation: setClasspathScriptPath: < ... lines omitted > testWindows: generateCoGLaunchersWindows: setup: BUILD SUCCESSFUL Total time: 13 seconds 3.2.4 Stage 2 - Set up GSI ========================== * If your /etc/grid-security directory is configured, Goto Stage 3. * If you want to set-up your own /etc/grid-security (because you want to use your own CA certificate), Go to step C. A. Set up Simple CA ------------------- * Setting environment variables. As root, [root@l59 ]# export GLOBUS_LOCATION=/opt/globus/gt3 * If you are installing gt3.0.2, goto step B. * For GT3.2, run set-up-simple-ca [root@l59 ] # cd $GLOBUS_LOCATION [root@l59 ]# setup/globus/setup-simple-ca C e r t i f i c a t e A u t h o r i t y S e t u p This script will set-up a Certificate Authority for signing Globus users certificates. It will also generate a simple CA package that can be distributed to the users of the CA. The CA information about the certificates it distributes will be kept in: /root/.globus/simpleCA/ The unique subject name for this CA is: < ... lines omitted > gpt-build ====> REMOVING empty package globus_simple_ca_2854b60c_setup-noflavor-pgm_static gpt-build ====> REMOVING empty package globus_simple_ca_2854b60c_setup-noflavor-rtl setup-ssl-utils: Configuring ssl-utils package Running setup-ssl-utils-sh-scripts... *************************************************************************** Note: To complete set-up of the GSI software you need to run the following script as root to configure your security configuration directory: /opt/globus/gt3.2b/setup/globus_simple_ca_2854b60c_setup/setup-gsi For further information on using the setup-gsi script, use the -help option. The -default option sets this security configuration to be the default, and -nonroot can be used on systems where root access is not available. *************************************************************************** setup-ssl-utils: Complete B. Grid Security Infrastructure (GSI) -------------------------------------- * [GT3.2] To run the setup-gsi script, type [root@l59 ]# setup/globus_simple_ca_2854b60c_setup/setup-gsi setup-gsi: Configuring GSI security Making /etc/grid-security... mkdir /etc/grid-security Making trusted certs directory: /etc/grid-security/certificates/ mkdir /etc/grid-security/certificates/ Installing /etc/grid-security/certificates//grid-security.conf.2854b60c... Running grid-security-config... Installing Globus CA certificate into trusted CA certificate directory... Installing Globus CA signing policy into trusted CA certificate directory... setup-gsi: Complete * [GT3.0] To run the setup-gsi script, type [root@l59 ]# setup/globus/setup-gsi setup-gsi: Configuring GSI security Making trusted certs directory: /etc/grid-security/certificates/ mkdir /etc/grid-security/certificates/ Installing /etc/grid-security/certificates//grid-security.conf.42864e48... Running grid-security-config... G S I: C O N F I G U R A T I O N P R O C E D U R E Before you use the Grid Security Infrastructure, you should first define the DN (distinguished name) that should be used for your organization's X509 certificates. If you do not define a DN, a default DN will be assigned to you. This script will ask some questions about site-specific information. This information is used to configure the Grid Security Infrastructure for your site. For some questions, a default response is given in []. Pressing RETURN in response to such a question will enable the default. This script will overwrite the file -- /etc/grid-security/certificates//grid-security.conf.42864e48 Do you wish to continue (y/n) [y] : ======================================================================== (1) Base DN for user certificates [ ou=dsg.port.ac.uk, o=Globus, o=Grid ] (2) Base DN for host certificates [ o=Globus, o=Grid ] ======================================================================== (q) save, configure the GSI and Quit (c) Cancel (exit without saving or configuring) (h) Help ======================================================================== q Installing Globus CA certificate into trusted CA certificate directory... Installing Globus CA signing policy into trusted CA certificate directory... setup-gsi: Complete C. Request for host and user certificate: ----------------------------------------- * Now, we need to request for a host certificate. This can be accomplished in several ways depending on your country and location. ** We used certificates from UK e-Science support centre. ** From UK e-Science support: follow the instructions on the web page http://www.grid-support.ac.uk/ca/documentation.htm. * To get a certificate signed by e-Science support centre, goto http://ca.grid-support.ac.uk/ ** Briefly, there are two ways to obtained an e-Science certificate: *** Method 1: Using NS 4.7 browser (note 7100+): 0. Goto http://ca.grid-support.ac.uk 1. Get the CA Root Certificate (click) 2. Request a Certificate (click) 3. Complete the request 4. Submit the forms 5. Remember the PIN 6. Arrange a visit to the RA < Takes a few days off .... > 7. Receive e-mail from CA 8. Download a certificate (click) 9. Import Certificate Revocation list in browser (click) 10. Test a Certificate (click) 11. Export a copy of the certificate for backup *** Method 2: Using the Java Certificate Request (JCR) utility (note 7400+): 0. Goto http://ca.grid-support.ac.uk 1. JCR Request (click) 2. Check the following are installed, if not they could be downloaded from the same page: 2.1 JRE 1.4.2 2.2 Sun unrestricted files for JCE 2.3 Bouncycastle cryptographic provider 2.4 The e-Science certificate 3. Click here to launch JCR (click) 4. Follow instructions on screen 5. Receive e-mail from CA 6. Download a certificate (click) 7. Import Certificate Revocation list in browser (click) 8. Test a Certificate (click) 9. Export a copy of the certificate for backup D. Set up the /etc/grid-security host certificate ------------------------------------------------- * Once you received certificates for the host, you need to convert the certificate. This is to be installed in GT3 and Apache server later. [root@l59 ]# openssl pkcs12 -in l59.dsg.port.ac.uk.p12 -clcerts -nokeys -out hostcert.pem [root@l59 ]# openssl pkcs12 -in l59.dsg.port.ac.uk.p12 -nodes -nocerts -out hostkey.pem ** Substitute l59.dsg.port.ac.uk.p12 for your host certificate. * Make a /etc/grid-security/certificates directory if it doesn't exist. [root@l59 ]# mkdir -p /etc/grid-security/certificates ** Change directory to /etc/grid-security/certificates [root@l59 ]# cd /etc/grid-security/certificates * Get e-Science CA certificate/public key [root@l59 ]# wget http://www.grid-support.ac.uk/ca/user-documentation/01621954.0 [root@l59 ]# wget http://www.grid-support.ac.uk/ca/user-documentation/01621954.signing_policy ** Edit the 01621954.signing_policy according to http://www.grid-support.ac.uk/downloads/pdf/6300_Signing_Policy_02.pdf. Look for access_id_CA X509 '/C=UK/O=eScience/OU=Authority/CN=CA/Email=ca-operator@grid-support.ac.uk' pos_rights globus CA:sign cond_subjects globus '/C=UK/O=eScience/*' *** Add extra info to 01621954.signing_policy, access_id_CA X509 '/C=UK/O=eScience/OU=Authority/CN=CA/Email=ca-operator@grid-support.ac.uk' pos_rights globus CA:sign cond_subjects globus '"/C=UK/O=eScience/*"' # This is the extra bit; the difference is in 'CA/emailAddress' access_id_CA X509 '/C=UK/O=eScience/OU=Authority/CN=CA/emailAddress=ca-operator@grid-support.ac.uk' pos_rights globus CA:sign cond_subjects globus '"/C=UK/O=eScience/*"' * Create globus-user-ssl.conf, globus-host-ssl.conf, and globus-security.conf [root@l59 ]# cp globus-user-ssl.conf.42864e48 globus-user-ssl.conf.01621954 [root@l59 ]# cp globus-host-ssl.conf.42864e48 globus-host-ssl.conf.01621954 [root@l59 ]# cp globus-security.conf.42864e48 globus-security.conf.01621954 * Edit globus-user-ssl.conf.01621954; Look for the following lines and change to your DN, and CN appropriately 1.organizationName_default = Portsmouth 0.organizationalUnitName_default = dsg.port.ac.uk commonName = DSG * Edit globus-host-ssl.conf.01621954. Look for the following lines and change accordingly: 1.organizationName_default = Portsmouth commonName = DSG * Edit grid-security.conf.01621954; Look for the following lines and change accordingly: SETUP_GSI_HOST_BASE_DN="o=eScience, o=Grid" SETUP_GSI_USER_BASE_DN="ou=Portsmouth, o=eScience, o=Grid" SETUP_GSI_CA_NAME="UK eScience CA" SETUP_GSI_CA_EMAIL_ADDR="ca@grid-support.ac.uk" DEFAULT_GSI_HOST_BASE_DN="o=eScience, o=Grid" DEFAULT_GSI_USER_BASE_DN="ou=${_domain}, o=eScience, o=Grid" DEFAULT_GSI_CA_NAME="UK eScience CA" DEFAULT_GSI_CA_EMAIL_ADDR="ca@grid-support.ac.uk" * Goto /etc/grid-security and create symbolic links [root@l59 ]# cd ../ [root@l59 ]# ln -s /etc/grid-security/certificates/globus-host-ssl.conf.01621954 globus-host-ssl.conf [root@l59 ]# ln -s /etc/grid-security/certificates/globus-user-ssl.conf.01621954 globus-user-ssl.conf [root@l59 ]# ln -s /etc/grid-security/certificates/grid-security.conf.01621954 grid-security.conf 3.2.5 Stage 3 - Installing MMJFS ================================ * DO NOT PROCEED UNTIL YOU HAVE A HOST CERTIFICATE!!!! A. Setting GRAM job manager --------------------------- * [Optional] If you receive a warning message in Stage 1 about setting up setup-globus-gram-job-manager, you need to run setup-gram-job-manager. As root, [root@l59 ]# export GLOBUS_LOCATION=/opt/globus/gt3 * Goto download directory. [globus@l59 ]$ cd /tmp/download/gt3.2beta-all-installer [globus@l59 ]$ (date; ./install-gt3-mmjfs /opt/globus/gt3.2b; date) | tee gt3.2b-mmjfs-install.log Wed Mar 3 14:37:14 GMT 2004 Build environment: ant is /opt/apache-ant/ant/bin/ant java is /usr/java/java/bin/java gcc is /usr/bin/gcc Building GPT ... build_gpt ====> installing GPT into /opt/globus/gt3.2b build_gpt ====> building support/Compress-Zlib-1.16 < ... lines omitted > postDeploy: [delete] Deleting directory /opt/globus/gt3.2b/tmp/gar [copy] Copying 1 file to /opt/globus/gt3.2b BUILD SUCCESSFUL Total time: 14 seconds Wed Mar 3 14:48:53 GMT 2004 C. Setup UID ------------ * Run setperm.sh to change the ownership and access permission; * As root, [root@l59 ]# bin/setperms.sh ** The detail of setperms.sh + chown root /opt/globus/gt3.2b/bin/launch_uhe_setuid + chmod 4750 /opt/globus/gt3.2b/bin/launch_uhe_setuid + chown root /opt/globus/gt3.2b/bin/globus-grim + chmod 4755 /opt/globus/gt3.2b/bin/globus-grim + chown root /opt/globus/gt3.2b/bin/launch_uhe.sh + chmod 755 /opt/globus/gt3.2b/bin/launch_uhe.sh ** The result: {hong@l59 < /opt/globus/gt3.2b >}[%4] ls -l bin/launch_uhe* bin/globus-grim* -rwsr-xr-x 1 root globus 12081012 Mar 5 11:18 bin/globus-grim* -rwsr-x--- 1 root globus 16532 Mar 5 11:20 bin/launch_uhe_setuid* -rw-r--r-- 1 globus globus 6956 Mar 5 11:20 bin/launch_uhe_setuid.c -rwxr-xr-x 1 root globus 1581 Mar 5 11:20 bin/launch_uhe.sh* ** Because of the above, it is important that the account under which you plan to run the GRAM master managed job factory is a member of the group that owns the launch_uhe_setuid program. This group defaults to the default group of the installing user and should only contain privileged members. 3.2.6 Stage 4 - Configuration ============================= A. Create Grid Security files ------------------------------ * Goto /etc/grid-security [root@l59 ]# cd /etc/grid-security/ * Create a grid-map file and insert all users DN: [root@l59 ]# vi grid-mapfile "/C=UK/O=eScience/OU=Portsmouth/L=DSG/CN=hong ong" hong ** The content of grid-map file comes from user certificate. If in doubt, use the following command to find out (replace the username with appropriate user): [root@l59 ]# grid-cert-info -s -f ~hong/.globus/usercert.pem ** Make sure you export GLOBUS_LOCATION and source the globus environment variable before you run grid-cert-info. Otherwise, it will fail. [root@l59 ]# export GLOBUS_LOCATION=/opt/globus/gt3 [root@l59 ]# source $GLOBUS_LOCATION/etc/globus-user-env.sh * Create XML document that maps the user id to the Grid service name (one line per user): [root@l59 ]# vi grim-port-type.xml http://www.globus.org/namespaces/managed_job/managed_job/ManagedJobPortType B. Set up relational databases for use with MMJFS. -------------------------------------------------- * You can either use mysql or postgresql database. Configure postgresql database ----------------------------- ** Make sure postgresql daemon is running: [root@l59 ]# ps -aux | grep postgres postgres 7657 0.0 0.1 18872 256 ? S Mar02 0:00 /usr/bin/postmaster -i -p 5432 -D /var/lib/pgsql/data postgres 7662 0.0 0.0 9672 4 ? S Mar02 0:00 postgres: stats buffer process postgres 7663 0.0 0.0 8736 4 ? S Mar02 0:00 postgres: stats collector process ** If not, refer to "OGSA Prerequisite Software Installation and Configuration on Linux RedHat 9.0 HOWTO" for starting postgres daemon * As postgres user, create the new database user named globus [root@l59 ]# su - postgres -bash-2.05b$ createuser globus Shall the new user be allowed to create databases? (y/n) y Shall the new user be allowed to create more new users? (y/n) y CREATE USER ** Note do not get confused with the database user 'globus' and the system user 'globus'; You may want to call your database user some other name. * As globus, populate the database. [globus@l59 ]$ createdb ogsa CREATE DATABASE * Then, create its table, [globus@l59 ]$ psql -d ogsa -f /opt/globus/gt3/etc/databaseSchema/jm_database_schema.sql CREATE SEQUENCE psql:/opt/globus/gt3/etc/databaseSchema/jm_database_schema.sql:15: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "jobmanagerstatetable_pkey" for table "jobmanagerstatetable" CREATE TABLE psql:/opt/globus/gt3/etc/databaseSchema/jm_database_schema.sql:25: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "filestreamtable_pkey" for table "filestreamtable" CREATE TABLE psql:/opt/globus/gt3/etc/databaseSchema/jm_database_schema.sql:35: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "stagingtable_pkey" for table "stagingtable" CREATE TABLE Configure MySQL database ------------------------- * Make sure mysql server is running [root@l59 ]# ps -aux | grep mysql root 897 0.0 0.0 4416 28 ? S Mar02 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/l59.pid mysql 927 0.0 0.2 30968 564 ? S Mar02 2:28 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/lib/mysql/l59.pid --skip-locking ** If you don't see the above output, you need to start mysql server. Please refer to "OGSA Prerequisite Software Installation and Configuration on Linux RedHat 9.0 HOWTO" on how to start mysql server. ** Connect to mysql server as the database user 'root' [root@l59 root]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 11 to server version: 4.0.18-standard Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> ** Create a new database user account mysql> GRANT ALL ON ogsa.* TO 'globus'@'localhost%'; Query OK, 0 rows affected (0.18 sec) mysql> FLUSH PRIVILEGES; Query OK, 0 rows affected (0.11 sec) ** Create the actual database mysql> create database ogsa; Query OK, 1 row affected (0.34 sec) mysql> show databases; +----------+ | Database | +----------+ | mysql | | ogsa | | test | +----------+ s rows in set (0.00 sec) mysql> quit; ** Alternately, you can issue the command as: mysql> GRANT ALL ogsa.* TO 'globus'@'localhost%' IDENTIFIED BY 'secret'; ** Essentially, the above line creates a new database user account 'globus' and a new database 'ogsa' that can be used only when connecting from local host. The first command does not require a password. The second command allows the user 'globus' to access the database using 'secret' as password. ** The database user 'globus' can change the password later by issuing the following command: [root@l59 ]# mysql -u root Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 11 to server version: 4.0.18-standard Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> SET PASSWORD FOR 'globus'@'localhost%' = PASSWORD('secret'); * Populate the database. As globus, [globus@l59 ]$ mysql -u globus -p ogsa < jm_database_schema.mysql ** The jm_databases_schema.mysql is an adaptation of jm_database_schma.sql. ** The content of jm_database_schma.sql #create sequence jobManagerId_seq; create table JobManagerStateTable ( # jobManagerId int primary key default nextval('jobManagerId_seq'), jobManagerId int not null auto_increment primary key, jobManagerState int not null, jobManagerStatus int not null, jobManagerFailureCode text, jobId text, rsl text not null, cacheTag text, commitTimeout int, scratchDirectory text, credentialPath text not null ); create table FileStreamTable ( streamType text not null, fileOffset int, # fileStreamUrl text not null, fileStreamUrl varchar(150) not null, jobManagerId int references JobManagerStateTable, primary key(fileStreamUrl, jobManagerId) ); create table StagingTable ( # srcURL text not null, srcURL varchar(150) not null, # destURL text not null, destURL varchar(150) not null, stagingType text not null, jobManagerId int references JobManagerStateTable, primary key(srcURL, destURL, jobManagerId) ); ** Note that the adaptation has not been optimized for speed. Also, the length for *URL are defined to be 80 characters. It is not clear what is the maximum length should be. ** If you see the following error message: ERROR 1171 at line ##: All parts of a PRIMARY KEY must be NOT NULL; If you need NULL in a key, use UNIQUE instead Then you are using an old version of Mysql (probably a version 3) which has a bug in it. (http://bugs.mysql.com/bug.php?id=390) ** The fix is to upgrade your mysql to 4.0.18. * To check, login to mysql server as globus [globus@l59 ]$ mysql -u globus -p Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 16 to server version: 4.0.18-standard Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> use ogsa; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +----------------------+ | Tables_in_ogsa | +----------------------+ | FileStreamTable | | JobManagerStateTable | | StagingTable | +----------------------+ 3 rows in set (0.00 sec) mysql> describe FileStreamTable; +---------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +---------------+--------------+------+-----+---------+-------+ | streamType | text | | | | | | fileOffset | int(11) | YES | | NULL | | | fileStreamUrl | varchar(150) | | PRI | | | | jobManagerId | int(11) | | PRI | 0 | | +---------------+--------------+------+-----+---------+-------+ 4 rows in set (0.04 sec) * You also need to edit the following files to use mysql instead of postgresql: $GLOBUS_LOCATION/etc/casDBProperties $GLOBUS_LOCATION/local-server-config.wsdd $GLOBUS_LOCATION/server-config.wsdd ** Replace psql with mysql C. Setup environment for users. ------------------------------- * Create a user login scripts - /etc/profile.d/globus.sh export GLOBUS_LOCATION=/opt/globus/gt3.2b source $GLOBUS_LOCATION/etc/globus-user-env.sh *** FINALLY, WE DONE INSTALLATION AND CONFIGURATION!!!! *** 3.2.7 Stage 5 - Testing ======================= A. Start/Stopping Globus Container ---------------------------------- * As globus, [globus@l59 ]# export GLOBUS_LOCATION=`pwd` [globus@l59 ]# bin/globus-start-container -p 8080 [03/05/2004 11:27:51:939 ] org.globus.ogsa.server.ServiceContainer [run:582] INFO: Starting SOAP server at: http://148.197.155.159:8080/ogsa/services/ With the following services: http://148.197.155.159:8080/ogsa/services/core/admin/AdminService http://148.197.155.159:8080/ogsa/services/core/management/OgsiManagementService http://148.197.155.159:8080/ogsa/services/core/registry/ContainerRegistryServicehttp://148.197.155.159:8080/ogsa/services/core/jmsadapter/JMSAdapterFactoryService http://148.197.155.159:8080/ogsa/services/core/logging/OgsiLoggingManagementService < ... lines omitted > http://148.197.155.159:8080/ogsa/services/gsi/AuthenticationService http://148.197.155.159:8080/ogsa/services/gsi/SecureNotificationSubscriptionFactoryService http://148.197.155.159:8080/ogsa/services/gsi/SecureNotificationSubscriptionFactoryService/hash-9555723-1078411050254 * To stop the container [globus@l59 ]# export X509_USER_PROXY=/tmp/x509cp_globus_grim [globus@l59 ]# bin/globus-stop-container -secure soft B. Preparation -------------- * MAKE SURE YOUR GLOBUS CONTAINER HAS BEEN STARTED BEFORE DOING ANY TESTING. * If you have your user certificate installed, go to C. Test Job Submission. * As normal user, make a globus directory and cd to it {hong@l59 < ~ >}[%1] mkdir .globus {hong@l59 < ~ >}[%2] cd .globus * Assume the user certificate is stored in ~/.globus directory. Convert the pk12 format to pem format {hong@l59 < ~/.globus >}[%3] openssl pkcs12 -in hong.p12 -clcerts -nokeys -out usercert.pem {hong@l59 < ~/.globus >}[%4] openssl pkcs12 -in hong.p12 -nocerts -out userkey.pem {hong@l59 < ~/.globus >}[%5] chmod 400 userkey.pem {hong@l59 < ~/.globus >}[%6] chmod 444 usercert.pem C. Test job submission: -------------------- * As user, if GT3 PATH is not set, then set it {hong@l59 < ~ >}[%1] export GLOBUS_LOCATION=/opt/globus/gt3 {hong@l59 < ~ >}[%2] source $GLOBUS_LOCATION/etc/globus-user-env.sh * Create proxy. {hong@l59 < ~ >}[%3] grid-proxy-init Your identity: /C=UK/O=eScience/OU=Portsmouth/L=DSG/CN=hong ong Enter GRID pass phrase for this identity: Creating proxy ............................................................. Done Your proxy is valid until: Fri Mar 5 23:31:22 2004 * Start MMJFS, {hong@l59 < ~ >}[%4] managed-job-globusrun -factory http://148.197.155.159:8080/ogsa/services/base/gram/MasterForkManagedJobFactoryService -file $GLOBUS_LOCATION/etc/test.xml * The output of job submission WAITING FOR JOB TO FINISH ========== Status Notification ========== Job Status: Done ========================================= DESTROYING SERVICE SERVICE DESTROYED D. Test Gatekeeper ------------------ * Start personal gatekeeper {hong@l59 < ~ >}[%5] globus-personal-gatekeeper -start GRAM contact: l59.dsg.port.ac.uk:34194:/C=UK/O=eScience/OU=Portsmouth/L=DSG/CN=hong ong * Run a command. {hong@l59 < ~ >}[%6] globus-job-run "l59.dsg.port.ac.uk:34194:/C=UK/O=eScience/OU=Portsmouth/L=DSG/CN=hong ong" /bin/date Sun Nov 9 16:35:48 GMT 2003 ** Replace "l59.dsg.port.ac.uk:34194" to what you see on your screen. E. Test Grid FTP ---------------- * Start ftp daemon, {hong@l59 < ~ >}[%7] $GLOBUS_LOCATION/sbin/in.ftpd -S -p 5678 {hong@l59 < ~ >}[%8] globus-url-copy -s "`grid-cert-info -subject`" gsiftp://148.197.155.159:5678/tmp/hong.gftp file:///tmp/file2 ** If you see the following error, error: the server sent an error response: 530 530 No local mapping for Globus ID ** This is a bug reported in http://www-unix.globus.org/mail_archive/discuss/2003/03/msg00037.html "When running as a normal user, Globus looks for the gridmap file in ~/.gridmap .... create this file and add an entry with your subject and username. ** So, {hong@l59 < ~ >}[%9] echo '"'`grid-cert-info -subject`'"' `whoami` > .gridmap" {hong@l59 < ~ >}[%10] globus-url-copy -s "`grid-cert-info -subject`" gsiftp://148.197.155.159:5678/tmp/hong.gftp file:///tmp/file2 F. Test service browser ----------------------- {hong@l59 < ~ >}[%11] globus-service-browser * (Double) Click on services. D. Advance testing ------------------ * Run the gt3gits - see URL here 4. Deploying GT3 under Tomcat ============================== This is only necessary if you want to run OGSA framework inside tomcat container. A. Deploying GT3 ---------------- * Make sure the environment variables ($CATALINA_HOME, $JAVA_HOME, $ANT_HOME) and their path are set * If you tomcat is installed as "tomcat" user, you want to become "tomcat" before deployment. Otherwise, you can install it as root * As tomcat, [tomcat@l59 ]# ant -Dtomcat.dir=$CATALINA_HOME deployTomcat Buildfile: build.xml deployTomcat: deployTomcat: deployTomcatLibs: [copy] Copying 14 files to /opt/tomcat4/tomcat/common/lib [copy] Copying 1 file to /opt/tomcat4/tomcat/server/lib [copy] Copying 1 file to /opt/tomcat4/tomcat/common/endorsed deployWebappRoot: [mkdir] Created dir: /opt/tomcat4/tomcat/webapps/ogsa/schema [copy] Copying 218 files to /opt/tomcat4/tomcat/webapps/ogsa/schema deployWebapp: [copy] Copying 2 files to /opt/tomcat4/tomcat/webapps/ogsa [copy] Copying 42 files to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF/lib [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF/classes [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF/classes [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF [copy] Copying 1 file to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF/classes [copy] Copying 1975 files to /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF/etc [copy] Copied 368 empty directories to 1 empty directory under /opt/tomcat4/tomcat/webapps/ogsa/WEB-INF/etc BUILD SUCCESSFUL Total time: 35 seconds B. Configuring Tomcat --------------------- * Edit tomcat web.xml file. Insert the following to include .wsdl, .gwsdl, .xsd: [tomcat@l59 ]# cd $CATALINA_HOME/conf [tomcat@l59 ]# vi web.xml wsdl text/xml xsd text/xml gwsdl text/xml * You may need to copy extra files from GT3 to Tomcat. ** Check the following files are copy to $CATALINA_HOME/lib xmlParserAPIs.jar xercesImpl.jar * you may need to copy log4j.properties [root@l59 gt3]# cp log4j.properties $CATALINA_HOME/webapps/ogsa/WEB-INF/classes/ C. Testing GT3/Tomcat -------------- * Startup Tomcat [tomcat@l59 ]# $CATALINA_HOME/bin/startup.sh * Visit the http://localhost:8080/ogsa/services * You should see a list of the services deployed on Tomcat * For extra loggin, export the following export CATALINA_OPTS="-Dlog4j.configuration=file://$CATALINA_HOME/webapps/ogsa/WEB-INF/classes/log4j.properties" * Restart TOMCAT. Log turns up in $CATALINA_HOME/logs/catalina.out 5. Concluding Remarks ===================== * Comments are welcome. * We welcome feedbacks on this document. 6. Troubleshooting ================== All the problems with GT3 go here. 6.1 Troubleshooting GT3 ---------------------- * If you see the following error, [03/04/2004 14:01:18:014 ] org.globus.ogsa.handlers.container.ContainerHandlerHelper [shutdown:93] ERROR: Shutting down container. Container handler failed ('grim setuid program failed. globus-grim:1078408878: globus ::ERROR: The proxy credential could not be to /tmp/x509cp_globus_grim.') java.lang.Exception: grim setuid program failed. < ... lines omitted > Failed to initialize container handler: java.lang.Exception: grim setuid program failed. globus-grim:1078408878: globus ::ERROR: The proxy credential could not be to /tmp/x509cp_globus_grim. ** Most likely, you have forgotten to run setperm.sh after installing MMJFS! ** Run the setperm.sh again. * If you see the following error, [03/04/2004 14:06:11:660 ] org.globus.ogsa.impl.base.providers.servicedata.ServiceDataProviderManager [enumProviders:389] ERROR: Error enumerating service data providers: [org.globus.ogsa.config.ConfigException] [java.io.FileNotFoundException] java.io.FileNotFoundException: etc/rips-service-config.xml (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:106) at java.io.FileInputStream.<init>(FileInputStream.java:66) < ... lines omitted > ** This basically means GT cannot find rips-service-config.xml. ** (Recommended) You most probably did not start the container in $GLOBUS_LOCATION. You could either start the container in $GLOBUS_LOCATION or ** (Work around) Edit $GLOBUS_LOCATION/server-config.wsdd; Look for the following line ** Replace it by ** In general, this error is not fatal. You can opt for doing nothing. * If you see this error: [globus@holly gt3.2b]$ bin/globus-start-container -p 8080 Failed to start container: Address already in use Check that you have not got Tomcat running on port 8080 (or anything else for that matter) 6.2 OGSA/Tomcat Troubleshooting ------------------------------- * Refer to "Experiences with using Tomcat as the GT3 container", Thierry Delaitre, University of Westminster for further configuration and troubleshooting Acknowledgement =============== This project is funded in parts by the UK e-Science programme. This document has been based on many installation guides found on the Internet. The author(s) of this document wish to thanks all the authors who wrote the guides.