Usage of the parallel machine priamos (SUN E3500):

Basic rules:

• Priamos should be primarily used for parallel jobs, that can be executed in batch.
  Such jobs will be givens priority over others.



• Every job must be submitted via the Load Sharing Facility (LSF), as described below.



• It is possible to enter interactive jobs using LSF. They will be niced.

How to run a job on priamos using LSF:

• In the future, it will be possible to submit a job from several machines.
  For the time being, you must first rlogin to priamos.



• If you want to use the new Workshop (compilers) make sure that you run the command:
source /usr/local/bin/workshop-5.0

check with the command "which f77". Output must be: "/opt/SUNWspro/bin/f77".


• Create a directory in one of the "scratch" areas on a local disk on priamos and copy there all what you need to run your job.
  Scratch areas are: /home/priamos15/scratch15 and /home/priamos16/scratch16
  You have access to them from your machine directly (are automounted): it is also NOT necessary to rlogin priamos in order to copy files there.
  
  
• To submit a job, use the tool "xlsbatch" that provides a GUI to LSF.
  (check: "which xlsbatch" ==> "/usr/local/lsf/bin/xlsbatch")


There are 'command line' commands which do the same, see further below.
• As you will see in the xlsbatch window, different queues are currently defined:

   - 1-CPU-normal  : for non-parallel jobs
   - 2-CPUs  : for parallel jobs that use 2 processors
   - 4-CPUs  : for parallel jobs that use 4 processors
   - 8-CPUs  : for parallel jobs that use 8 processors
   - 4-CPUs-slow   : For very long jobs, running on 4 processors.
   - short   : for short (max. 20 Min CPU time) non-parallel jobs

• To submit a job with xlsbatch, use the menu "Job" -> "Submit" (Batch) and type (or pick up from list) the script you want to execute in the field "Command Line".
  (don't forget to give execution permission to that script)

Here is such a very simple script that will just print a message: (the "sleep 10" is just put there so that you have a chance to see the job while it passes through the queue)

#!/bin/sh
cd $HOME
# better you cd to one of the scratch areas:
cd /home/priamos15/scratch15
echo "This message is from your batch job on priamos"
sleep 10
echo "Here we would have run your job"
exit


A real script would have of course changed to an appropriate directory, runned one or several programs and eventually post-processed your data and sent a pretty plot to the printer.



• By default, the output of your batch job is sent to you by e-mail.



• A last hint: use as much as possible /tmp for your I/O, it's much faster.

You can always transfer the output to your home directory later.

Command line LSF batch commands

• bsub - submit a job for batched execution by the LSF system
• bjobs - get information about batch jobs in the LSF system
• bqueues - get information about batch queues in the LSF system

     % bsub sleep 100
          Submit the UNIX command sleep together with  its  argu-
          ment 100 as a batch job.

     % bsub -q short -o my_output_file "pwd; ls"
          Submit the UNIX command pwd and ls as a  batch  job  to
          the  queue  named  short  and  store  the job output in
          my_output file.
..

     % bsub my_script
          Submit my_script as a batch job.   Since  my_script  is
          specified  as  a  command  line argument, the my_script
          file is not spooled.  Later changes  to  the  my_script
          file before the job completes may affect this job.

     % bsub < default_shell_script
          where default_shell_script contains:

          sim1.exe
          sim2.exe

          The file default_shell_script is spooled, and the  com-
          mands  will be run under the Bourne shell since a shell
          specification is not given in the  first  line  of  the
          script.

     % bsub < csh_script
          where csh_script contains:

          #! /bin/csh
          sim1.exe
          sim2.exe

          csh_script is spooled and  the  commands  will  be  run
          under /bin/csh.


     Use the -N option to request mail,  and/or  the  -o  and  -e
     options  to  specify  an output file and error file, respec-
     tively.



Example for bqueues :

priamos:ballisti:[fortran]:61 %	bqueues
QUEUE_NAME      PRIO STATUS          MAX JL/U JL/P JL/H NJOBS  PEND   RUN  SUSP 
short            35   Open:Active      -    4    3   12     0     0     0     0
4-CPUs           30   Open:Active      -    2    -    2     0     0     0     0
8-CPUs           26   Open:Active      -    1    -    2     0     0     0     0
2-CPUs           25   Open:Active      6    4    1    5     0     0     0     0
4-CPUs-slow      20   Open:Active      3    2    2    2     0     0     0     0
1-CPU-normal      5   Open:Active      8    8    1   10    12     4     8     0
priamos:ballisti:[fortran]:62 %	

Important for you here is mostly:

1-CPU-normal      5   Open:Active      8    8    1   10    12     4     8     0
                                                           ^^     ^^    ^^

NJOBS is the number of submitted jobs

PEND is the number of waiting jobs in queue while RUN jobs are being executed.

SUSP is the number of jobs which have been stopped for some reason (mostly because
     there is a high rate of paging). Those jobs will be started automatically when
     the reason for the stopping goes away. 

Last update: 12th July 2000 by Ray