NAME
qsub - submit a batch job to Grid Engine.
qsh - submit an interactive X-windows session to Grid
Engine.
qlogin - submit an interactive login session to Grid
Engine.
qrsh - submit an interactive rsh session to Grid Engine.
qalter - modify a pending batch job of Grid Engine.
qresub - submit a copy of an existing Grid Engine job.
SYNTAX
qsub [ options ] [ scriptfile | - [ script_args ]]
qsh [ options ] [ -- xterm_args ]
qlogin [ options ]
qrsh [ options ] [ command [ command_args ]]
qalter [ options ] job/task_id_list [ -- [ script_args ]]
qalter [ options ] -u user_list | -uall [ -- [ script_args
]]
qresub [ options ] job_id_list
DESCRIPTION
Qsub submits batch jobs to the Grid Engine queuing system.
Grid Engine supports single and multiple node jobs.
scriptfile contains the commands to be run by the job using
a shell (for example, sh(1) or csh(1)). Arguments to the
job script are given by script_args. Grid Engine flags may
be entered as arguments to qsub or as embedded flags in the
scriptfile if the first two characters of a script line
either match '#$' or are equal to the prefix string defined
with the -C option described below.
Qsh submits an interactive X-windows session to Grid Engine.
An xterm(1) is brought up from the executing machine with
the display directed either to the X-server indicated by the
DISPLAY environment variable or as specified with the
-display qsh option. Interactive jobs are not spooled if no
resource is available to execute them. They are either
dispatched to a suitable machine for execution immediately
or the user submitting the job is notified by qsh that
appropriate resources to execute the job are not available.
xterm_args are passed to the xterm(1) executable.
Qlogin is similar to qsh in that it submits an interactive
job to the queueing system. It does not open an xterm(1)
window on the X display, but uses the current terminal for
user I/O. Usually, qlogin establishes a telnet(1) connection
with the remote host, using standard client- and server-side
commands. These commands can be configured with the
qlogin_daemon (server-side, Grid Engine telnetd if not set,
otherwise something like /usr/sbin/in.telnetd) and
qlogin_command (client-side, Grid Engine telnet if not set,
otherwise something like /usr/bin/telnet) parameters in the
global and local configuration settings of sge_conf(5). The
client side command is automatically parameterized with the
remote host name and port number to connect to (i.e. result-
ing in an invocation like /usr/bin/telnet my_exec_host
2442). Qlogin is invoked exactly like qsh and its jobs can
only run on INTERACTIVE queues. Qlogin jobs can only be
used if the sge_execd(8) is running under the root account.
Qrsh is similar to qlogin in that it submits an interactive
job to the queuing system. It uses the current terminal for
user I/O. Usually, qrsh establishes a rsh(1) connection with
the remote host. If no command is given to qrsh, a rlogin(1)
session is established. The server-side commands used can
be configured with the rsh_daemon and rlogin_daemon parame-
ters in the global and local configuration settings of
sge_conf(5). A Grid Engine rshd or rlogind is used, if the
parameters are not set or otherwise something like
/usr/sbin/in.rshd or /usr/sbin/in.rlogind. On the client-
side, the rsh_command and rlogin_command parameters can be
set in the global and local configuration settings of
sge_conf(5). If they are not set, special Grid Engine
rsh(1) and rlogin(1) binaries delivered with Grid Engine are
used. Use the cluster configuration parameters to integrate
mechanisms like ssh or the rsh(1) and rlogin(1) facilities
supplied with the operating system.
Qrsh jobs can only run in INTERACTIVE queues unless the
option -now no is used (see below). They can only be used,
if the sge_execd(8) is running under the root account.
Qrsh provides an additional feature useful for the integra-
tion with interactive tools providing a specific command
shell. If the environment variable QRSH_WRAPPER is set when
qrsh is invoked, the command interpreter pointed to by
QRSH_WRAPPER will be executed to run qrsh commands instead
of the users login shell or any shell specified in the qrsh
command-line.
Qalter can be used to change the attributes of pending jobs.
Once a job is executing, changes are no longer possible. For
job arrays, for which a part of the tasks can be pending and
another part can be running (see the -t option below),
modifications with qalter only affect the pending tasks.
Qalter can change most of the characteristics of a job (see
the corresponding statements in the OPTIONS section below),
including those which were defined as embedded flags in the
script file (see above).
Qresub allows to create jobs as copies from existing pending
or running jobs. The copied jobs will have exactly the same
attributes as the ones from which they are copied, but a new
job ID. The only modification to the copied jobs supported
by qresub is to assign a hold state with the -h option. This
can be used to first copy a job and then change its attri-
butes via qalter.
For qsub, qsh, qrsh, and qlogin the administrator and the
user may define default request files (see sge_request(5))
which can contain any of the options described below. If an
option in a default request file is understood by qsub and
qlogin but not by qsh the option is silently ignored if qsh
is invoked. Thus you can maintain shared default request
files for both qsub and qsh.
A cluster wide default request file may be placed under
$SGE_ROOT/$SGE_CELL/common/sge_request. User private
default request files are processed under the locations
$HOME/.sge_request and $cwd/.sge_request. The working direc-
tory local default request file has the highest precedence,
then the home directory located file and then the cluster
global file. The option arguments, the embedded script
flags and the options in the default request files are pro-
cessed in the following order:
left to right in the script line,
left to right in the default request files,
from top to bottom of the script file (qsub only),
from top to bottom of default request files,
from left to right of the command line.
In other words, the command line can be used to override the
embedded flags and the default request settings. The embed-
ded flags, however, will override the default settings.
Note, that the -clear option can be used to discard any pre-
vious settings at any time in a default request file, in the
embedded script flags, or in a command-line option. It is,
however, not available with qalter.
The options described below can be requested either hard or
soft. By default, all requests are considered hard until
the -soft option (see below) is encountered. The hard/soft
status remains in effect until its counterpart is encoun-
tered again. If all the hard requests for a job cannot be
met, the job will not be scheduled. Jobs which cannot be
run at the present time remain spooled.
OPTIONS
-@ optionfile
Forces qsub, qrsh, qsh, or qlogin to use the options
contained in optionfile. The indicated file may contain
all valid options. Comment lines are starting with a
"#" sign.
-a date_time
Available for qsub and qalter only.
Defines or redefines the time and date at which a job
is eligible for execution. Date_time conforms to
[[CC]]YY]MMDDhhmm.[ss], where:
CC denotes the century in 2 digits.
YY denotes the year in 2 digits.
MM denotes the month in 2 digits.
DD denotes the day in 2 digits.
hh denotes the hour in 2 digits.
mm denotes the minute in 2 digits.
ss denotes the seconds in 2 digits (default 00).
If any of the optional date fields is omitted, the
corresponding value of the current date is assumed.
Usage of this option may cause unexpected results if
the clocks of the hosts in the Grid Engine pool are out
of sync. Also, the proper behavior of this option very
much depends on the correct setting of the appropriate
timezone, e.g. in the TZ environment variable (see
date(1) for details), when the Grid Engine daemons
sge_qmaster(8) and sge_execd(8) are invoked.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-ac variable[=value],...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Adds the given name/value pair(s) to the job's context.
Value may be omitted. Grid Engine appends the given
argument to the list of context variables for the job.
Multiple -ac, -dc, and -sc options may be given. The
order is important here.
Qalter allows changing this option even while the job
executes.
-A account_string
Available for qsub, qsh, qrsh, qlogin and qalter only.
Identifies the account to which the resource consump-
tion of the job should be charged. The account_string
may be any arbitrary ASCII alphanumeric string but may
contain no blank or separator characters. The underbar
'_' is considered a non-separator. In the absence of
this parameter Grid Engine will place the default
account string "sge" in the accounting record of the
job.
Qalter allows changing this option even while the job
executes.
-c occasion_specifier
Available for qsub and qalter only.
Defines or redefines whether the job should be check-
pointed, and if so, under what circumstances. The
specification of the checkpointing occasions with this
option overwrites the definitions of the when parameter
in the checkpointing environment (see checkpoint(5))
referenced by the qsub -ckpt switch. Possible values
for occasion_specifier are
n no checkpoint is performed.
s checkpoint when batch server is shut down.
m checkpoint at minimum CPU interval.
x checkpoint when job gets suspended.
<interval> checkpoint in the specified time interval.
The minimum CPU interval is defined in the queue confi-
guration (see queue_conf(5) for details). <interval>
has to be specified in the format hh:mm:ss. The max-
imum of <interval> and the queue's minimum CPU interval
is used if <interval> is specified. This is done to
ensure that a machine is not overloaded by checkpoints
being generated too frequently.
-ckpt ckpt_name
Available for qsub and qalter only.
Selects the checkpointing environment (see check-
point(5)) to be used for a checkpointing the job. Also
declares the job to be a checkpointing job.
-clear
Available for qsub, qsh, qrsh, and qlogin only.
Causes all elements of the job to be reset to the ini-
tial default status prior to applying any modifications
(if any) appearing in this specific command.
-cwd Available for qsub, qsh, qrsh, qlogin and qalter only.
Execute the job from the current working directory.
This switch will activate Grid Engine's path aliasing
facility, if the corresponding configuration files are
present (see sge_aliases(5)).
In case of qalter, the previous definition of the
current working directory will be overwritten, if
qalter is executed from a different directory than the
preceding qsub or qalter.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-C prefix_string
Available for qsub only.
Prefix_string defines the prefix that declares a direc-
tive to qsub in the job's scriptfile. The prefix is not
a job attribute, but affects the behavior of qsub. If
the -C option is presented with the value of the direc-
tive prefix as a null string, qsub will not scan the
scriptfile
The directive prefix consists of two ASCII characters
which when appearing in the first two bytes of a script
line indicate that what follows is a Grid Engine com-
mand (default is "#$").
The user should be aware that changing the first delim-
iter character can produce unforeseen side effects. If
the script file contains anything other than a "#"
character in the first byte position of the line, the
shell processor for the job will reject the line and
may exit the job prematurely.
If the -C option is present in the script file, it is
ignored.
-dc variable,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Removes the given variable(s) from the job's context.
Multiple -ac, -dc, and -sc options may be given. The
order is important here.
Qalter allows changing this option even while the job
executes.
-display display_specifier
Available for qsh only.
Directs xterm(1) to use display_specifier in order to
contact the X server.
-dl date_time
Available for qsub, qsh, qrsh, qlogin and qalter only.
This option is only supported in case of a Grid Engine
Enterprise Edition system. It is not available for Grid
Engine systems.
Specifies the deadline initiation time in
[[CC]YY]DDhhmm[.SS] format (see -a option above). The
deadline initiation time is the time at which a dead-
line job has to reach top priority to be able to com-
plete within a given deadline. Before the deadline ini-
tiation time the priority of a deadline job will be
raised steadily until it reaches the maximum as config-
ured by the Grid Engine administrator.
This option is applicable for users allowed to submit
deadline jobs only.
-e [hostname:]path,...
Available for qsub and qalter only.
Defines or redefines the path used for the standard
error stream of the job. If the path constitutes an
absolute path name, the error-path attribute of the job
is set to its value including the hostname. If the path
name is relative, Grid Engine expands path either with
the current working directory path in case the -cwd
(see above) switch is also specified or with the home
directory path otherwise. If hostname is present, the
standard error stream will be placed under the
corresponding location if the job runs on the specified
host.
By default the file name for standard error has the
form job_name.ejob_id and job_name.ejob_id.task_id for
job array tasks (see -t option below).
If path is a directory, the standard error stream of
the job will be put in this directory under the default
file name. If the pathname contains certain pseudo
environment variables, their value will be expanded at
runtime of the job and will be used to constitute the
standard error stream path name. The following pseudo
environment variables are supported currently:
$HOME home directory on execution machine
$USER user ID of job owner
$JOB_ID current job ID
$JOB_NAME current job name (see -N option)
$HOSTNAME name of the execution host
$SGE_TASK_ID job array task index number
Alternatively to $HOME the tilde sign "~" can be used
as common in csh(1) or ksh(1). Note, that the "~" sign
also works in combination with user names, so that
"~<user>" expands to the home directory of <user>.
Using another user ID than that of the job owner
requires corresponding permissions, of course.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-hard
Available for qsub, qsh, qrsh, qlogin and qalter only.
Signifies that all resource requirements following in
the command line will be hard requirements and must be
satisfied in full before a job can be scheduled.
As Grid Engine scans the command line and script file
for Grid Engine options and parameters it builds a list
of resources required by a job. All such resource
requests are considered as absolutely essential for the
job to commence. If the -soft option (see below) is
encountered during the scan then all following
resources are designated as "soft requirements" for
execution, or "nice-to-have, but not essential". If the
-hard flag is encountered at a later stage of the scan,
all resource requests following it once again become
"essential". The -hard and -soft options in effect act
as "toggles" during the scan.
-h | -h {u|s|o|n|U|O|S}...
Available for qsub, qsh, qrsh, qlogin, qalter and
qresub.
List of holds to place on the job.
`u' denotes a user hold.
`s' denotes a system hold.
`o' denotes a operator hold.
`n' denotes no hold.
As long as any hold other than `n' is assigned to the
job the job is not eligible for execution. Holds can be
released via qalter and qrls(1). In case of qalter
this is supported by the following additional option
specifiers for the -h switch:
`U' removes a user hold.
`S' removes a system hold.
`O' removes a operator hold.
Grid Engine managers can assign and remove all hold
types, Grid Engine operators can assign and remove user
and operator holds and users can only assign or remove
user holds.
In the case of qsub only user holds can be placed on a
job and thus only the first form of the option with the
-h switch alone is allowed. As opposed to this, qalter
requires the second form described above.
An alternate means to assign hold is provided by the
qhold(1) facility.
If the job is a job array (see the -t option below),
all tasks specified via -t are affected by the -h
operation simultaneously.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-help
Prints a listing of all options.
-hold_jid job_id,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines the job dependency list of the
submitted job. The submitted job is not eligible for
execution unless all jobs referenced in the coma
separated job id list have completed successfully.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-inherit
Available only for qrsh and qmake(1).
qrsh allows to start a task in an already scheduled
parallel job. The option -inherit tells qrsh to read a
job id from the environment variable JOB_ID and start
the specified command as a task in this job. Please
note that in this case, the hostname of the host where
the command shall be executed, must precede the command
to execute; the syntax changes to
qrsh -inherit [ other options ] hostname command [
command_args ]
Note also, that in combination with -inherit, most
other command line options will be ignored. Only the
options -verbose, -v and -V will be interpreted. As a
replacement to option -cwd please use -v PWD.
Usually a task should have the same environment
(including the current working directory) as the
corresponding job, so specifying the option -V should
be suitable for most applications.
Note: If in your system the commd port is not config-
ured as service, but via environment variable
COMMD_PORT, make sure that this variable is set in the
enviroment when calling qrsh or qmake with option
-inherit. If you call qrsh or qmake with option
-inherit from within a job script, export COMMD_PORT
with the submit option or special comment "-v
COMMD_PORT".
-j y|n
Available for qsub and qalter only.
Specifies whether or not the standard error stream of
the job is merged into the standard output stream.
If both the -j y and the -e options are present, Grid
Engine sets, but ignores the error-path attribute.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-l resource=value,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Launch the job in a Grid Engine queue meeting the given
resource request list. In case of qalter the previous
definition is replaced by the specified one.
complex(5) describes how a list of available resources
and their associated valid value specifiers can be
obtained.
There may be multiple -l switches in a single command.
You may request multiple -l options to be soft or hard
both in the same command line. In case of a serial job
multiple -l switches refine the definition for the
sought queue.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-m b|e|a|s|n,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines under which circumstances mail is
to be sent to the job owner or to the users defined
with the -M option described below. The option argu-
ments have the following meaning:
`b' Mail is sent at the beginning of the job.
`e' Mail is sent at the end of the job.
`a' Mail is sent when the job is aborted.
`s' Mail is sent when the job is suspended.
`n' No mail is sent.
Currently no mail is sent when a job is suspended.
For qsh and qlogin mail at the beginning or end of the
job is suppressed when it is encountered in a default
request file.
Qalter allows changing the b, e, and a option arguments
even while the job executes. The modification of the b
option argument will only be in effect after a restart
or migration of the job, however.
-M user[@host],...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines the list of users to which the
server that executes the job has to send mail, if the
server sends mail about the job. Default is the job
owner at the originating host.
Qalter allows changing this option even while the job
executes.
-masterq queue,...
Available for qsub, qrsh, qsh, qlogin and qalter. Only
meaningful for parallel jobs, i.e. together with the
-pe option.
Defines or redefines a list of queues which may be used
to become the so called master queue of this parallel
job. The master queue is defined as the queue where the
parallel job is started. The other queues to which the
parallel job spawns tasks are called slave queues. A
parallel job only has one master queue.
This parameter has all the properties of a resource
request and will be merged with requirements derived
from the -l option described above.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-notify
Available for qsub, qsh, qrsh, qlogin and qalter only.
This flag, when set causes Grid Engine to send "warn-
ing" signals to a running job prior to sending the sig-
nals themselves. If a SIGSTOP is pending the job will
receive a SIGUSR1 several seconds before the SIGSTOP.
If a SIGKILL is pending the job will receive a SIGUSR2
several seconds before the SIGKILL. The amount of time
delay is controlled by the notify parameter in each
queue configuration (see queue_conf(5)).
Note, that the Linux operating system "misuses" the
user signals SIGUSR1 and SIGUSR2 in its current Posix
thread implementation. You might not want to use the
-notify option if you are running threaded applications
in your jobs under Linux.
Qalter allows changing this option even while the job
executes.
-now y[es]|n[o]
Available for qsub, qsh, qlogin and qrsh.
-now y tries to start the job immediately or not at
all. The command returns 0 on success, or 1 on failure
(also if the job could not be scheduled immediately).
-now y is default for qsh, qlogin and qrsh
With option -now n the job will be put into the pending
queue, if it cannot be executed immediately. -now n is
default for qsub.
-N name
Available for qsub, qsh, qrsh, qlogin and qalter only.
The name of the job. The name can be any printable set
of characters.
If the -N option is not present Grid Engine assigns the
name of the job script to the job after any directory
pathname has been removed from the script-name. If the
script is read from standard input the job name
defaults to STDIN.
In case of qsh or qlogin and if the -N option is absent
the string `INTERACT' is assigned to the job.
Qalter allows changing this option even while the job
executes.
-nostdin
Available only for qrsh.
Suppress the input stream STDIN - qrsh will pass the
option -n to the rsh(1) command. This is especially
usefull, if multiple tasks are executed in parallel
using qrsh, e.g. in a make(1) process - it would be
undefined, which process would get the input.
-o [hostname:]path,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
The path used for the standard output stream of the
job. The path is handled as described in the -e option
for the standard error stream.
By default the file name for standard output has the
form job_name.ojob_id and job_name.ojob_id.task_id for
job array tasks (see -t option below).
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-ot override_tickets
Available for qalter only. This option is only sup-
ported in case of a Grid Engine Enterprise Edition sys-
tem. It is not available for Grid Engine systems.
Changes the number of override tickets for the speci-
fied job. Requires manager/operator privileges.
-P project_name
Available for qsub, qsh, qrsh, qlogin and qalter only.
This option is only supported in case of a Grid Engine
Enterprise Edition system. It is not available for Grid
Engine systems.
Specifies the project to which this job is assigned.
The administrator needs to give permission to indivi-
dual users to submit jobs to a specific project. (see
-aprj option to qconf(1)).
-p priority
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines the priority of the job relative
to other jobs. Priority is an integer in the range
-1023 to 1024. The default priority value for the jobs
is 0.
In a Grid Engine system, users may only decrease the
priority of their jobs. Grid Engine managers and
administrators may also increase the priority associ-
ated with jobs. If a pending job has higher priority,
it is earlier eligible for being dispatched by the Grid
Engine scheduler. The job priority has no effect on
running jobs in Grid Engine.
In Grid Engine Enterprise Edition, the job priority
influences the Share Tree Policy and the Functional
Policy. It has no effect on the Deadline and Override
Policies (see share_tree(5), sched_conf(5) and the Grid
Engine Enterprise Edition Installation and for further
information on the resource management policies sup-
ported by Grid Engine Enterprise Edition).
In case of the Share Tree Policy, users can distribute
the tickets, to which they are currently entitled,
among their jobs using different priorities assigned
via -p. If all jobs have the same priority value, the
tickets are distributed evenly. Jobs receive tickets
relative to the different priorities otherwise. Priori-
ties are treated like an additional level in the share
tree in the latter case.
In connection with the Functional Policy, the priority
can be used to weight jobs within the functional job
category. Again tickets are distributed relative to
any uneven priority distribution treated as a virtual
share distribution level underneath the functional job
category.
If both, the Share Tree and the Functional Policy are
active, the job priorities will have an effect in both
policies and the tickets independently derived in each
of them are added up to the total number of tickets for
each job.
-pe parallel_environment n[-[m]]|[-]m,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Parallel programming environment (PE) to instantiate.
The range descriptor behind the PE name specifies the
number of parallel processes to be run. Grid Engine
will allocate the appropriate resources as available.
The sge_pe(5) manual page contains information about
the definition of PEs and about how to obtain a list of
currently valid PEs.
You can specify the PE name by using the wildcard
character "*", thus the request "pvm*" will match any
parallel environment with a name starting with the
string "pvm".
The range specification is a list of range expressions
of the form n-m (n as well as m being positive non-zero
integer numbers), where m is an abbreviation for m-m,
-m is a short form for 1-m and n- is an abbreviation
for n-infinity. The range specification is processed as
follows: The largest number of queues requested is
checked first. If enough queues meeting the specified
attribute list are available, all are allocated. The
next smaller number of queues is checked next and so
forth.
If additional -l options are present, they restrict the
set of eligible queues for the parallel job.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-q queue,...
Available for qsub, qrsh, qsh, qlogin and qalter.
Defines or redefines a list of queues which may be used
to execute this job. This parameter has all the proper-
ties of a resource request and will be merged with
requirements derived from the -l option described
above.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-qs_args ... -qs_end
Available for qsub and qalter only. Deprecated in the
current version.
Valid with the Queuing System Interface (QSI) option
only. Please ask your system administrator.
The options between -qs_args and -qs_end braces are
passed from Grid Engine to a foreign queuing system
which interfaced via the Grid Engine QSI facility.
-r y|n
Available for qsub and qalter only.
Identifies the ability of a job to be rerun or not. If
the value of -r is 'y', rerun the job if the job was
aborted without leaving a consistent exit state (this
is typically the case if the node on which the job is
running crashes). If -r is 'n', do not rerun the job
under any circumstances.
Interactive jobs submitted with qsh or qlogin are not
re-runable.
Qalter allows changing this option even while the job
executes.
-t n[-m[:s]]
Available for qsub and qalter only.
Submits a so called Job Array, i.e. an array of identi-
cal tasks being only differentiated by an index number
and being treated by Grid Engine almost like a series
of jobs. The option argument to -t specifies the number
of job array tasks and the index number which will be
associated with the tasks. The index numbers will be
exported to the job tasks via the environment variable
SGE_TASK_ID.
The task id range specified in the option argument may
be a single number, a simple range of the form n-m or a
range with a step size. Hence, the task id range speci-
fied by 2-10:2 would result in the task id indexes 2,
4, 6, 8, and 10, i.e. in a total of 5 tasks identical
tasks with the environment variable SGE_TASK_ID con-
taining one of the 5 index numbers each.
All job array tasks inherit the same resource requests
and attribute definitions as specified in the qsub or
qalter command line, except for the -t option. The
tasks are scheduled independently and, provided enough
resources, concurrently very much like separate jobs.
However, a job array or a sub-array thereof can be
accessed as a total by commands like qmod(1) or
qdel(1). See the corresponding manual pages for
further detail.
Job arrays are commonly used to execute the same type
of operation on varying input data sets correlated with
the task index number. The number of tasks in a job
array is unlimited.
STDOUT and STDERR of job array tasks will be written
into different files with the default location
<jobname>.['e'|'o']<job_id>'.'<task_id>
In order to change this default, the -e and -o options
(see above) can be used together with the pseudo
environment variables $HOME, $USER, $JOB_ID, $JOB_NAME,
$HOSTNAME, and $SGE_TASK_ID.
Note, that you can use the output redirection to divert
the output of all tasks into the same file, but the
result of this is undefined.
-sc variable[=value],...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Sets the given name/value pairs as the job's context.
Value may be omitted. Grid Engine replaces the job's
previously defined context with the one given as the
argument. Multiple -ac, -dc, and -sc options may be
given. The order is important here.
Contexts are a way to dynamically attach and remove
meta-information to and from a job. The context vari-
ables are not passed to the job's execution context in
its environment.
Qalter allows changing this option even while the job
executes.
-soft
Available for qsub, qsh, qrsh, qlogin and qalter only.
Signifies that all resource requirements following in
the command line will be soft requirements and are to
be filled on an "as available" basis.
As Grid Engine scans the command line and script file
for Grid Engine options and parameters it builds a list
of resources required by a job. All such resource
requests are considered as absolutely essential for the
job to commence. If the -soft option is encountered
during the scan then all following resources are desig-
nated as "soft requirements" for execution, or "nice-
to-have, but not essential". If the -hard flag (see
above) is encountered at a later stage of the scan, all
resource requests following it once again become
"essential". The -hard and -soft options in effect act
as "toggles" during the scan.
-S [host:]pathname,...
Available for qsub, qsh, qlogin and qalter.
Specifies the interpreting shell for the job. Only one
pathname component without a host specifier is valid
and only one path name for a given host is allowed.
Shell paths with host assignments define the interpret-
ing shell for the job if the host is the execution
host. The shell path without host specification is used
if the execution host matches none of the hosts in the
list.
Furthermore, the pathname can be constructed with
pseudo environment variables as described for the -e
option above.
In the case of qsh the specified shell path is used to
execute the corresponding command interpreter in the
xterm(1) (via its -e option) started on behalf of the
interactive job.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-u username,... | -uall
Available for qalter only. Changes are only made on
those jobs which were submitted by users specified in
the list of usernames. For managers it is possible to
use the qalter -uall command to modify all jobs of all
users.
If you use the -u or -uall switch it is not permitted
to specify an additional job/task_id_list.
-v variable[=value],...
Available for qsub, qsh, qrsh, qlogin, qalter and
qresub.
Defines or redefines the environment variables to be
exported to the execution context of the job. If the
-v option is present Grid Engine will add the environ-
ment variables defined as arguments to the switch and,
optionally, values of specified variables, to the exe-
cution context of the job.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
-verbose
Available only for qrsh and qmake(1).
Unlike qsh and qlogin, qrsh does not output any infor-
mational messages while establishing the session com-
pliant with the standard rsh(1) and rlogin(1) system
calls. If the option -verbose is set, qrsh behaves as
verbose as the qsh and qlogin commands and outputs
informations about the process of establishing the
rsh(1) or rlogin(1) session.
-verify
Available for qsub, qsh, qrsh, qlogin, qalter and
qresub.
Does not submit a job but prints information on the job
as being represented by the current command-line and
all pertinent external influences.
-V Available for qsub, qsh, qrsh, qlogin, qalter and
qresub.
Specifies that all environment variables active within
the qsub utility be exported to the context of the job.
-w e|w|n|v
Available for qsub, qsh, qrsh, qlogin, qalter and
qresub.
Specifies a validation level applied to the job to be
submitted (qsub, qlogin, and qsh) or the specified
queued job (qalter). The information displayed indi-
cates whether the job possibly can be scheduled assum-
ing an empty system with no other jobs. Resource
requests exceeding the configured maximal thresholds or
requesting unavailable resource attributes are possible
causes for jobs to fail this validation.
The specifiers e, w, n and v define the following vali-
dation modes:
`e' error - jobs with invalid requests will be
rejected; the default for qrsh, qsh
and qlogin.
`w' warning - only a warning will be displayed
for invalid requests.
`n' none - switches off validation; the default for
qsub and qalter.
`v' verify - does not submit the job but prints
extensive validation report.
Note, that the necessary checks are performance consum-
ing and hence the checking is switched off by default.
Note also, that the reasons for job requirements being
invalid with respect to resource availability of queues
are displayed in the "-w v" case using the format as
described for the qstat(1) -F option (see description
of Full Format in section OUTPUT FORMATS of the
qstat(1) manual page.
job/task_id_list
Specified by the following form:
job_id[.task_range][,job_id[.task_range],...]
If present, the task_range restricts the effect of the
operation to the job array task range specified as suf-
fix to the job id (see the -t option to qsub(1) for
further details on job arrays).
The task range specifier has the form n[-m[:s]]. The
range may be a single number, a simple range of the
form n-m or a range with a step size.
Instead of job/task_id_list it is possible to use the
keyword 'all' to modify all jobs of the current user.
scriptfile
Available for qsub only.
The job's scriptfile. If not present or if the operand
is the single-character string '-', qsub reads the
script from standard input.
script_args
Available for qsub and qalter only.
Arguments to the job. Not valid if the script is
entered from standard input.
Qalter allows changing this option even while the job
executes. The modified parameter will only be in effect
after a restart or migration of the job, however.
xterm_args
Available for qsh only.
Arguments to the xterm(1) executable, as defined in the
configuration. For details, refer to sge_conf(5)).
ENVIRONMENTAL VARIABLES
SGE_ROOT Specifies the location of the Grid Engine
standard configuration files.
SGE_CELL If set, specifies the default Grid Engine
cell. To address a Grid Engine cell qsub,
qsh, qlogin or qalter use (in the order of
precedence):
The name of the cell specified in the
environment variable SGE_CELL, if it is
set.
The name of the default cell, i.e.
default.
SGE_DEBUG_LEVEL
If set, specifies that debug information
should be written to stderr. In addition the
level of detail in which debug information is
generated is defined.
COMMD_PORT If set, specifies the tcp port on which
sge_commd(8) is expected to listen for com-
munication requests. Most installations will
use a services map entry instead to define
that port.
COMMD_HOST If set, specifies the host on which the
particular sge_commd(8) to be used for Grid
Engine communication of the qsub, qsh, qlogin
or qalter client resides. Per default the
local host is used.
In addition to those environment variables specified to be
exported to the job via the -v or the -V option (see above)
qsub, qsh, and qlogin add the following variables with the
indicated values to the variable list:
SGE_O_HOME the home directory of the submitting client.
SGE_O_HOST the name of the host on which the submitting
client is running.
SGE_O_LOGNAME the LOGNAME of the submitting client.
SGE_O_MAIL the MAIL of the submitting client. This is
the mail directory of the submitting client.
SGE_O_PATH the executable search path of the submitting
client.
SGE_O_SHELL the SHELL of the submitting client.
SGE_O_TZ the time zone of the submitting client.
SGE_O_WORKDIR the absolute path of the current working
directory of the submitting client.
Furthermore, Grid Engine sets additional variables into the
job's environment, as listed below.
ARC The Grid Engine architecture name of the node
on which the job is running. The name is
compiled-in into the sge_execd(8) binary.
SGE_CKPT_ENV Specifies the checkpointing environment (as
selected with the -ckpt option) under which a
checkpointing job executes. Only set for
checkpointing jobs.
SGE_CKPT_DIR Only set for checkpointing jobs. Contains
path ckpt_dir (see checkpoint(5) ) of the
checkpoint interface.
SGE_STDERR_PATH
the pathname of the file to which the stan-
dard error stream of the job is diverted.
Commonly used for enhancing the output with
error messages from prolog, epilog, parallel
environment start/stop or checkpointing
scripts.
SGE_STDOUT_PATH
the pathname of the file to which the stan-
dard output stream of the job is diverted.
Commonly used for enhancing the output with
messages from prolog, epilog, parallel
environment start/stop or checkpointing
scripts.
SGE_JOB_SPOOL_DIR
The directory used by sge_shepherd(8) to
store job related data during job execution.
This directory is owned by root or by a Grid
Engine administrative account and commonly is
not open for read or write access to regular
users.
SGE_TASK_ID The index number of the current job array
task (see -t option above). This is an unique
number in each job array and can be used to
reference different input data records, for
example. This environment variable is not set
for non-array jobs.
ENVIRONMENT The ENVIRONMENT variable is set to BATCH to
identify that the job is being executed under
Grid Engine control.
HOME The user's home directory path from the
passwd(5) file.
HOSTNAME The hostname of the node on which the job is
running.
JOB_ID A unique identifier assigned by the
sge_qmaster(8) when the job was submitted.
The job ID is a decimal integer in the range
1 to 99999.
JOB_NAME The job name, either `INTERACT' for interac-
tive jobs or built from the qsub script
filename, a period, and the digits of the job
ID. This default may be overwritten by the
-N. option.
LAST_HOST The name of the preceding host in case of
migration of a checkpointing job.
LOGNAME The user's login name from the passwd(5)
file.
NHOSTS The number of hosts in use by a parallel job.
NQUEUES The number of queues allocated for the job
(always 1 for serial jobs).
NSLOTS The number of queue slots in use by a paral-
lel job.
PATH A default shell search path of:
/usr/local/bin:/usr/ucb:/bin:/usr/bin
PE The parallel environment under which the job
executes (for parallel jobs only).
PE_HOSTFILE The path of a file containing the definition
of the virtual parallel machine assigned to a
parallel job by Grid Engine. See the descrip-
tion of the $pe_hostfile parameter in
sge_pe(5) for details on the format of this
file. The environment variable is only avail-
able for parallel jobs.
QUEUE The name of the queue in which the job is
running.
REQUEST Available for batch jobs only.
The request name of a job as specified with
the -N switch (see above) or taken as the
name of the job script file.
RESTARTED This variable is set to 1 if a job was res-
tarted either after a system crash or after a
migration in case of a checkpointing job. The
variable has the value 0 otherwise.
SHELL The user's login shell from the passwd(5)
file. Note: This is not necessarily the shell
in use for the job.
TMPDIR The absolute path to the job's temporary
working directory.
TMP The same as TMPDIR; provided for compatibil-
ity with NQS.
TZ The time zone variable imported from
sge_execd(8) if set.
USER The user's login name from the passwd(5)
file.
RESTRICTIONS
There is no controlling terminal for batch jobs under Grid
Engine and any tests or actions on a controlling terminal
will fail. If these operations are in your .login or .cshrc
file, they will possibly cause your job to abort.
Insert the following test before any commands that are not
pertinent to batch jobs in your .login:
if ( $?JOB_NAME) then
echo "Grid Engine spooled job"
exit 0
endif
Don't forget to set your shell's search path in your shell
start-up before this code.
EXAMPLES
The following is the simplest form of a Grid Engine script
file.
=====================================================
#!/bin/csh
a.out
=====================================================
The next example is a more complex Grid Engine script.
=====================================================
#!/bin/csh
# Which account to be charged cpu time
#$ -A santa_claus
# date-time to run, format [[CC]yy]MMDDhhmm[.SS]
#$ -a 12241200
# to run I want 6 or more parallel processes
# under the PE pvm. the processes require
# 128M of memory
#$ -pe pvm 6- -l mem=128
# If I run on dec_x put stderr in /tmp/foo, if I
# run on sun_y, put stderr in /usr/me/foo
#$ -e dec_x:/tmp/foo,sun_y:/usr/me/foo
# Send mail to these users
#$ -M santa@heaven,claus@heaven
# Mail at beginning/end/on suspension
#$ -m bes
# Export these environmental variables
#$ -v PVM_ROOT,FOOBAR=BAR
# The job is located in the current
# working directory.
#$ -cwd
a.out
==========================================================
FILES
$REQUEST.oJID[.TASKID] STDOUT of job #JID
$REQUEST.eJID[.TASKID] STDERR of job
$REQUEST.poJID[.TASKID] STDOUT of par. env. of job
$REQUEST.peJID[.TASKID] STDERR of par. env. of job
$REQUEST.hostsJID[.TASKID] hosts file of par. env. of job
$cwd/.sge_aliases cwd path aliases
$cwd/.sge_request cwd default request
$HOME/.sge_aliases user path aliases
$HOME/.sge_request user default request
<sge_root>/<cell>/common/.sge_aliases
cluster path aliases
<sge_root>/<cell>/common/.sge_request
cluster default request
<sge_root>/<cell>/common/act_qmaster
Grid Engine master host file
SEE ALSO
sge_intro(1), qconf(1), qdel(1), qhold(1), qmod(1), qrls(1),
qstat(1), accounting(5), sge_aliases(5), sge_conf(5),
sge_request(5), sge_pe(5), complex(5).
COPYRIGHT
If configured correspondingly, qrsh and qlogin contain por-
tions of the rsh, rshd, elnet and telnetd code copyrighted
by The Regents of the University of California. Therefore,
the following note applies with respect to qrsh and qlogin:
This product includes software developed by the University
of California, Berkeley and its contributors.
See sge_intro(1) as well as the information provided in
<sge_root>/3rd_party/qrsh and <sge_root>/3rd_party/qlogin
for a statement of further rights and permissions.
Man(1) output converted with
man2html