Command-line Interface

The best way to try the cli and see its capabilities is with the --help option in every subcommand. You can also find here a complete list of the subcommands, options and arguments.

CLI reference

firecrest

CLI for FirecREST

Before running you need to setup the following variables or pass them as required options: - FIRECREST_URL: FirecREST URL - FIRECREST_CLIENT_ID: registered client ID - FIRECREST_CLIENT_SECRET: secret for the client - AUTH_TOKEN_URL: URL for the token request in the authorization server (e.g. https://auth.your-server.com/auth/…/openid-connect/token)

firecrest [OPTIONS] COMMAND [ARGS]...

Options

--version: Show the application’s version and exit.

--firecrest-url <firecrest_url>: Required FirecREST URL.

--client-id <client_id>: Required Registered client ID.

--client-secret <client_secret>: Required Secret for the client.

--token-url <token_url>: Required URL of the token request in the authorization server (e.g. https://auth.com/auth/…/openid-connect/token).

--api-version <api_version>: Set the version of the api of firecrest. By default it will be assumed that you are using version 1.13.0 or compatible. The version is parsed by the packaging library.

-v, --verbose: Enable verbose mode.

--timeout <timeout>: How many seconds to wait for the FirecREST server to send data before giving up.

--auth-timeout <auth_timeout>: How many seconds to wait for the authorization server to send data before giving up.

--debug, --no-debug: Enable debug mode.

--install-completion: Install completion for the current shell.

--show-completion: Show completion for the current shell, to copy it or customize the installation.

Environment variables

FIRECREST_URL: Provide a default for --firecrest-url

FIRECREST_CLIENT_ID: Provide a default for --client-id

FIRECREST_CLIENT_SECRET: Provide a default for --client-secret

AUTH_TOKEN_URL: Provide a default for --token-url

FIRECREST_API_VERSION: Provide a default for --api-version

cancel

Cancel job

firecrest cancel [OPTIONS] MACHINE JOB

Arguments

MACHINE: Required argument

JOB: Required argument

checksum

Calculate the SHA256 (256-bit) checksum

firecrest checksum [OPTIONS] MACHINE PATH

Arguments

MACHINE: Required argument

PATH: Required argument

chmod

Change the file mod bits of a given file according to the specified mode

firecrest chmod [OPTIONS] MACHINE PATH MODE

Arguments

MACHINE: Required argument

PATH: Required argument

MODE: Required argument

chown

Change the user and/or group ownership of a given file.

If only owner or group information is passed, only that information will be updated.

firecrest chown [OPTIONS] MACHINE PATH

Options

--owner <owner>: Owner ID for target.

--group <group>: Group ID for target.

Arguments

MACHINE: Required argument

PATH: Required argument

cp

Copy files

firecrest cp [OPTIONS] MACHINE SOURCE DESTINATION

Arguments

MACHINE: Required argument

SOURCE: Required argument

DESTINATION: Required argument

download

Download a file

Direct download will download the file to the DESTINATION but it will work only for small files. You can find the maximum size in UTILITIES_MAX_FILE_SIZE by running the parameters command.

External download will return with a link in case of success. The file can be downloaded locally from there without any authentication.

firecrest download [OPTIONS] MACHINE SOURCE [DESTINATION]

Options

--type <transfer_type>

Select type of transfer.

Default:: TransferType.direct
Options:: direct | external

Arguments

MACHINE: Required argument

SOURCE: Required argument

DESTINATION: Optional argument

file

Determine file type

firecrest file [OPTIONS] MACHINE PATH

Arguments

MACHINE: Required argument

PATH: Required argument

head

Display the beginning of a specified file. By default the first 10 lines will be returned. Bytes and lines cannot be specified simultaneously.

You view only files smaller than UTILITIES_MAX_FILE_SIZE bytes. This variable is available in the parameters command.

firecrest head [OPTIONS] MACHINE PATH

Options

-n, --lines <-]NUM>: Print NUM lines of each of the specified files; with a leading ‘-’, print all but the last NUM lines of each file

-c, --bytes <-]NUM>: Print NUM bytes of each of the specified files; with a leading ‘-’, print all but the last NUM bytes of each file

Arguments

MACHINE: Required argument

PATH: Required argument

ls

List directory contents

firecrest ls [OPTIONS] MACHINE PATH

Options

-a, --show-hidden

Include directory entries whose names begin with a dot (‘.’).

Default:: False

--raw

Print unformatted.

Default:: False

Arguments

MACHINE: Required argument

PATH: Required argument

mkdir

Create new directories

firecrest mkdir [OPTIONS] MACHINE PATH

Options

-p

Create intermediate directories as required, equivalent to -p of the unix command.

Default:: False

Arguments

MACHINE: Required argument

PATH: Required argument

mv

Rename/move files, directory, or symlink at the source_path to the target_path on machine’s filesystem

firecrest mv [OPTIONS] MACHINE SOURCE DESTINATION

Arguments

MACHINE: Required argument

SOURCE: Required argument

DESTINATION: Required argument

parameters

Configurable parameters of FirecREST

firecrest parameters [OPTIONS]

poll

Retrieve information about submitted jobs. This call uses the sacct command

firecrest poll [OPTIONS] MACHINE [JOBS]...

Options

--start-time <start_time>: Start time (and/or date) of job’s query. Allowed formats are HH:MM[:SS] [AM|PM] or MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] or MM/DD[/YY]-HH:MM[:SS] or YYYY-MM-DD[THH:MM[:SS]].

--end-time <end_time>: End time (and/or date) of job’s query. Allowed formats are HH:MM[:SS] [AM|PM] or MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] or MM/DD[/YY]-HH:MM[:SS] or YYYY-MM-DD[THH:MM[:SS]].

--raw

Print unformatted.

Default:: False

Arguments

MACHINE: Required argument

JOBS: Optional argument(s)

poll-active

Retrieves information about active jobs. This call uses the squeue -u <username> command

firecrest poll-active [OPTIONS] MACHINE [JOBS]...

Options

--raw

Print unformatted.

Default:: False

Arguments

MACHINE: Required argument

JOBS: Optional argument(s)

reservation

Create, list, update and delete reservations

firecrest reservation [OPTIONS] COMMAND [ARGS]...

create

Create a reservation

firecrest reservation create [OPTIONS] MACHINE NAME ACCOUNT NUM_NODES
                             NODE_TYPE START_TIME END_TIME

Arguments

MACHINE: Required argument

NAME: Required argument

ACCOUNT: Required argument

NUM_NODES: Required argument

NODE_TYPE: Required argument

START_TIME: Required argument

END_TIME: Required argument

delete

Delete a reservation

firecrest reservation delete [OPTIONS] MACHINE NAME

Arguments

MACHINE: Required argument

NAME: Required argument

list

List all active reservations and their status

firecrest reservation list [OPTIONS] MACHINE

Arguments

MACHINE: Required argument

update

Update a reservation

firecrest reservation update [OPTIONS] MACHINE NAME ACCOUNT NUM_NODES
                             NODE_TYPE START_TIME END_TIME

Arguments

MACHINE: Required argument

NAME: Required argument

ACCOUNT: Required argument

NUM_NODES: Required argument

NODE_TYPE: Required argument

START_TIME: Required argument

END_TIME: Required argument

rm

Remove directory entries

firecrest rm [OPTIONS] MACHINE PATH

Options

--force, --no-force: Required Attempt to remove the files without prompting for confirmation, regardless of the file’s permissions.

Arguments

MACHINE: Required argument

PATH: Required argument

services

Provides information for the services of FirecREST

firecrest services [OPTIONS]

Options

-n, --name <name>: Get information for only one service.

stat

Use the stat linux application to determine the status of a file on the machine’s filesystem

firecrest stat [OPTIONS] MACHINE PATH

Options

-L, --dereference

Follow links.

Default:: False

--raw

Print unformatted.

Default:: False

Arguments

MACHINE: Required argument

PATH: Required argument

submit

Submit a batch script to the workload manger of the target system

firecrest submit [OPTIONS] MACHINE JOB_SCRIPT

Options

--account <account>: Charge resources used by this job to specified account.

--local, --no-local

The batch file can be local (default) or on the machine’s filesystem.

Default:: True

Arguments

MACHINE: Required argument

JOB_SCRIPT: Required argument

submit-template

Create and submit a job for internal transfers

Possible to stage-out jobs providing the SLURM ID of a production job. More info about internal transfer: https://user.cscs.ch/storage/data_transfer/internal_transfer/

firecrest submit-template [OPTIONS] COMMAND [ARGS]...

cp

Copy file

firecrest submit-template cp [OPTIONS] MACHINE SOURCE DESTINATION

Options

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

MACHINE: Required argument

SOURCE: Required argument

DESTINATION: Required argument

mv

Move/rename file

firecrest submit-template mv [OPTIONS] MACHINE SOURCE DESTINATION

Options

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

MACHINE: Required argument

SOURCE: Required argument

DESTINATION: Required argument

rm

Remove files

firecrest submit-template rm [OPTIONS] MACHINE PATH

Options

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

MACHINE: Required argument

PATH: Required argument

rsync

Transfer/synchronize files or directories efficiently between filesystems

firecrest submit-template rsync [OPTIONS] MACHINE SOURCE DESTINATION

Options

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

MACHINE: Required argument

SOURCE: Required argument

DESTINATION: Required argument

symlink

Create a symbolic link

firecrest symlink [OPTIONS] MACHINE TARGET LINK_NAME

Arguments

MACHINE: Required argument

TARGET: Required argument

LINK_NAME: Required argument

systems

Provides information for the available systems in FirecREST

firecrest systems [OPTIONS]

Options

-n, --name <name>: Get information for only one system.

tail

Display the end of a specified file. By default the last 10 lines will be returned. Bytes and lines cannot be specified simultaneously.

You view only files smaller than UTILITIES_MAX_FILE_SIZE bytes. This variable is available in the parameters command.

firecrest tail [OPTIONS] MACHINE PATH

Options

-n, --lines <+]NUM>: Output the last NUM lines; or use +NUM to output starting with line NUM

-c, --bytes <+]NUM>: Output the last NUM bytes; or use +NUM to output starting with byte NUM

Arguments

MACHINE: Required argument

PATH: Required argument

tasks

Retrieve information about the FirecREST tasks of the users

firecrest tasks [OPTIONS] [TASKIDS]...

Options

--pager, --no-pager

Display the output in a pager application.

Default:: True

Arguments

TASKIDS: Optional argument(s)

upload

Upload a file

Direct upload will upload the file to the DESTINATION directory but it will work only for small files. You can find the maximum size in UTILITIES_MAX_FILE_SIZE by running the parameters command.

External download will return with a command that will need to be run in case of success. The file can be uploaded to a stage area without any authentication and FirecREST will move the file to the cluster’s filesystem as soon as it finished.

firecrest upload [OPTIONS] MACHINE SOURCE DESTINATION_DIRECTORY [FILENAME]

Options

--type <transfer_type>

Select type of transfer.

Default:: TransferType.direct
Options:: direct | external

Arguments

MACHINE: Required argument

SOURCE: Required argument

DESTINATION_DIRECTORY: Required argument

FILENAME: Optional argument

whoami

Return the username that FirecREST will be using to perform the other calls. If no machine name is passed the username will be deduced from the token.

firecrest whoami [OPTIONS] [MACHINE]

Arguments

MACHINE: Optional argument