Command-line Interface

The best way to try the cli and see its capabilities is with the --help option in every subcommand. You can also find here a complete list of the subcommands, options and arguments.

CLI reference

firecrest

CLI for FirecREST

Before running you need to setup the following variables or pass them as required options: - FIRECREST_URL: FirecREST URL - FIRECREST_CLIENT_ID: registered client ID - FIRECREST_CLIENT_SECRET: secret for the client - AUTH_TOKEN_URL: URL for the token request in the authorization server (e.g. https://auth.your-server.com/auth/…/openid-connect/token)

firecrest [OPTIONS] COMMAND [ARGS]...

Options

--config <config>

--version: Show the application’s version and exit.

--firecrest-url <firecrest_url>: Required FirecREST URL.

--client-id <client_id>: Required Registered client ID.

--client-secret <client_secret>: Required Secret for the client.

--token-url <token_url>: Required URL of the token request in the authorization server (e.g. https://auth.com/auth/…/openid-connect/token).

--api-version <api_version>: Set the version of the api of firecrest. By default it will be assumed that you are using version 1.13.1 or compatible. The version is parsed by the packaging library.

-v, --verbose: Enable verbose mode.

--timeout <timeout>: How many seconds to wait for the FirecREST server to send data before giving up.

--auth-timeout <auth_timeout>: How many seconds to wait for the authorization server to send data before giving up.

--debug, --no-debug: Enable debug mode.

--install-completion: Install completion for the current shell.

--show-completion: Show completion for the current shell, to copy it or customize the installation.

Environment variables

FIRECREST_CONFIG: Provide a default for --config

FIRECREST_URL: Provide a default for --firecrest-url

FIRECREST_CLIENT_ID: Provide a default for --client-id

FIRECREST_CLIENT_SECRET: Provide a default for --client-secret

AUTH_TOKEN_URL: Provide a default for --token-url

FIRECREST_API_VERSION: Provide a default for --api-version

cancel

Cancel job

firecrest cancel [OPTIONS] JOB

Options

-s, --system <system>: Required The name of the system.

Arguments

JOB: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

checksum

Calculate the SHA256 (256-bit) checksum

firecrest checksum [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

chmod

Change the file mod bits of a given file according to the specified mode

firecrest chmod [OPTIONS] PATH MODE

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

Arguments

PATH: Required argument

MODE: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

chown

Change the user and/or group ownership of a given file.

If only owner or group information is passed, only that information will be updated.

firecrest chown [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

--owner <owner>: Owner ID for target.

--group <group>: Group ID for target.

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

cp

Copy files

firecrest cp [OPTIONS] SOURCE DESTINATION

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

Arguments

SOURCE: Required argument

DESTINATION: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

download

Download a file

Direct download will download the file to the DESTINATION but it will work only for small files. You can find the maximum size in UTILITIES_MAX_FILE_SIZE by running the parameters command.

External download will return with a link in case of success. The file can be downloaded locally from there without any authentication.

firecrest download [OPTIONS] SOURCE [DESTINATION]

Options

-s, --system <system>: Required The name of the system where the source filesystem belongs to.

--type <transfer_type>

Select type of transfer.

Default:: TransferType.direct
Options:: direct | external

Arguments

SOURCE: Required argument

DESTINATION: Optional argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

file

Determine file type

firecrest file [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

filesystems

Information about the filesystems that are available through FirecREST

firecrest filesystems [OPTIONS]

Options

-s, --system <system>: The name of the system where the filesystems belongs to.

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

head

Display the beginning of a specified file. By default the first 10 lines will be returned. Bytes and lines cannot be specified simultaneously.

You view only files smaller than UTILITIES_MAX_FILE_SIZE bytes. This variable is available in the parameters command.

firecrest head [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

-n, --lines <-]NUM>: Print NUM lines of each of the specified files; with a leading ‘-’, print all but the last NUM lines of each file

-c, --bytes <-]NUM>: Print NUM bytes of each of the specified files; with a leading ‘-’, print all but the last NUM bytes of each file

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

id

Return the identity of the user in the remote system.

firecrest id [OPTIONS]

Options

-s, --system <system>: The name of the system where the id command will run.

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

ls

List directory contents

firecrest ls [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

-a, --show-hidden

Include directory entries whose names begin with a dot (‘.’).

Default:: False

--raw

Print unformatted.

Default:: False

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

mkdir

Create new directories

firecrest mkdir [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

-p

Create intermediate directories as required, equivalent to -p of the unix command.

Default:: False

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

mv

Rename/move files, directory, or symlink at the source_path to the target_path on system’s filesystem

firecrest mv [OPTIONS] SOURCE DESTINATION

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

Arguments

SOURCE: Required argument

DESTINATION: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

parameters

Configurable parameters of FirecREST

firecrest parameters [OPTIONS]

poll

Retrieve information about submitted jobs. This call uses the sacct command

firecrest poll [OPTIONS] [JOBS]...

Options

-s, --system <system>: Required The name of the system.

--start-time <start_time>: Start time (and/or date) of job’s query. Allowed formats are HH:MM[:SS] [AM|PM] or MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] or MM/DD[/YY]-HH:MM[:SS] or YYYY-MM-DD[THH:MM[:SS]].

--end-time <end_time>: End time (and/or date) of job’s query. Allowed formats are HH:MM[:SS] [AM|PM] or MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] or MM/DD[/YY]-HH:MM[:SS] or YYYY-MM-DD[THH:MM[:SS]].

--raw

Print unformatted.

Default:: False

Arguments

JOBS: Optional argument(s)

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

poll-active

Retrieves information about active jobs. This call uses the squeue -u <username> command

firecrest poll-active [OPTIONS] [JOBS]...

Options

-s, --system <system>: Required The name of the system.

--raw

Print unformatted.

Default:: False

Arguments

JOBS: Optional argument(s)

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

reservation

Create, list, update and delete reservations

firecrest reservation [OPTIONS] COMMAND [ARGS]...

create

Create a reservation

firecrest reservation create [OPTIONS] NAME ACCOUNT NUM_NODES NODE_TYPE
                             START_TIME END_TIME

Options

-s, --system <system>: Required The name of the system.

Arguments

NAME: Required argument

ACCOUNT: Required argument

NUM_NODES: Required argument

NODE_TYPE: Required argument

START_TIME: Required argument

END_TIME: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

delete

Delete a reservation

firecrest reservation delete [OPTIONS] NAME

Options

-s, --system <system>: Required The name of the system.

Arguments

NAME: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

list

List all active reservations and their status

firecrest reservation list [OPTIONS]

Options

-s, --system <system>: Required The name of the system.

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

update

Update a reservation

firecrest reservation update [OPTIONS] NAME ACCOUNT NUM_NODES NODE_TYPE
                             START_TIME END_TIME

Options

-s, --system <system>: Required The name of the system.

Arguments

NAME: Required argument

ACCOUNT: Required argument

NUM_NODES: Required argument

NODE_TYPE: Required argument

START_TIME: Required argument

END_TIME: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

rm

Remove directory entries

firecrest rm [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

--force, --no-force: Required Attempt to remove the files without prompting for confirmation, regardless of the file’s permissions.

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

services

Provides information for the services of FirecREST

firecrest services [OPTIONS]

Options

-n, --name <name>: Get information for only one service.

stat

Use the stat linux application to determine the status of a file on the system’s filesystem

firecrest stat [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

-L, --dereference

Follow links.

Default:: False

--raw

Print unformatted.

Default:: False

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

submit

Submit a batch script to the workload manager of the target system

firecrest submit [OPTIONS] JOB_SCRIPT

Options

-s, --system <system>: Required The name of the system.

--account <account>: Charge resources used by this job to specified account.

--local, --no-local

The batch file can be local (default) or on the system’s filesystem.

Default:: True

-e, --env-var <VAR=VALUE>

Environment variable to be exported in the environment where the job script will be submitted (format VAR=VALUE).

Default:

Arguments

JOB_SCRIPT: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

submit-template

Create and submit a job for internal transfers

Possible to stage-out jobs providing the SLURM ID of a production job. More info about internal transfer: https://user.cscs.ch/storage/data_transfer/internal_transfer/

firecrest submit-template [OPTIONS] COMMAND [ARGS]...

cp

Copy file

firecrest submit-template cp [OPTIONS] SOURCE DESTINATION

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

SOURCE: Required argument

DESTINATION: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

mv

Move/rename file

firecrest submit-template mv [OPTIONS] SOURCE DESTINATION

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

SOURCE: Required argument

DESTINATION: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

rm

Remove files

firecrest submit-template rm [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the source filesystem belongs to.

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

rsync

Transfer/synchronize files or directories efficiently between filesystems

firecrest submit-template rsync [OPTIONS] SOURCE DESTINATION

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

--job-name <job_name>: Job name in the script.

--time <time>

Limit on the total run time of the job.

Acceptable time formats ‘minutes’, ‘minutes:seconds’, ‘hours:minutes:seconds’, ‘days-hours’, ‘days-hours:minutes’ and ‘days-hours:minutes:seconds’.

--jobid <jobid>: Transfer data after job with ID JOBID is completed.

--account <account>: Name of the project account to be used in SLURM script. If not set, system default is taken.

Arguments

SOURCE: Required argument

DESTINATION: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

symlink

Create a symbolic link

firecrest symlink [OPTIONS] TARGET LINK_NAME

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

Arguments

TARGET: Required argument

LINK_NAME: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

systems

Provides information for the available systems in FirecREST

firecrest systems [OPTIONS]

Options

-n, --name <name>: Get information for only one system.

tail

Display the end of a specified file. By default the last 10 lines will be returned. Bytes and lines cannot be specified simultaneously.

You view only files smaller than UTILITIES_MAX_FILE_SIZE bytes. This variable is available in the parameters command.

firecrest tail [OPTIONS] PATH

Options

-s, --system <system>: Required The name of the system where the filesystem belongs to.

-n, --lines <+]NUM>: Output the last NUM lines; or use +NUM to output starting with line NUM

-c, --bytes <+]NUM>: Output the last NUM bytes; or use +NUM to output starting with byte NUM

Arguments

PATH: Required argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

tasks

Retrieve information about the FirecREST tasks of the users

firecrest tasks [OPTIONS] [TASKIDS]...

Options

--pager, --no-pager

Display the output in a pager application.

Default:: True

Arguments

TASKIDS: Optional argument(s)

upload

Upload a file

Direct upload will upload the file to the DESTINATION directory but it will work only for small files. You can find the maximum size in UTILITIES_MAX_FILE_SIZE by running the parameters command.

External download will return with a command that will need to be run in case of success. The file can be uploaded to a stage area without any authentication and FirecREST will move the file to the cluster’s filesystem as soon as it finished.

firecrest upload [OPTIONS] SOURCE DESTINATION_DIRECTORY [FILENAME]

Options

-s, --system <system>: Required The name of the system where the source filesystem belongs to.

--type <transfer_type>

Select type of transfer.

Default:: TransferType.direct
Options:: direct | external

Arguments

SOURCE: Required argument

DESTINATION_DIRECTORY: Required argument

FILENAME: Optional argument

Environment variables

FIRECREST_SYSTEM: Provide a default for -s

whoami

Return the username that FirecREST will be using to perform the other calls. If the name of the system is not passed the username will be deduced from the token.

firecrest whoami [OPTIONS]

Options

-s, --system <system>: The name of the system where the whoami command will run.

Environment variables

FIRECREST_SYSTEM: Provide a default for -s