Command Line¶
ymp¶
Welcome to YMP!
Please find the full manual at https://ymp.readthedocs.io
ymp [OPTIONS] COMMAND [ARGS]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
--version¶ Show the version and exit.
-
--install-completion¶ Install command completion for the current shell. Make sure to have psutil installed.
-
--profile<profile>¶ Profile execution time using Yappi
env¶
Manipulate conda software environments
These commands allow accessing the conda software environments managed by YMP. Use e.g.
>>> $(ymp env activate multiqc)
to enter the software environment for multiqc.
ymp env [OPTIONS] COMMAND [ARGS]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
activate¶
source activate environment
Usage: $(ymp activate env [ENVNAME])
ymp env activate [OPTIONS] ENVNAME
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
Arguments
-
ENVNAME¶ Required argument
clean¶
Remove unused conda environments
ymp env clean [OPTIONS] [ENVNAMES]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-a,--all¶ Delete all environments
Arguments
-
ENVNAMES¶ Optional argument(s)
export¶
Export conda environments
Resolved package specifications for the selected conda
environments can be exported either in YAML format suitable for
use with conda env create -f FILE or in TXT format containing
a list of URLs suitable for use with conda create --file
FILE. Please note that the TXT format is platform specific.
If other formats are desired, use ymp env list to view the
environments’ installation path (“prefix” in conda lingo) and
export the specification with the conda command line utlity
directly.
ymp env export [OPTIONS] [ENVNAMES]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-d,--dest<FILE>¶ Destination file or directory. If a directory, file names will be derived from environment names and selected export format. Default: print to standard output.
-
-f,--overwrite¶ Overwrite existing files
-
-c,--create-missing¶ Create environments not yet installed
-
-s,--skip-missing¶ Skip environments not yet installed
-
-t,--filetype<filetype>¶ Select export format. Default: yml unless FILE ends in ‘.txt’
- Options
yml|txt
Arguments
-
ENVNAMES¶ Optional argument(s)
install¶
Install conda software environments
ymp env install [OPTIONS] [ENVNAMES]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-p,--conda-prefix<conda_prefix>¶ Override location for conda environments
-
-e,--conda-env-spec<conda_env_spec>¶ Override conda env specs settings
-
-n,--dry-run¶ Only show what would be done
-
-f,--force¶ Install environment even if it already exists
Arguments
-
ENVNAMES¶ Optional argument(s)
list¶
List conda environments
ymp env list [OPTIONS] [ENVNAMES]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
--static,--no-static¶ List environments statically defined via env.yml files
-
--dynamic,--no-dynamic¶ List environments defined inline from rule files
-
-a,--all¶ List all environments, including outdated ones.
-
-s,--sort<sort_col>¶ Sort by column
- Options
name|hash|path|installed
-
-r,--reverse¶ Reverse sort order
Arguments
-
ENVNAMES¶ Optional argument(s)
prepare¶
Create envs needed to build target
ymp env prepare [OPTIONS] TARGET_FILES
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-n,--dryrun¶ Only show what would be done
-
-p,--printshellcmds¶ Print shell commands to be executed on shell
-
-k,--keepgoing¶ Don’t stop after failed job
-
--lock,--no-lock¶ Use/don’t use locking to prevent clobbering of files by parallel instances of YMP running
-
--rerun-incomplete,--ri¶ Re-run jobs left incomplete in last run
-
-F,--forceall¶ Force rebuilding of all stages leading to target
-
-f,--force¶ Force rebuilding of target
-
--notemp¶ Do not remove temporary files
-
-t,--touch¶ Only touch files, faking update
-
--shadow-prefix<shadow_prefix>¶ Directory to place data for shadowed rules
-
-r,--reason¶ Print reason for executing rule
-
-N,--nohup¶ Don’t die once the terminal goes away.
Arguments
-
TARGET_FILES¶ Optional argument(s)
remove¶
Remove conda environments
ymp env remove [OPTIONS] [ENVNAMES]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
Arguments
-
ENVNAMES¶ Optional argument(s)
run¶
Execute COMMAND with activated environment ENV
Usage: ymp env run <ENV> [–] <COMMAND…>
- (Use the “–” if your command line contains option type parameters
beginning with - or –)
ymp env run [OPTIONS] ENVNAME [COMMAND]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
Arguments
-
ENVNAME¶ Required argument
-
COMMAND¶ Optional argument(s)
update¶
Update conda environments
ymp env update [OPTIONS] [ENVNAMES]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
--reinstall<reinstall>¶ Remove and reinstall environments rather than trying to update
Arguments
-
ENVNAMES¶ Optional argument(s)
init¶
Initialize YMP workspace
ymp init [OPTIONS] COMMAND [ARGS]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
cluster¶
Set up cluster
ymp init cluster [OPTIONS]
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-y,--yes¶ Confirm every prompt
make¶
Build target(s) locally
ymp make [OPTIONS] TARGET_FILES
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-n,--dryrun¶ Only show what would be done
-
-p,--printshellcmds¶ Print shell commands to be executed on shell
-
-k,--keepgoing¶ Don’t stop after failed job
-
--lock,--no-lock¶ Use/don’t use locking to prevent clobbering of files by parallel instances of YMP running
-
--rerun-incomplete,--ri¶ Re-run jobs left incomplete in last run
-
-F,--forceall¶ Force rebuilding of all stages leading to target
-
-f,--force¶ Force rebuilding of target
-
--notemp¶ Do not remove temporary files
-
-t,--touch¶ Only touch files, faking update
-
--shadow-prefix<shadow_prefix>¶ Directory to place data for shadowed rules
-
-r,--reason¶ Print reason for executing rule
-
-N,--nohup¶ Don’t die once the terminal goes away.
-
-j,--cores<CORES>¶ The number of parallel threads used for scheduling jobs
-
--dag¶ Print the Snakemake execution DAG and exit
-
--rulegraph¶ Print the Snakemake rule graph and exit
-
--debug-dag¶ Show candidates and selections made while the rule execution graph is being built
-
--debug¶ Set the Snakemake debug flag
Arguments
-
TARGET_FILES¶ Optional argument(s)
show¶
Show configuration properties
ymp show [OPTIONS] PROPERTY
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-h,--help¶
-
-s,--source¶ Show source
Arguments
-
PROPERTY¶ Optional argument
stage¶
Manipulate YMP stages
ymp stage [OPTIONS] COMMAND [ARGS]...
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
list¶
List available stages
ymp stage list [OPTIONS] STAGE
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-l,--long¶ Show full stage descriptions
-
-s,--short¶ Show only stage names
-
-c,--code¶ Show definition file name and line number
-
-t,--types¶ Show input/output types
Arguments
-
STAGE¶ Optional argument(s)
submit¶
Build target(s) on cluster
The parameters for cluster execution are drawn from layered profiles. YMP includes base profiles for the “torque” and “slurm” cluster engines.
ymp submit [OPTIONS] TARGET_FILES
Options
-
-P,--pdb¶ Drop into debugger on uncaught exception
-
-q,--quiet¶ Decrease log verbosity
-
-v,--verbose¶ Increase log verbosity
-
--log-file<log_file>¶ Specify a log file
-
-n,--dryrun¶ Only show what would be done
-
-p,--printshellcmds¶ Print shell commands to be executed on shell
-
-k,--keepgoing¶ Don’t stop after failed job
-
--lock,--no-lock¶ Use/don’t use locking to prevent clobbering of files by parallel instances of YMP running
-
--rerun-incomplete,--ri¶ Re-run jobs left incomplete in last run
-
-F,--forceall¶ Force rebuilding of all stages leading to target
-
-f,--force¶ Force rebuilding of target
-
--notemp¶ Do not remove temporary files
-
-t,--touch¶ Only touch files, faking update
-
--shadow-prefix<shadow_prefix>¶ Directory to place data for shadowed rules
-
-r,--reason¶ Print reason for executing rule
-
-N,--nohup¶ Don’t die once the terminal goes away.
-
-P,--profile<NAME>¶ Select cluster config profile to use. Overrides cluster.profile setting from config.
-
-c,--snake-config<FILE>¶ Provide snakemake cluster config file
-
-d,--drmaa¶ Use DRMAA to submit jobs to cluster. Note: Make sure you have a working DRMAA library. Set DRMAA_LIBRAY_PATH if necessary.
-
-s,--sync¶ Use synchronous cluster submission, keeping the submit command running until the job has completed. Adds qsub_sync_arg to cluster command
-
-i,--immediate¶ Use immediate submission, submitting all jobs to the cluster at once.
-
--command<CMD>¶ Use CMD to submit job script to the cluster
-
--wrapper<CMD>¶ Use CMD as script submitted to the cluster. See Snakemake documentation for more information.
-
--max-jobs-per-second<N>¶ Limit the number of jobs submitted per second
-
-l,--latency-wait<T>¶ Time in seconds to wait after job completed until files are expected to have appeared in local file system view. On NFS, this time is governed by the acdirmax mount option, which defaults to 60 seconds.
-
-J,--cluster-cores<N>¶ Limit the maximum number of cores used by jobs submitted at a time
-
-j,--cores<N>¶ Number of local threads to use
-
--args<ARGS>¶ Additional arguments passed to cluster submission command. Note: Make sure the first character of the argument is not ‘-‘, prefix with ‘ ‘ as necessary.
-
--scriptname<NAME>¶ Set the name template used for submitted jobs
Arguments
-
TARGET_FILES¶ Optional argument(s)