Shellapi

Shellapi

shell-api

In a nutshell

shell-api is:

  • a library of human-readable bash functions organized as modules

  • a framework for building efficient, scalable, distributable bash apps (Debian package)

  • Genapp, a tool for generating ready-to-go bash app skeletons

  • weak system dependencies (standard linux tools)

Disclaimer

The shell-api developped along with other projects that built progressively upon it, like sumo and arcv. At this stage, the documentation may not be complete. Use cases and examples may also to be found in the source code of the aforementionned tools.

Release history

More details can be checked in the Change log section.

Release tag Revision Log

R1.2.0

267

Release coming along arcv 2.0.0

R1.1.4

211

Release coming along latest bookmake 27/09/26

R1.1.3

188

Release coming along with arcv 1.2.1

R1.1.2

168

Release coming along with arcv 1.2.0

R1.1.1

107

Release coming with hot fix for arcv 1.0.1

R1.1.0

100

First release officially used by an external tool , i.e. arcv revision control tool

R1.0.0

20

First release

Installation

Installing the package shell-api_1.1-3_amd64.deb for example will be installed in /usr/bin/shell-api-1.1-3/.

As Debian system package (requires root rights)
wget https://slashetc.fr/download/shell-api_1.1-1_amd64.deb
sudo dpkg -i shell-api_1.1-1_amd64.deb
As free installation
# Sample, select the target installation folder and desired version
wget https://slashetc.fr/download/shell-api_1.1-1.zip
unzip shell-api_1.1-1.zip

The following table lists all the available versions. You can also click on the link to download instead of using wget:

Debian Package Zip Creation time

shell-api_1.2-0_amd64.deb

shell-api_1.2-0.zip

2026-10-25 18:58:01 +01 CET

shell-api_1.1-5_amd64.deb

shell-api_1.1-5.zip

2026-10-02 22:15:27 +02 CEST

shell-api_1.1-4_amd64.deb

shell-api_1.1-4.zip

2026-10-02 03:05:14 +02 CEST

shell-api_1.1-3_amd64.deb

shell-api_1.1-3.zip

2026-09-07 00:33:03 +02 CEST

shell-api_1.1-2_amd64.deb

shell-api_1.1-2.zip

2026-08-28 23:24:51 +02 CEST

shell-api_1.1-1_amd64.deb

shell-api_1.1-1.zip

2026-07-26 12:03:48 +02 CEST

shell-api_1.1-0_amd64.deb

shell-api_1.1-0.zip

2026-07-23 22:05:29 +02 CEST

shell-api_1.0-0_amd64.deb

shell-api_1.0-0.zip

2026-05-20

Bash modules

Outline

The co-related functionalities are grouped into modules , i.e separate, independent bash source files. The list of functions are gathered at the end of the document.

Module file Macro function

shell-api-core.sh

  • Base framework API

  • Basic type manipulation : String, Integer, Arrays, Date

  • User input

  • File path manipulation and file

  • Terminal display control

  • Color and palette manipulation

  • Environment manipulation

  • Math and conversion functions

shell-api-dev.sh

  • System devices manipulation

  • VERA-related helpers

  • ADB-related helpers

shell-api-multimedia.sh

  • Image manipulation

shell-api-net.sh

  • Network address manipulation and network access

shell-api-packing.sh

  • System packages management

shell-api-sys.sh

  • System process management

  • Screen and desktop manipulation

  • File system helpers

shell-api-yaml.sh

  • YAML file manipulation

shell-api-xslt.sh

  • XSLT transformation helpers

shell-api-dock.sh

  • Wrapper functions for docker commands

shell-api-mongodb.sh

  • Wrapper functions for mongodb commands

shell-api-selftest.sh

  • Framework self-tests

shell-api-yaml.sh

The module shell-api-yaml.sh contains a mix of 2 APIs:

  • a set of functions used to manipulate YAML file with yq. Basically each function call, requires to launch yq. This is suited for rare and sporadic use, unavoidable for the cases where the other inhouse API is not capable of performing the operation

  • a set of functions to support very fast read-in of YAML data, whereby all data are read into a bash map all at once.

The fast inhouse API does not support all YAML syntax and is not capable of writing YAML content.

The inhouse YAML API is mainly achieved by calling YAML__setFile "yourfilepath" true and then invoking YAML__get "youryamlkey" yourvariable

The reason of the inhouse API is not only about speed. There are actually 2 versions yq providing slightly different usage, behaviors and installation methods (APT and snap). Because its use and installation can be cumbersome, e.g. snap is not possible inside a docker container, the inhouse function should be preferred whenever possible, if necessary, extended.

Framework features

Foreword - Practical learning from examples

The following section documents some few typical framework features supplied by the library. There’s also an API documentation. However, some parts of it are still lacking.

As complement, it might be useful to check actual implementations relying on it, for example, the tools sumo, arc and shotplan, which are readable bash scripts.

Instant step-in

An app generated with Genapp app skeketon generator enables an instant step-in as follows:

  • Implement the app-specific behavior in the […​]__main() function

  • Add options in the […​]__options.sh files.

Single dash '-' are for any one-letter option, with an optional value

Double dashes -- are for multi-letter option names, optionally followed by '=' and a value

Dashless value can also be used, typically for commands

  • For multiple dashless arguments which positions are relevant, adapt […​]__parseArgsHandleOptionLessArg() rather than using the options file.

  • Implement the checks related to the parsed options and arguments inside […​]__parseArgs()

  • Implement […​]__cleanup() for system signals handling, if necessary

  • Customize […​]__loadDep() to manage custom dependencies, if necessary

Loading a module (sourcing)

A module is loaded (sourced) using the following syntax, e.g. here loading module shell-api-dev.sh:

eval $_loadm<<<'shell-api-dev.sh'

Basically it "sources" the script into the current one, additionally:

  • it ensures the file is not sourced multiple times

  • sets internal variable SHELL_SRC_NAME to the loaded file for all the timespan the module is being sourced.

Loading a source file of the same app project (sourcing)

Any other source script part of the app project, i.e. located below the source directory, can be loaded (sourced) using the following syntax, e.g. here the loading of source file arcv__vars.sh of the sumo project can be loaded as follows:

eval $_loada<<<'arcv__vars.sh'

This is the equivalent of the following which is less convenient to use:

source "${ARCV__VARS["MYDIR"]}/arcv__vars.sh"

Managed configuration

By default, a configuration file is at the app’s disposal, located in the .config folder of the user’s home directory (letter case is relevant):

$HOME/.config/<Appname>/<appname>.yml

If this file does not exist, a default one is created from the default configuration file generated along with the app (.yml extension) upon first launch.

At startup, the library automatically loads the configuration data and stores it in the bash map named <APPNAME>__VARS[].

The bash map key is the uppercased YAML key where spaces are replaced with underscores. The following examples shows how the read data are mapped:

The configuration line will result in the execution of the following during app initialization:

Configuration file line in <appname>.yml Code executed at loading

a sample parameter: parameter value

<APPNAME>__VARS["A_SAMPLE_PARAMETER"]="parameter value"

The accepted format is here specifically a very restricted subset YAML syntax. Only basic pairs key: value are accepted. If a more complex configuration is required and may have to be put under configuration management, a rule of thumb is to define a configuration parameter in $HOME/.config/<Appname>/<appname>.yml giving the path to the actual app configuration, which may then be YAML or any other format required by the specific app.

Logging

A basic logging API is available. The generated logs are plain text and the log file is located in the .local folder of the user’s home directory (letter case is relevant):

$HOME/.local/<appname>/log.txt

Prior to version 1.2.0, the logs endlessly grew.

Afterwards, logs are circular, with a maximum of 15000 lines. The 2000 last lines are repeated upon wraparound.

Logging levels

There are 4 types of messages which are highlighted with different colors when also displayed on the terminal:

Message type Console output background color

Information

standard output (stdout)

none

Important information

standard output (stdout)

blue

Warning

standard error (stderr)

yellow

Error

standard error (stderr)

red

Debug

standard error (stderr)

green

For apps generated by Genapp app skeketon generator, the debug messages can be activated with the --debug option. This should be reserved for development and debugging purposes.

On-the-fly installation of system packages

If the app which is developped from the generated skeleton requires additional system packages, those can be programmed to be installed automatically on-the-fly, anywhere where required in the source code, by using the built-in _loadDep function.

Sample on-the-fly installation using _loadDep()
    _loadDep "lsb-release"
    _loadDep "gawk@gawk"                    # awk
    _loadDep "parted@parted"                # parted
    _loadDep "gdisk@gdisk"                  # gdisk

The prerequisite is that the app implements the callback […​]__loadDep(). An app generated with Genapp app skeketon generator contains by default the following callback, which attempts to load the package via APT:

loadDep() callback
<Appname>__loadDep()
{
    if ! Args__checkCount ${FUNCNAME[0]} 1 "$#" "Usage: <dependency name>"; then return 1; fi

    # By default, attempts to install an APT package of the passed name
    Pkg__install "$1" "" apt
}

Other package types can be installed, see the following sample code:

Example of system package installation using the Pkg__install function
    # Downloads the package from the passed URL and installs directly via dpkg -i
    Pkg__install "veracrypt-console" "1.23.0" dpkg "$URL"
    # Installs via snap
    Pkg__install "yq" "" snap
    # Installs via gem
    Pkg__install "yq" "" gem
The @ writing is optional, it gives an hint of which tool part of the package is actually used.

For optimisation purposes, automatically installed packages are tracked in the cache file dependencies.yml , located in the .local folder of the user’s home directory (letter case is relevant):

$HOME/.local/<appname>/dependencies.yml

System signals handling

By default, the libray catches the system signals and invokes the user-defined […​]__cleanup() callback, if any, passing on the caught exit code as argument.

Managed temporary files and folders

Temporary files or folders created with File__createTempFile and File__createTempDir can be automatically cleaned up at application exit by calling File__deleteAllTempItems.

Managed XSLT transformations

XSLT__transformToText and XSLT__modifyXML eases XSLT transformation with xsltproc :

  • temporary XSLT file creation from passed arguments

  • automatic formatting of parameter arguments for the xsltproc call

  • automatic generation of XSLT parameters arguments for the xsltproc

  • generation of the suitable XSLT header including identity transform if necessary/

So that the developer can only focus on the core transformation rules;

Genapp app skeketon generator

The Genapp tool generates ready-to-go bash app skeletons.

It also generates a Makefile to perform the following operation:

Table 1. Available Makefile actions

make man

Make and install man page for the tool

make release

Generate both a Debian and a zip package according to VERSION.txt

Managed shell-api Dependency

A symbolic link to the shell api library from which Genapp app skeketon generator was executed will be set up inside the generated app. The user is then free to manage the link, e.g. by changing it to another installed version.

The generated Makefile enables to manage the dependency transparently as follows:

  • The dependency version of shell api will be read out of shell-api/VERSION.txt

  • The build_release target ensures that a link to the shell-api of the matching version installed below /usr/bin will be set up when the debian package of the generated app is installed. This is achieved via the debian/link file.

For this reason, it is recommended to use installation via Debian packages, since the dependencies will be managed consistently across different versions of shell api.

Configuration

By default, a configuration file is at the app’s disposal as described in the shell-api documentation. Here, for the sample 'TestApp' app generated in Example, the configuration file is:

.config/TestApp/testapp.yml

The accepted format is a very restricted subset version of YAML. Only basic pairs key: value are accepted. Please check the shell-api documentation.

Example

Generate an app
$ ./shell-api/genapp TestApp -y --author="Michel Mehl" --email="michel.mehl@slashetc.fr" --desc="A test application" --github-id="michelm33" --root-release-dir="../release"

info generated testapp/VERSION.txt
info generated testapp/CHANGELOG.txt
info generated testapp/EXAMPLES.txt
info generated testapp/LICENSE.txt
info generated testapp/COPYRIGHT.txt
info generated testapp/Makefile
info generated testapp/pack/
info generated testapp/testapp.yml
info generated testapp/testapp
info generated testapp/testapp__vars.sh
info generated testapp/testapp__options.sh
info generated testapp/testapp__help.sh

Genapp has finished.
Execute the app (outputs Hello by default)
$ testapp/testapp
info Creating user's configuration file '/home/michel/.config/TestApp/testapp.yml' from '/home/michel/Data/Data/admin/linux/testapp/testapp.yml'
Hello!
Show default available usage help
$ testapp/testapp -h

Usage: testapp OPTIONS [<sample usage arg>]
or: testapp OPTIONS [<sample usage arg 2>]

OPTIONS:

 --silent      Silent mode
 --verbose     Verbose mode
 --debug       Activate debug logs
 --help, -h    Displays app usage
 -y            Assume 'Yes' when prompted for confirmation
 -n            Assume 'No' when prompted for confirmation
 -v, --version Displays the app version
 --man         Displays the manual page
 --files       Lists all the files used by the app (config, log etc)

Arguments:

 <sample arg>       put your argument short description here. Copy/paste in new line and change for additional ones.
List the configuration files managed by the app
$ ./testapp/testapp --files
/home/michel/.config/TestApp/testapp.yml
/home/michel/.local/testapp/log.txt
/home/michel/.local/testapp/dependencies.yml
Make the release packages
$ cd testapp
$ make release

# ..
# MAKE OUTPUT INTENTIONALY REMOVED FOR THE SAKE OF READABILITY
# ..


$ ls ../release/testapp/
testapp-1.0-0              testapp_1.0-0.zip          testapp_1.0-0_amd64.buildinfo  testapp_1.0-0_amd64.deb
testapp-1.0_0.orig.tar.xz  testapp_1.0-0_amd64.build  testapp_1.0-0_amd64.changes
Make the man page and show man
$ cd testapp
$ make man

help2man is installed
help2man -L en_EN@euro --no-info --section 8 --name "Test suit for sumo" --help-option="--man" --output=testapp.8 ./testapp


Installing man pages and building gzip for /usr/share/man/man8//testapp.8

sudo install -g 0 -o 0 -m 0644 testapp.8 /usr/share/man/man8/
sudo gzip -f /usr/share/man/man8//testapp.8

$ man testapp

TESTAPP(8)                                 System Administration Utilities                                 TESTAPP(8)

NAME
       TestApp - A test application

SYNOPSIS
       testapp OPTIONS [<sample usage arg>]
       testapp OPTIONS [<sample usage arg 2>]

       Arguments:

# REMAINING STANDARD OUTPUT TRACES INTENTIONALLY REMOVED FOR THE SAKE OF READABILITY
# ...
Inspect the copyright file
$ cat testapp/COPYRIGHT.txt
TestApp
Copyright (c) 2026 Test User. All rights reserved.
License terms written down in file LICENSE.txt

API for each module

See API page

Lifecycle

Configuration management

The configuration management of the original software was originally managed on a non-disclosed server with arcv. The original revision log of arcv can be tracked on this page: Full revision log page.

Since version 1.1.2, the full original source code is available on GitHub under LGPL license. Therefore, it will be used as the main source reference for the public version.

The clean release packages and the release source code files are available on GitHub at https://github.com/michelm33/shell-api

Bug tracking

Reported bugs are tracked via the GitHub page https://github.com/michelm33/shell-api

Tests

There’s a bunch of unit tests available in the 'self-test' module. However, many of them were one-shot runs commented out afterwards.

In the long run, the tests should ideally be organized in an automated test hardness, which is run prior to any release.

The API is deemed to be somehow instrinsically tested through the various tools using it extensively for their running, and indireclty via their own automated tests.

LGPL License

The library is under LGPL-3 license.

Supporting this work

You can contribute by enriching yourself the library. You can also support the original author via its donation page Donate & Gifts online page.

Thanks

Probably this list may be completed over time, sorry if I missed anyone!

Appendix

Change log

Release 1.2.0

  • New features:

    • core API: _run_check, _makeCd, _navCd, _backCd, _undefined, Int__nears, Int__not0, Int__dec, Int__sumUpSeries, Int__gt, Int__lt, Float__withinRange, Test__assertDirsShouldNotExist, Test__assertFileSizeGreaterThan, Test__assertNotSameFiles, Math__convertHumanReadableSizeToMiB, Array__prependItems, Array__appendItems, Array__dup, Array__sizeMatch, Map__printKeys, Map__merge, Term__title, User__getLogin, Date__year, Date__date, File__exePath, File__SHA256, File__isExe, File__modificationTimeEpoch, File__writeLines, File__readLines, Str__stretch , Str__align, Str__center, Array__SHA256

    • net API: Net__externalIP, Net__isSSLExpirted

    • dev API: Dev__getBlockDeviceSize, Dev__getAvailableSpace,

    • pack API : DPKG__isPackageFileInstalled to check if a package which name and version is given by the file name is actually installed.

    • mongo API: mongodb__isURL, mongodb__decodeURL, mongodb__probeShell, mongodb__dump, mongodb__restore, mongodb__dumpVersion,

    • docker API: dock__isURL, dock__decodeURL, dock__lastImageId, dock__copy: made the copy bidirectional, from container to system , system to container, or container to another container

    • YAML API: YAML__getBatch, YAML__isVoid, YAML__writeFile

    • NEW 'runapi': tool to execute directly any function defined by the shell api

    • NEW 'testapi' which runs 'runapi' and prints OK if execution returned 0 (success)

    • NEW circular logs: max 15000 lines, with last 2000 taken over in the log,

    • NEW Array__toTextTableExt as extension of Array__toTextTable to pass dynamic parameters like display of title, colors and alignments

  • Fixes:

    • fixed Math__convertSizeToMiB

    • fixed _setColored

    • fixed in Array__toTextTableExt

    • fixed 2xdouble quote error in generated 'build_release' make target (genapp)

    • fixed Str__escapeChar

    • handling shell-api symlink for genapidoc at package installation

  • Improvs/extensions:

    • improved color related functions

    • extended Array__toTextTableExt with line coloring

    • extended Array__print with capability to add prefix and suffix to items

    • extended Int__dec with optional arg to decrement of the specified amount

    • added Str__progressBar

    • some Math__* functions for converting bytes to readable form

    • extended Int__inc with optional number of increments (1 default)

    • improved runapi and testapi to enable passing refs to variable

    • worked out CHANGELOG to adoc format to make it usable in online documentation

    • Extended the temp file and dirs API to be able to specify an alternative root folder for temp item

    • Numerous more functions now documented

Release 1.1.4

  • New features:

    • New module 'dock' implementing wrapper functions for docker commands

    • New module 'mongodb' implementing wrapper functions for mongodb

    • New module 'git' implementing wrapper functions for git commands

    • net API: enhanced Net__checkOpenPort see API doc

    • core API: new functions _run_exit, _monitor , Map__toRecord , File__isAbsPath, Str__same, Int__inc

    • log_status now with a blinking running status and can be used to update end status, enabling to pass on a regular bash return code (0=success) as well alternatively to an allowed predefined keyword (ok, nok…​)

    • Now documented Int functions

    • completed generated app susage to display commands help

  • Fixes:

    • _coloredText not displaying a specified optional bg color (_colorText was correct)

    • multimedia API: Fixed Image__watermark following last changes in sys API.

  • Improvs:

    • More documented functions

Release 1.1.3

  • GenApidoc tool for generating AsciiDoctor files for the online doc

  • Makefile fixes

  • Minor change in API doc

  • Some very few more functions in core

  • Changed man section from 8 to 1

Release 1.1.2

Release 1.1.1

  • Release coming with hot fix for arcv 1.0.1

  • added 'export' target in template genapp makefile for exporting a folder to GitHub

  • added File__deleteAllTempItems() for convenience

  • There could be words in local language appearing in the generated man page (month at end of page)

Release 1.1.0

  • CORE API:

    • Options help now alphabetically ordered

    • added default supported options : --revison, --hash, --version-num

    • added implicit supported config parameter 'default app options', which enables to implicit activate options for each app call

    • various changes and addons log API

    • additional functions (string, array, input,test)

    • _cleanup fixes: now called also on normal app exit, correct caught signal value now forwarded to _cleanup

  • NET API (added):

    • Net__getDeviceName,

    • Net__getDeviceMAC,

    • Net__getMAC.

    • add dynamic dep loading

  • YAML API:

    • strings starting with '#' where removed when appearing as first line of a multiline string,

    • added YAML__isUndefined for convenience.

  • PACK API:

    • DPKG__exists robustness fix,

    • status display when downloading. CORE API:

  • SYS API:

    • OVERVIEW NEW FUNCTIONS:

      • Sys__terminate

      • Input__confirm_high

      • Int__getIntTrail

      • Test__assertSymlinkPath

      • Str__eraseCommonTail

      • Str__toCharArray

      • _log_if, _log_ifnot

      • _optionsN() a better variant than _options() to display actual arg types

      • help function 'xxx__versionnum

      • added an API to auto manage and cleanup temporary dirs and files.

    • ADDONS/IMPROVS:

      • Enabling option combinations (eg -ym for -y -m)

      • Input__cursorSelect: predefined prompt can now be extended using '+' as 1st char,

      • input__cursor : limiting displayed width of menu items to the terminal width and minor robustness fixes;

      • added builtin cfg param 'default app options' which enables to inject default parameters at each app launch;

      • in Input__confirm, added forced input 'no' along the 'yes

      • Input__memsize allowing function to abort on invalid input depending on passed bool (could result in endlessloop)

    • BUGFIXES:

      • fixes: Input__cursorSelect added 9th param to return value of sections greater than 255; fixed the signal trapping and handling ; extended soptions() to work in manpage mode;

      • fixed endless loop in Input__sentence when forced -y is specified, now returning directly default value,

      • fixed option help display in case of a dashless option (command) not requiring '='

      • improved _appendConfig by removing any prior existing config param for not creating duplicates if operation is performed multiple times

  • GENAPP:

    • GENERATED FILES:

      • added default generated option --log;

      • Makefile: cleanup after package creation, added --config param

      • update/extended template app for genapp, various addons

      • completed genapp with 'verbose' option

Release 1.0.0

  • initial version separated from sumo