makeutils - support for the make

makeinstall: NEWS

mkdist.sh: mkdist -h

usage: mkdist pkgname

This will make pkgname.tar.bz2 from contents of current directory and will put it in ../tgz directory or in .. directory if ../tgz is not available.

If Makefile is present in current directory make clean will be run prior to making dist file.

---

install2report: install2report -h

Name

install2report - analyzes output of make install

Usage

install2report [OPTIONS] LOGFILE

Description

Read the LOGFILE from make install, parse it, remove junk, highlight interesting stuff, and write the report to stdout.

To create the LOGFILE, use for instance:

make install 2>&1 | tee -a LOGFILE

Options

-h

This help.

-d

Debug.

-o

FILE Write output (report without highlights) to the FILE.

-b

BASENAME The package base name to be used in analysis. If not provided, any string will be considered a valid basename.

See also

man tee

Version

install2report.0.2 © R.Jaksa 2011, GPLv3

---

unimakefile: NEWS

unimakefile: README

Name

unimakefile - universal makefile (beta)

Usage

This Makefile should automatically detect structure of package and provide build and install support of it.

Author

R.Jaksa 2009, GPLv3

---

mkdist: NEWS

mkdist: README

Name

mkdist - simple versioning utilities

Usage

This package provides simple versioning support. It provides the functionality to:

1. make .tar.bz2 archive from the current directory by single command without any options provided (if possible),
2. make sub-versioned archives, storing work-in-progress snapshots of the package,
3. provide basic versioning management - recognition of package configuration files with versions defined (perl only for yet), and automatic derivation of version data from the package name,
4. upload of packages to pre-defined site,
5. generation of INDEX file for the html site describing the directory with archived packages, including support for basic presentation of package documentation.

The purpose is to simplify the task of maintaining archive of past versions of software package.

Example

1. We have a package of perl scripts with the VERSION.pl file which defines the $VERSION and $PACKAGE names. By simple call "mkdist", we will make .tar.bz2 archive of package with proper version number. It will be stored in ../tgz directory.
2. Further, if we have the upload site for package defined in /etc/mkdist directory, the package will be uploaded onto this site.
3. By calling "mkdist2html" we will create/update index.html file for this upload site, and upload it itself there.
4. If we have package without any version information defined, we can run "mkdist pkgname-version", thus we will define package name and version directly from command line. This method can also been used from the Makefile.

Context

Package archiving and versioning is a common task in software engineering. Systematic approach is helpful in development, maintenance, and distribution of software. Packaging and versioning can be used not only for software, but for any kind of data.

Creating packages is simple enough task to be pursued manually. Versioning schemes differ, but it is only matter of policy to use some common scheme or define own.

The sub-version archiving is more difficult task, and sophisticated tools are available - the revision control systems. The team work on common subject is the main problem addressed by revision control. Web based revision control services are used frequently.

Problem

The balance between manual packaging and revision control approach is the simplicity of concept versus comfort of usage. Manual packaging is simple to set, but tiresome to exercise. Revision control is more difficult to set, but simplifies everyday usage.

No tools manual packaging is robust. Everybody anytime can do it. But, revision control has a no-tool output too - the final package. However, the process - sub-versions are served by protocol. Protocols are not robust.

Revision control architecture is client-server. Two sides have to be configured - this is far more then zero-configuration manual packaging! But, mentally, in manual packaging we maintain information about status of project - the client, and status of archive the - server.

Isn't it, versioning is one task, and sub-versions another one? Is it orthogonal?

Solution

The server part of manual packaging is the address of web-site. The server part of revision control is management of team work. Unless anything ingenious emerges, we can automatize only the client side of manual packaging, and not enter the team-work area. If we do, we risk sinking into complexity.

The manual approach is the command line interface. This is foundation of robustness. Pursuing the (re)use of command line procedure we will not lose the robustness. Not only on the end but also in the process.

Interpreted scripts are not bad for automatizing the command line sentences.

Version information and package name is often used in package contents itself. We have to reuse this information if it is already available. If not available, this is the main single argument of our commands.

Upload-site data are sensitive information, which is better not stored in the package itself. Also, they are useful only to the package authors, not for users. Therefore the /etc directory is more appropriate place for them.

Single author of distributed software will manage simple to maintain web page for project and simple to maintain software archive. We will support this working scenario and idea of simplicity. Simple web-page is single web-page, the opposite will be complex CMS system or team-mode wiki.

Re use of software documentation for web presentation is: the README file describes the package as whole, the manuals do describe particular programs or other aspects of elements of package.

Lets store sub-versions as full directories packed into single archive and compressed. And lets store them as the single full directory and set of diffs compressed. Because the compression eliminates duplicity, the resulting files in both cases will be equal! However, the full versions packed method is much more simple and direct, although any manipulation with it requires a lot of space.

When storing sub-versions, two schemes of versioning are available: the commonly used "pre" scheme, like 1.2pre1, 1.2pre2, etc. Or the "post" scheme, like 1.2 are stable versions, and sub-versions for 1.4 are 1.3.0, 1.3.1, etc. In general, for any versioning depth (1, 1.2, 1.3.12, 1.0.0.4) the "post" scheme is more straight to implement, as we may not know the future version depth of versioning, which is necessary for proper "pre" sub-versioning.

For single-file programs (or data) it will be nice to pack them as single file too, without directory overhead. It is possible and not difficult.

Installation

1. make
2. PREFIX=/usr make -n install
3. PREFIX=/usr make install

Licence

This package contents can be licensed under the GNU General Public License version 3.

Author

R.Jaksa 2008,2009

---

mkdist: mkdist -hh

Name

mkdist - make distribution archive

Usage

mkdist [OPTIONS] [PKGNAME]

Description

The mkdist does make distribution archive from the content of current directory. The archive file PKGNAME.tar.bz2 will be saved into ../tgz directory, or into .. directory if ../tgz is not available.

The PKGNAME consists of the package name itself, and major and minor package version numbers. In the version number "1.2.13" the major part is "1.2" and minor part is "13". Missing sections of name are autodetected, guessed, or set to default values. Several types of version notation are supported, so try interactively whether particular notation will be recognized.

The package name, project name, the version, and copyleft information is autodetected from configuration files using the getversion utility. See "getversion -h". Also configuration files can be used to supply some values, see "mkdist -hh".

If particular archive "xyz.1.2.13.tar.bz2" does already exists, the subversions archive "xyz.1.2.13.arch.tar.bz2" will be created. This is archive of "1.2.13a", "1.2.13b", "1.2.13c", etc. subversions of "1.2.13", which all do precede next "1.2.14" or "1.3" version.

The mkdist proceeds in three stages: context-detection -> confirmation -> processing. The context-detection stage is passive - no actions on disk are performed, but all harvested information is displayed. Try it with the -q switch. After interactive confirmation of displayed information the processing stage consists of: archive making, and uploading to the web site.

After this, if it is possible, the mkdist2html is called to generate webpage - index file for processed package. Then swnews.html file is updated to indicate newly uploaded package.

Options

-h

Basic help.

-hh

Long help.

-d

Debug.

-u

No web-site upload.

-r

Don't make subversions, rewrite regular archive instead.

-f

Forced noninteractive mode - proceed without questions.

-q

Query mode - no actions, only context detection.

-l

Listing mode - display directories contents.

-bw

Black & white only messages printing.

-2h

Suppress the mkdist2html webpage update.

+2h

Force the mkdist2html webpage update.

Makefile

If the Makefile is present in current directory, make clean is called in advance to packaging.

The recommended way to call the mkdist from inside the Makefile is:

VERSION=1.0.2
PACKAGE=xyz
dist:
mkdist $(PACKAGE)-$(VERSION)

Configuration files

Configuration files are stored in the /map/mkdist/etc or /etc/mkdist directories. The XYZ.cfg is the configuration file for the XYZ package. The ABC.prj is the common configuration file for all packages belonging to the ABC project. The "DEFAULT.cfg" file contains defaults for all packages. First, the defaults are loaded, then project specific settings, then the package specific. Every next setting overrides the previous one. Follows examples of configuration entries - lines in the config file:

project: read-utils
The read-utils is the name of project. This is useful in package specific config to define the project to which the package belongs.

projdesc: utilities to help to read
This config entry is short description of project, used in page generation utilities for the titles.

pkgdesc: the glasses utility
Short description of the package.

obsoleted: obsoleted by view-utils
This string indicates package obsolescence by another package, used for page generation.

site: www.read-utils.org:/html/download
This entry is used to define the site, where to upload webpages and store the web-related data. Up to now only the ssh/scp transfers are supported.

pageup: ../#Download
Used as hyperlink to the parent page.

swnews: www.read-utils.org:/html
This defines where the swnews.html page will be placed. The swnews page is used to report new uploads on the site which hosts several projects.

arch: nord
Whether to upload also archive packages, or only the final ones. The "no" means no uploads, the "nord" means upload, but do not allow download (no read).

nohhtest:
Do not check the long-manual with the -hh switch.

Caveats

The mkdist utilities do expect dark background terminal. Filenames (also package names or directories) with spaces may not work.

In case of error (like: No space left on device, etc.), please examine all the temporary files and the report of the mkdist. Some data might be saved only in intermediate file or backup, which will be rewritten in the next run of mkdist.

Version

mkdist.0.6 © R.Jaksa 2008, GPLv3

---

mkdist: mkdist2html -hh

Name

mkdist2html - index-file maker for mkdist

Usage

mkdist2html [OPTIONS]

Description

The mkdist2html does make html index page for distribution site summarizing its contents. See the mkdist manual for the PACKAGE name definition.

Options

-h

Basic help.

-hh

Long help.

-d

Debug.

-u

No web-site upload.

-bw

Black & white only messages printing.

-nm

PKGNAME Use the PKGNAME as the name of package (not the autodected one).

Configuration files

Configuration files are stored in the /map/mkdist/etc or /etc/mkdist directories. The XYZ.cfg is the configuration file for the XYZ package. The ABC.prj is the common configuration file for all packages belonging to the ABC project. The "DEFAULT.cfg" file contains defaults for all packages. First, the defaults are loaded, then project specific settings, then the package specific. Every next setting overrides the previous one. Follows examples of configuration entries - lines in the config file:

project: read-utils
The read-utils is the name of project. This is useful in package specific config to define the project to which the package belongs.

projdesc: utilities to help to read
This config entry is short description of project, used in page generation utilities for the titles.

pkgdesc: the glasses utility
Short description of the package.

obsoleted: obsoleted by view-utils
This string indicates package obsolescence by another package, used for page generation.

site: www.read-utils.org:/html/download
This entry is used to define the site, where to upload webpages and store the web-related data. Up to now only the ssh/scp transfers are supported.

pageup: ../#Download
Used as hyperlink to the parent page.

swnews: www.read-utils.org:/html
This defines where the swnews.html page will be placed. The swnews page is used to report new uploads on the site which hosts several projects.

arch: nord
Whether to upload also archive packages, or only the final ones. The "no" means no uploads, the "nord" means upload, but do not allow download (no read).

nohhtest:
Do not check the long-manual with the -hh switch.

Caveats

The mkdist utilities do expect dark background terminal. Filenames (also package names or directories) with spaces may not work.

Version

mkdist.0.6 © R.Jaksa 2008, GPLv3

---

getversion: getversion -h

Name

getversion - get version information for package

Usage

getversion [OPTIONS]

Description

The getversion utility auto-detects the version information for the package rooted in current directory. It does so by examination of possible configuration files, or filenames. For not identified values the "unknown" keyword will be used.

Options

-pkg

Print the name of package only.

-ver

Print the version number of package only.

-prj

Print the name of parent project for this package.

-pnm

Print the full package name, including the version number.

-cpy

Print the copyleft/copyright announcement of the package.

-dot

Print the dot character between package name and version.

-h

This help.

-version

The version of this (getversion) utility.

Perl language

For the perl language, files CONFIG.pl and VERSION.pl are examined (in this order). In the case of singlefile package, this single file is examined. Following code is the example of information, we look for:

$PACKAGE="package-name";
$VERSION="1.13";
$PROJECT="project-name";
$COPYLEFT="© The.Author 2009, GPLv3";

C language

In the c language, the VERSION.h file is scanned for the code:

#define PACKAGE "package-name"
#define VERSION "1.13"
#define PROJECT "project-name"
#define COPYLEFT "© The.Author 2009, GPLv3"

Shell scripts

The shell file (the singlefile package) is scanned for:

PACKAGE="package-name"
VERSION="1.13"
PROJECT="project-name"
COPYLEFT="© The.Author 2009, GPLv3"

Version file

In other types of packages the VERSION file can be used. It's syntax is:

PACKAGE: package-name
VERSION: 1.13
PROJECT: project-name
COPYLEFT: The.Author 2009, GPLv3

Head comment

Alternatively a note in head comment of file (a comment in first ten lines of file) can be used. Keywords are Version, Copyleft, Copyright, and © or ©. For instance:

/* Package ABC Version 2.13, Copyleft The.Author 2009, GPLv3 */

Version

getversion.0.4 © R.Jaksa 2009, GPLv3

---

perlpp: NEWS

perlpp: README

Name

perlpp - perl preprocessor

Usage

The perlpp is simplistic perl preprocessor. It allows to use '#include "file.pl"' instead of 'require file.pl'. Like in the C.

The purpose is to manage perl project the C-way: split the code into as many files as you want -> run make -> get single-file result -> install this file into /bin directory.

Context

Perl language is powerful for utilities scripting. Perl is valued as the right tool for short programs, although not so ideal for longer ones. However, for utility scripts usually the short length is an advantage. Short single-file scripted utility, is easiest to understand, and to modify.

Multi-file structure of perl packages is handled with the perl's native 'require' call. This is quite complex mechanism. Perl packages are directories with heterogeneous content: perl-code files, binary modules, configuration files etc. Arbitrary subdirectory structure is allowed. This richness can be achieved thanks to the power of the 'require'.

In the C the transformation from multi-file sources to single-file result is achieved using two tools: the C preprocessor's #include directive, and the linker. Both are simple single-task tools. The structure of sources in C is somewhat limited (compare to the perl). Sources are divided into .h and .c files. Structure of results is left unregulated, so libraries and binary modules sometimes end up on arbitrary places on disk.

Problem

To achieve some functionality of concrete utility script, some concrete minimal length of the script is required. There is a good chance, that this length is bigger then programmer's optimal length of single source file. So, multi-file structure is a must. But, in perl, the source code is the executable code. Once we go multi-file we stay multi-file. Our result becomes a multi-file package. We want multi-file programming, but we want single-file result, instead!

Solution

If the 'make' paradigm is applied to the perl scripting, it can trade off the non-necessity of compilation of perl code, for the ability to freely distribute the source code into files, while still keeping single-file result.

From the C approach we can borrow the preprocessor idea. But, actually, in the role of linker -- for putting the blocks together (usually we don't care about declarations in perl). With a preprocessor, the project can be divided into several .pl files, and glued together using the #include syntax. This nicely matches with perl's # comments.

Licence

The perlpp can be licensed under the GNU General Public License version 3.

Author

R.Jaksa 2008

---

perlpp: perlpp -h

Name

perlpp - perl preprocessor

Usage

perlpp [OPTIONS] FILE [FILE...]

Description

The perlpp does preprocess directives in the perl source code.

If more files are listed, they are concatenated together. Duplicate files are included only once. Files are processed in alphabetical order.

Files are searched for. Recursively in all subdirectories. Lets have files with the same name a.pl in three directories dir1, dir2 and dir3
. We call perlpp a.pl a.pl, while in the file dir1/a.pl we have #include "a.pl". The output will be concatenated dir1/a.pl, dir2/a.pl, dir3/a.pl
.

Directives

Spaces between the sharp # and directives names are allowed, but not mandatory.

# include "FILE"

Includes given FILE into output. It recursively looks for the FILE in all subdirectories of current working directory. Does not include files twice. Quotation marks are mandatory.

# eval $VARIABLE

Defines given VARIABLE according to the environment status of such variable when calling the perlpp. Dollar is mandatory. To define the variable we can use either shell-variable syntax, or regular exported variables:

variable=value perlpp script.pl export variable=value; perlpp script.pl

Then, variable will be included into perl code as a line:

$variable="value";

Debug labels

The constructs #(label)# on the end of line are debug labels. Equally labeled lines from the appendix-file will be added to such line in code. This is useful to unobtrusively add debug or verbose outputs to the code.

1.

If the labeled line has single pair of {} braces, the referenced code from appendix-file will be added just after the opening brace {.

2.

Otherwise, if the labeled line has the closing brace } on the end of line, the referenced code will be placed just before it.

3.

Otherwise the refernced code is placed on the end of line.

TODO

Labeled lines dosomething if condition; transform to if(condition) { referencedcode; dosomething; }.

Appendix-files to be scanned for referenced code have to be specified using -a switch.

Options

-a

FILE Appendix-file to be scanned for LABELS and referenced code.

-c

Remove comments (most of them).

-d

Debug mode.

-h

This help.

-m

Mark includes in output file (using comments).

-p

Input files heuristic prepreprocessing. (as of now, only remove comments on the last line)

-x

Overwrite the input file. The last input file if more then one file is presented. Such a file must not exist in advance.

Version

perlpp.0.5 © R.Jaksa 2008, GPLv3

---

replacer: NEWS

replacer: receipt -h

Name

receipt - analyze replacer receipts

Usage

receipt [OPTIONS] [FILES]

Description

The receipt utility displays content of replacer's receipts. If FILES are provided, then their influence on final receipt is considered too.

First, receipt is loaded from file and raw-parsed. Second, lists are expanded. Third, patterns are ordered from longest to shortest. Next, character trees are built.

If no RECEIPT is prescribed using the -r option, the defaults are tried.

Options

-h

This help.

-r

RECEIPT The path to the receipt file. Several receipts may be combined by multiple used -r option.

-1

Print raw receipt.

-2

Print parsed receipt.

-3

Print receipt with lists expanded.

-4

Print ordered receipt.

-pl

Print list of patterns to search (aka grep -F).

-c

NUMBER Print receipt adjusted for given context number. Without NUMBER specified print the default receipt (context 0), which is used for files which don't belong to any explicite context.

-p

NUMBER Print patterns list, for guiding the 1st fast stage of search for given context. Without NUMBER consider the default context.

-cx

Print contexts, every file belongs to particular search context. We print only contexts applicable to given FILES.

-f

Print files, with corresponding context info.

-i

Print inputs (from command invocation).

Debug strains

receipts Debug the receipt files location.

Receipt file

Comments are lines starting with # on begin of line, and not containing the -> or => symbols.

Basic rule is PATTERN -> REPLACEMENT, it's for simple text by text replace.

Extended rules have => symbol instead of ->. In such rules patterns or replacement strings are expanded - the \n and \t are expanded.

List rules are extended rules with the pattern starting with the "list:" keyword. If colon in keyword is preceded with any symbol, this symbol will be used as list delimiter. Default delimiter is space. After the list keyword a list of space separated patterns follows. Any list of these patterns (delimited with specified delimiter) will be replaced with replacement part of rule.

Files

/dat/replacer/universal.rep First default choice for implicite RECEIPT file is this path. It is tried only if no explicite receipt is prescribed with the -r option. /map/replacer/dat/universal.rep Second choice, tried only if the first choice fails, is this. It is installation dependent path constructed from: DATADIR/universal.rep.

See also

replacer -h

Version

replacer.1.3 © R.Jaksa 2004 GPLv3

---

replacer: replacer -h

Name

replacer - fast automatic multirule replacer

Usage

replacer [OPTIONS] FILES

Description

Replacer replaces text patterns in FILES according to rules from the RECEIPT files. If the specified FILE is a directory, replacer will recursively process whole directory.

If no RECEIPT is prescribed using the -r option, the defaults are tried.

Options

-h

This help.

-d

STRAIN Debug. Without argument turns on all debugging. When STRAIN is specified only matching debugs will be displayed.

-q

Quiet mode.

-f

Forced mode, replace in all, also in already processed files.

-g

Grep mode, don't do any replacing, only look into files.

-r

RECEIPT The path to the receipt file. Several receipts may be combined by multiple used -r option.

-nn

No-numbers mode, don't display line numbers.

-pi

Print also identical patterns (replacement equal to match).

Debug strains

list Lists search verbose/debug messages. lsrch The 2nd stage - linear search verbose/debug messages. receipts Debug the receipt files location.

Files

/dat/replacer/universal.rep First default choice for implicite RECEIPT file is this path. It is tried only if no explicite receipt is prescribed with the -r option. /map/replacer/dat/universal.rep Second choice, tried only if the first choice fails, is this. It is installation dependent path constructed from: DATADIR/universal.rep.

See also

receipt -h

Version

replacer.1.3 © R.Jaksa 2004 GPLv3

---