As mentioned at the end of part one, this article about creating PackMan packages is going to look at what's necessary to generate distribution index files, ROOL pointer files, and how these tasks can be automated. Towards the end I'll also be taking a look at some options for automating the uploading of the files to your website.
 

Index files and pointer files

Distribution index files

Distribution index files are the mechanism by which the PackMan app learns about packages - each 'distribution' (i.e. collection of package zip files hosted on a website) is accompanied by an index file. This index file contains information about each package; at first glance it looks like it's just a copy of the RiscPkg.Control file from within the package, but there are actually a handful of extra fields that are included within each 'control record':

• Size - The size of the zip file
• MD5Sum - The md5 of the zip file
• URL - The (relative) URL to the zip file

Additionally, by convention the distribution index file should only list the most recent version of each package. It's also common (but not necessary) for the package files to contain the package version as part of their name, e.g. SunEd_2.33-1.zip. This way, although the index only lists the most recent version, you can still host the previous versions on your site if you want or need to.

ROOL pointer files

For the community distribution that's managed by ROOL, ROOL require package mainteners to provide a 'pointer file' containing a simple list of links/URLs to packages. ROOL's server will then regularly fetch this pointer file, fetch all the (changed) packages, and merge everything into the central distribution index file.

What we need is some kind of packaging tool

These rules for distributions, index files, and pointer files mean that the task of preparing package zip files for publishing is too complex to be attempted using simple Obey file or BASIC scripting. OK, if you were only interested in ROOL pointer files you could probably cobble something together without too much effort, but in most cases the use of a 'proper' tool is going to be required.
 
The good news is that to generate a distribution index file, a tool only needs two bits of information about each package: the URL the package is going to be hosted at, and a copy of the zip file itself. Everything else can be determined by examining the contents of the zip. This means that once you've got such a tool for generating index files, it should be very easy to plug it into your publishing pipeline.
 
And the even better news is that I've already written a tool - skopt, a.k.a. Some Kind Of Packaging Tool. Currently its feature set is quite small, but since it appears to be the only tool of its kind in the standard PackMan distributions, it's something that's very much needed if the ecosystem is to continue to grow. Additionally, it's built ontop of the same LibPkg library that RiscPkg/PackMan use internally - helping to simplify the code (just a few hundred lines of C++) and ensure consistent behaviour with PackMan itself.

Functions

The current version of skopt has three modes of operation:

• copy - Copy package file(s) from one location to another. However, unlike a simple *Copy command, this will rename the package zip to the 'canonical' form (i.e. SunEd_2.33-1.zip). It'll also raise an error if it tries to overwrite a package which has the same name but different content (which could indicate that you're trying to publish a new version of a package without increasing the version number).
• gen-binary-index - Scans a list of folders containing (binary) package zip files and generates the corresponding distribution index file. It assumes that the filenames of the packages within the folders will match the names by which the files will be uploaded to your website. However you can specify per-folder URL prefixes.
• gen-pointer-file - Scans a list of folders containing binary packages and generates a ROOL pointer file. As with gen-binary-index, it's assumed that the names of the files will match the names used once uploaded, and URL prefixes are applied on a per-folder basis.

Uploading packages

With skopt managing generating the index file(s) (and optionally package file naming), there's only one piece of the puzzle left: how can you upload the packages to your site?
 
Ignoring the obvious choice of doing it manually, there are at least a few RISC OS-friendly solutions for automation:

!FTPc

!FTPc can be driven programmatically by the use of Wimp messages, and comes with a handy BASIC library to simplify usage. So if you have FTP/FTPS access to your web site, only a few lines of BASIC are needed to upload files or folders.

scp, sftp

If you have SSH access to the web server, then scp (as included in the OpenSSH PackMan package) may be an easy solution for you.
 
The OpenSSH package also includes a copy of the sftp command, which is useful if your web site supports SFTP rather than the older FTP & FTPS protocols. sftp can easily be driven from scripts by the use of the -b option, which will read a series of commands from a text file.

rsync

rsync (also available via PackMan) is another option for copying files via SSH connections (or via the bespoke rsync protocol).

Next time

Uploading binary packages to PackMan distributions is all well and good, but what about the source code? Or what if you want to have regular web pages through which people can also download your software? Stay tuned!