Getting rid of install/upgrade scripting

The design of SVR4 packaging relies heavily on the use of scripting to

achieve common packaging operations.  These scripts are delivered by

the package developers, and run in multiple installation contexts, 

which may include install, alternate roots, zones, cross architecture 

and cross OS versions.  This has caused a host of complex and thorny

issues, and led directly to the IPS team's decision to eliminate the

use of scripting in IPS.

Instead, the IPS architecture requires software to be largely

self-assembling; changes in configuration are either detected at boot

time or the appropriate SMF services are restarted in the case of live

installation of packages.  [In the few cases when this is not possible

due to assembly being needed for boot, the required support is being

built into IPS directly; such cases are actually rare].

Some concrete examples of how such "self-assembly" can be realized may

be worthwhile:

The recent hostid project moved the storage of the hostid information

on i386 architecture machines from the sysinit kernel module (which

was bpatch'd during install) into the /etc/hostid file.  The initial

design for handling upgrade added code to both BFU and the installer,

which decoded the hostid from the sysinit module and created the

hostid file during upgrade.  Freshly installed systems had no hostid

file, so the kernel generated one dyanmically and a SMF service

created the /etc/hostid file after boot.  Since neither bfu nor

install code runs during IPS image-update operations, this strategy

did not work on OpenSolaris and the hostid was not preserved on

upgrade.

To fix this, the kernel was modified to search for a sysinit module

and read the hostid from that if /etc/hostid didn't already exist.

Only if that was not present did the kernel generate a new hostid.  As

a result of this change, the upgrade code could be eliminated from

both BFU and upgrade, and the system upgrade process just works the

same in all cases.

The key design pattern here is to have the consumer of the

configuration file (here, genunix) handle the old file format/location

if the new file doeesn't exist; writing out the new file resolves the

upgrade process and completes the self-assembly.

Another common problem is the configuration file management problem.

A canonical example might be /etc/security/exec_attr.  Here many

different packages contribute lines into this file.  With SVR4

packages, a package's file fragments are merged in by the class action

script i.rbac during installation or upgrade.  Removal is problematic

on uninstall and is not attempted.

For IPS, the proposed solution is to have each package deliver its

portion of this file into a separate directory, using the name of the

package (suitably escaped, of course) as the file name.  A service

that runs at boot determines whether or not the file needs to be

re-composited from the file fragments.  This cleanly handles both

install and uninstall.

The key design pattern here is to have packages deliver their

components as files, which is something the packaging system is

designed to do.  Assembly of the composited file is left to boot time

and an SMF service, which deals with older package fragments, etc.

Another approach to the configuration file problem is to change the

format of the file.  A classic problem is the management of

/etc/driver_aliases, which is "maintained" by add_drv and update_drv.

This file maintains a mapping between a PCI id and the driver that

supports that device.  Note that rather than maintaining a text file

which would be read in by the kernel at boot, an alternate approach

would be to encode the same information as a directory of symbolic

links from the PCI id to the name of the driver.  This would eliminate

the potential confusion of two different drivers trying to own the

same PCI id, the need for locking the file during updates, and the

need to run programs on install and uninstall to manage this file

since the links would be simply installed and uninstalled as part of

the normal packaging operations.

The design pattern here is also simple - the filesystem already

maintains a single namespace, locking, conflict detection, etc;

reusing these attributes saves time and effort.

Another common problem is moving files, links and directories around

the filesystem and between packages; this is often handled via

scripting in the case of SVR4 packages.  In IPS, normal (not editable)

files can simply move between packages; this results in their removal

from the original location and then their re-installation in their new

location.  To insure that editable files retain their customizations,

the receiving package must annotate the actions that deliver the new

file name/location with an attribute that defines the original owning

packages and location. This looks like this:

original_name=SUNWfoo:etc/foo.txt

indicating that the name of original owning package was SUNWfoo and

that the original path was etc/foo.txt.  Note that no matter where

this file moves to subsequently, it should always maintain this

identifier.

Directories in IPS are automatically reference counted, and are

deleted when there are no longer any explicit declarations of

those directories or no files under packaging system control in

the directory.  Left-over files & directories not under packaging 

control are moved to a lost-and-found directory under var/pkg;

this is needed to support changing directories to symbolic links,

etc., w/o scripting.

There is not yet a simple method to move unpackaged files, etc

from an old location to a new one; developers facing this problem

should move those files as part of their self-assembly at first

boot and avoid removing the references to the original directories

to prevent their movement to lost-and-found during upgrade.  

We are considering various solutions to this problem.

In the short term, those developers working on OpenSolaris have a 

foot in both worlds - since OpenSolaris is built from Nevada SVR4

packages, their code needs to work in both environments.  With

the exception of driver post-install scripts (which are converted

by the folks working on OpenSolaris to IPS driver actions) the goal

is to remove all post-install/class action processing other than

SMF service restarting on live systems using the patterns shown

here.