.. This document is formatted using reStructuredText, which is a Markup

   Syntax and Parser Component of Docutils for Python.  An html version

   of this document can be generated using the following command:

     rst2html.py doc/linked-images.txt > doc/linked-images.html

============================

Ips pkg(5) zones integration

============================

:Author: Edward Pilatowicz

:Version: 0.6

.. sectnum::

.. contents::

Introduction

============

To allow for support of pkg(5) within zones and the automatic management

and upgrading of zones during global zone pkg(5) operations, this

proposal describes enhancements to pkg(5) to support "linked images".

The pkg(5) linked images functionality proposed herein is intended to

be generic, with zones being one type of linked images that can be

supported. In addition to supporting zones linked images we also propose

supporting another "system" type of linked images. The details of how

these linked image types differ will be explained in more detail below.

Another goal of the pkg(5) linked image support is to make all the

linked image functionality visible to other pkg(5) subsystems common,

and to isolate code dealing with the different linked image types within

well defined modules contained within the linked image subsystem. IE,

while other pkg(5) subsystems may need to be aware of linked images,

they should not have to worry about specific linked image types.

(zones, system, etc.)

Linked images will have properties associated with them. The set of

available properties may vary across different linked image types.  The

storage location for these properties values (IE, linked image metadata)

may be plugin specific. For the "system" plugin, property data

configuration will live within a /var/pkg configuration file.  For the

"zones" linked image plugin, property configuration will be derived from

the zones subsystem (some properties will have implicit values, others

may be derived from zonecfg(1m) parameters, etc.)

Zones and linked images

=======================

The design for pkg(5) linked images as proposed herein is primarily

being driven by the need to support zones. Hence, before jumping into

the specifics of linked images support it is worth discussing how zones

user are expected to interact with linked images, and also the

requirements that zones have upon the design of linked images.

Zone users and linked images

----------------------------

Ideally, zone users should be unaware of all linked image functionality

proposed herein.  They should never need to run any of the proposed

pkg(1) linked image commands.  Linked images will do their work

automatically under the hood when users do operations like:

- Run pkg(1) in the global zone.  In this case pkg(1) operations will

  automatically keep non-global zones in sync with the global zone.

- Run pkg(1) in a non-global zone.  In this case pkg(1) will prevent

  operations that would allow a non-global zone to go out of sync with

  the global zone.

- Run zoneadm(1m) install/uninstall/detach/attach/etc.  In this case the

  zone brand callbacks will utilize with the pkg(5) linked image

  functionality to manage zones as linked images.

Zones requirements

------------------

Zones is a OS level virtualization technology that allows us to run

multiple userland execution environment on a single kernel. Each of

these userland execution environments is a separate virtual machine,

with it's own filesystem, processes, network stack, etc. The default

execution environment (IE, the one you get when you install Solaris or

OpenSolaris) is called the global zone.  Subsequent execution

environments are called non-global zones and can be created and managed

via zonecfg(1M) and zoneadm(1M).

All the zones on a system share the same kernel, and the kernel is

managed by the global zone. Non-global zones can not supply their own

kernel modules. Hence, any software within a zone which spans the

user/kernel boundary must be kept in sync with the kernel installed in

the global zone. This puts constraints on what software can be installed

within zones. The basic requirements here can be summed up as:

- Software installed in a non-global zone that depends on specific

  kernel functionality must be kept in sync with the kernel software

  installed within the global zone.  examples:

	libzfs and drv/zfs (system/file-system/zfs)

	libdladm (system/library) and drv/dld (system/kernel)

- Software installed in a non-global that communicates to the global

  zone via private interfaces must be kept in sync with the kernel

  software installed within the global zone.  examples:

	zones proxy (system/zones)

	libdladm (system/library) and dladm (system/network)

- Software that depends on specific kernel functionality can only be

  installed in a non-global zone if the corresponding kernel

  functionality is installed within the global zone.

Since non-global zones are separate virtual machines with their own

filesystems and software, these machines (and their software contents)

may not be trusted by the global zone. Hence any software management

operations being done on non-global zone should not be able to affect

the global zone. Effectively this means that all actions performed on a

zone cannot safely be done from the global zone environment. This means

that software management operations initiated from a global zone will

either have to "enter" the zone (if it's running) to perform their

operation, or they must take special precautions when accessing zone

images to ensure that all filesystem operations are performed safely.

The basic requirements here can be summed up as:

- Software management operations will need to span multiple processes

  operating in different zones.

- Since zones are untrusted execution environment, global zone pkg(5)

  operations should not read data from non-global zone.  IE, any data

  flow required to support linked images must be from a global zone to a

  non-global zone, and never in the reverse direction.  Also, write

  accesses to a non-global zone from the global zone should be kept to

  an absolute minimum, and any such accesses must be provably safe via

  code inspection.

Lastly, since non-global zones are separate virtual machines, they will

normally not have access to the contents of the global zone. Yet the

software that can be run in non-global zones is constrained by the

software installed in the global zone. Hence:

- All the constraints required to perform software management operations

  within a non-global zone must be persistently available within that

  zones image/filesystem.

Zones non-requirements

----------------------

While developing linked images, existing Solaris 10 zones users have

asked how linked images will enable them to continue to do certain zones

administrative operations that they are familiar with.  Unfortunately,

some of these operations were possible mainly as side effects of the

zones and SVR4 packaging implementations on Solaris 10.  Since pkg(5) is

significantly changing the way zones are used and managed, some of these

legacy operations will no longer be possible and users will need to

adopt new ways of managing their zones.  So here are some of the

requests from Solaris 10 users that we're not initially addressing with

linked images.

**The ability to install a patch on all zones on a system.**

Patching of systems with pkg(5) will be substantially different from the

patching of Solaris 10 system.  This change is the administrative model

is being driven by pkg(5) and is largely orthogonal to zones.  With

pkg(5), patching of systems will be done via pkg(1) update, where

repositories are populated with the latests versions of packages that

all system should be running, and when systems are image-updated they

will automatically install the latest versions of software available

within the repositories they are using.  Hence, if an administrator

wants to install updated packages onto a system with zones, they should

push the updated packages into their repositories and update their

zones.

**The ability to install a package on all zones on a system.**

Previously in Solaris 10, the package content of zones closely tracked

the package content of the global zone.  In most cases installing a

package in the global zone would result in that package being installed

in all zones.  With pkg(5) the contents of zones are much more decoupled

from the contents of global zones.  We will not be providing a way to

install packages across multiple zones.  In general, system should only

contain the software that they need for their operation.  Hence, this

software should be installed at zone installation time.  Or if added

later, it needs to be added directly into the zone image (via a pkg(1)

install run within that zone image).

We may subsequently create new mechanisms to help with operations like

the ones above.  Such mechanisms might take the form of recursive

operation support, a simple image content policy mechanism, or some

larger system life cycle management mechanism that defines the package

software content of systems for their entire deployment (instead of just

at install time).

Possible linked image types

===========================

So in addition to supporting zones linked image, the design herein has

also considered other future possible types of linked images. So before

going into the details of the linked image design it's worth while to

first describe some possible linked image types to understand when and

where they might be used.

**Zones linked images**

  Support for zones linked images is included in this proposal. Users

  will not be able to directly create or manage zones linked images.

  Instead the system will automatically manage zones linked images when

  zones are used. Zones linked images should provide us with with a

  means to do the following:

  - Allow for the creation of non-global zone images where the contents

    of those images is constrained by the software installed in the

    global zone.

  - Allow for the updating of software within non-global zones while

    taking into account the constraints placed upon the zone by the

    software installed in the global zone.

  - Allow pkg(5) operations initiated from (and operating on) the global

    zone to update non-global zone images as required to keep them in

    sync with software being updated in the global zone.

  - Allow for pkg(5) operations initiated directly upon non-global zone

    image to take into account the constraints placed upon then by the

    software installed in the global zone.

  - Allow for the auditing of non-global zones to verify that their

    software contents are in sync with the software installed in the

    global zone.

**System linked images**

  Support for system linked images is included in this proposal. These

  types of linked images will be very similar to zone linked images. All

  the features listed above that will be available for zones linked

  images will also be available for system linked images. But unlike

  zone linked images, these images can be created via new pkg(1)

  interfaces.

  Support for system linked images is included in this proposal because

  it is anticipated that system linked images will be used internally

  within Solaris.  Also, having a "system" linked image type will

  greatly facilitate the testing of linked image functionality, the

  large majority of which is common to all linked image types.

**User linked images**

  Support for user linked images is NOT included in this proposal. These

  type of images would be managed very differently from zones or system

  linked images. User linked images could provide us with the following

  functionality:

  - Allow for arbitrary users to create user linked images, where the

    contents of that image are in sync with the software installed in

    another image.

  - Allow for a user to update the software within a user linked image

    while staying in sync with the software installed in another image.

  - Allow for the auditing of a user linked image to verify that their

    software contents are in sync with the software installed in another

    image.

  So here's an example of how a user linked image might work. Say a user

  named Ed wants to run a copy of SpamAssassin on a system named

  jurassic, but jurassic doesn't have SpamAssassin installed. Ed could

  then create a "user" linked image that is linked to the jurassic

  system image. Then he could install SpamAssassin into that image.

  Pkg(5) would install a version of SpamAssassin that is compatible with

  the software contents already installed on jurassic. (In the process

  of doing this pkg(5) would also install into the user image any

  dependencies required by SpamAssassin that were missing from

  jurassic.) Ed could then run this version of SpamAssassin without

  having to worry about his software being out of sync with the system.

  The system administrator would have no knowledge of user images that

  might be created on a system, hence, if the administrator updates the

  software on a system then any user images may now be out of sync. In

  this case, Ed would be able to perform an audit of all his user linked

  images to determine if they are in sync with their specified policy

  and the contents of the system they are being used on. If they were

  out of sync (due to the system being updated) he could then initiate a

  sync operation to update them and bring them back in sync.

**Diskless client linked images**

  Support for diskless client linked images is NOT included in this

  proposal. These types of images would probably not be managed

  directly, but would probably be managed indirectly by diskless client

  management tools. One possible deployment model for diskless clients

  would be to create a parent image which represents the standard

  diskless client configuration deployment, and then to create linked

  images all parented to that one parent image. These linked images

  would actually be the diskless client root filesystems. Subsequent

  software management operations could be performed on the parent image,

  with changes automatically propagated to all the client images

  automatically based on the content policy of the linked images.

**Distributed linked images**

  Support for Distributed linked images is NOT included in this

  proposal.  But if pkg(5) functionality was accessible over the network

  via rad interfaces, it should be possible to create linked image

  relationships that span multiple machines.  This could be used in a

  manager similar to diskless clients where a master deployment image is

  created on one machine, and this master image is then linked to

  machines which deploy the image.  Subsequently, updates to the master

  image would automatically be propagated to deployment machines, based

  of the content management policy being used.

Out of scope

============

There are many components which will probably be required to allow

pkg(5) and zones to work seamlessly together. This proposal does not

address all of them. Specifically, the following features are not being

delivered by this project.

**User, diskless, and distributed linked images.**

  While this project will provide basic support for creating system and

  zone linked images, it will not provide support for any other types of

  linked linked images, even though other types of linked images may be

  discussed within this document.

**Offline zone install.** [1]_

  This proposal does nothing to address the current requirement that an

  active network connection to a valid package repository be available

  during most pkg(5) operations. If anything, this proposal will

  increase the number of operations that may require such a connection

  to be available.

**The system repository.** [2]_

  When managing zone linked images, it's critical that certain packages

  be kept in sync between global and non-global zones. This means that

  zones must have access to any publishers that contain packages that

  must be kept in sync between the images, regardless of the zones

  publisher configuration. Some additional complications to this problem

  are that for zones which are not running, we may not be able to

  instantiate their network configuration, so we may not be able to talk

  to any of their configured network based publishers.  The planned

  solution to address these problems is to create "system repository".

  The system repository would allow a linked image to access any

  publishers configured within the parent image that contain packages

  which needed to be kept in sync with the child image.

  Since delivery of the system repository is out of scope for this

  project, initially zones linked image support will rely on zones

  having a publisher configuration which provides access to all the

  packages required to keep the zone in sync.

**Zones image locking.** [3]_

  When performing pkg(5) operations in a global zone that affect the

  contents of the global zone, we need a locking mechanism to prevent

  concurrent operations from being done within a non-global zone. We

  also need a locking mechanism that allows for the reverse. Ie, if a

  zone is in the middle of a pkg(5) operation we don't want to initiate

  an operation from the global zone which might also require updating

  that zone. Other scenarios that we will probably also need to protect

  against include a zone changing state while we're performing a pkg(5)

  operation on that zone. For example, if we're installing a package in

  a global zone which results in us also installing a package in a

  non-global zone, we need to prevent that non-global zone from

  rebooting while we're installing the package.  Another example is that

  if we're installing or removing a package from a global zone, we will

  probably want to prevent a concurrent install of a non-global zone

  since that could results in the freshly installed zone being out of

  sync with the global zone. This proposal does not address any of the

  possible race conditions wrt pkg(5) and zone linked image operations.

**Pkg(5)/beadm(1M) image filesystem management.** [4]_

  Currently, beadm(1M) allows for versioning (via snapshotting and

  cloning) of multiple filesystem in a global zone image, assuming all

  those filesystems are zfs based and are all children of the current

  root filesystem. beadm(1M) treats any other filesystems in the current

  BE as shared and lofs mounts them into any other alternate BE. This

  means that if any pkg(5) software is installed into these "shared"

  filesystems that software will become out of sync wrt some BE. Pkg(5)

  and beadm(1M) do not perform any check to ensure that all the software

  being installed is not being installed into "shared" filesystems. This

  same problem also affects zones. This proposal does not address this

  issue in any way and assumes that eventually pkg(5) or beadm(1M) will

  provide more flexible and robust functionality to manage images that

  span multiple filesystem.

**Beadm(1M) support for linked images.**

  Currently, beadm(1M) can be run within the global zone to manage

  global zone boot environments and their associated zones. But

  beadm(1M) does not support running within a non-global zone and

  creating snapshots of zone boot environments from within a zone.

  Beadm(1M) support within a zone is a requirements for supporting

  pkg(5) image-update within a zone.  (Since image-update only operates

  on alternate boot environments.) It is also the case that other pkg(5)

  operations may refuse to operate on the currently running image and

  may only be run on cloned and offline boot environments. Since

  beadm(1M) can not be run within a zone, we can't create cloned boot

  environments within a zone, so none of these operations will be

  supported.

  Additionally, beadm(1M) is currently aware of zones images, but

  eventually, beadm(1m) should probably become linked image aware, since

  all linked images should be snapshotted, cloned, and upgraded in sync.

  Enhancing beadm(1M) will be required to support non-zone types of

  linked images that live outside the current boot environment. Once

  this project delivers initial pkg(5) linked image interfaces it will

  be possible to update beadm(1M) to consume these interfaces so that it

  can be aware of all linked images on the system, instead of just zone

  images. These beadm(1M) enhancements are out of the scope of this

  proposal.

pkg(5) linked image overview

============================

pkg(5) linked images will always always have a parent and child

relationship. Parent images may have multiple children, but each child

image may only have one parent. It's possible for there to be multiple

levels of linked images (i.e., nested linked image), for example, you

could have a system linked image, which is a child of a zone linked

image, which is the child of a global zone image.

pkg(5) linked image name

------------------------

All pkg(5) linked images are uniquely identified by a name. A fully

qualified linked image name is <linked image type>:<linked image name>.

The linked image name must begin with an alphanumeric character and can

contain alphanumeric characters, underscores (_), hyphens (-), and dots

(.).  Additional restrictions on the <linked image name> format may be

defined by the linked image plugin handling that type of linked image.

pkg(5) linked image attach mode

-------------------------------

As previously mentioned, all linked images will have a parent/child

relationship.  But there are two distinct ways that this parent/child

link can be established, which in turn determines what operations are

possible on each image and how the linked image relationship is managed.

First, a parent may link/attach a child image to itself, in which case,

the parent image will be authoritative for the linked image

relationship.  This means that:

- The parent image is aware of the child image.

- The child image will know that it is a linked image, but it will

  not know the location of it's parent image.

- Linked image properties can only be modified from the parent image and

  are treated as read-only within the child image.

- Packaging operations on the parent image which require updating child

  images to keep them in sync will attempt to do so automatically.

Second, an image may make itself into a child by linking/attaching

itself to a parent.  In this case the parent has no awareness of the

child image and the child image is authoritative for and must manage the

linked image relationship.  This means that:

- The parent image is unaware of the child image.

- The child image will know that it is a linked image, and it will

  know the location of the parent image.

- Linked image properties only exist within (and there for must be

  modified from within) the child image.

- Packaging operations on the parent image will not directly affect the

  child image.  It is the responsibility of the child image to make sure

  that it remains in sync with it's parent.

In the former image linking mode, the parent must "push" constraint (and

property) information to child images.  While in the latter mode the

child will "pull" constraint information from the parent.

Zones linked images will exclusively use the push linking mode.

System linked images will support both the push and pull linking modes.

pkg(5) linked image properties

------------------------------

As mentioned earlier, each child linked image will have properties

associated with it.

In the case of both push and pull based child images, linked image

property information will always be stored within a private file and

directory in the pkg(5) image metadata directory.  Currently either

/.org.opensolaris,pkg/linked/ or /var/pkg/linked/.

In the case of push based parent images, the property information for

child images is accessible through a plugin for managing each different

type of linked image.  This allows each linked image type to manage the

storage of linked image properties.  In the case of system linked

images, child linked image properties will be stored within the image

configuration.  In the case of Zones linked images, linked image

properties may either be fixed or derived from different zonecfg(1m)

options.

Initially the following properties will be supported:

**li-name**

  This is a linked image name (as described above).

  This property is common to all linked image types, both push and

  pull based.