278
|
1 |
Template Version: @(#)onepager.txt 1.31 07/08/08 SMI
|
|
2 |
|
|
3 |
This information is Copyright 2008 Sun Microsystems
|
|
4 |
|
|
5 |
1. Introduction
|
|
6 |
1.1. Project/Component Working Name:
|
|
7 |
|
|
8 |
pkg(5): image packaging system
|
|
9 |
|
|
10 |
1.2. Name of Document Author/Supplier:
|
|
11 |
|
|
12 |
Stephen Hahn, Sun Microsystems,
|
|
13 |
on behalf of the pkg(5) project team
|
|
14 |
|
|
15 |
1.3. Date of This Document:
|
|
16 |
|
|
17 |
03/10/2008
|
|
18 |
|
|
19 |
1.4. Name of Major Document Customer(s)/Consumer(s):
|
|
20 |
1.4.1. The Community you expect to review your project:
|
|
21 |
|
|
22 |
Install and Packaging CG
|
|
23 |
|
|
24 |
1.4.2. The ARC(s) you expect to review your project:
|
|
25 |
|
|
26 |
PSARC
|
|
27 |
|
|
28 |
1.5. Email Aliases:
|
|
29 |
1.5.2. Responsible Engineer:
|
|
30 |
|
|
31 |
[email protected]
|
|
32 |
|
|
33 |
1.5.4. Interest List:
|
|
34 |
|
|
35 |
[email protected]
|
|
36 |
|
|
37 |
2. Project Summary
|
|
38 |
2.1. Project Description:
|
|
39 |
|
|
40 |
The image packaging system, pkg(5), is a portable software
|
|
41 |
packaging and delivery system intended to allow efficient,
|
|
42 |
observable, and controllable transitions between known
|
|
43 |
configurations of software content. pkg(5) will subsume the
|
|
44 |
functionality of the of the packaging and patching utilities
|
|
45 |
included in historical Solaris releases. A primary goal for
|
|
46 |
this project is to improve and extend the usability and
|
|
47 |
functionality of our packaging system.
|
|
48 |
|
|
49 |
The project includes a set of recommended changes to the
|
|
50 |
existing software groupings--the package definitions--in an
|
|
51 |
attempt to produce a more rational and flexible organization of
|
|
52 |
the current components.
|
|
53 |
|
|
54 |
2.2. Risks and Assumptions:
|
|
55 |
|
|
56 |
We intend to preserve the legacy packaging system functionality
|
|
57 |
to support compatibility of existing packages. We believe that,
|
|
58 |
if migration and compatibility practices are made available, the
|
|
59 |
provision of a new packaging mechanism will be followed by
|
|
60 |
adoption.
|
|
61 |
|
|
62 |
We strongly believe that the refactoring and renaming of the
|
|
63 |
existing package graph is not achievable with reasonable cost
|
|
64 |
and duration with the existing packaging/patching/installation
|
|
65 |
software. We also believe compatibility with the existing graph
|
|
66 |
can be preserved, to support the earlier assumption about
|
|
67 |
preserving legacy package operations.
|
|
68 |
|
|
69 |
We also believe that, for the majority of the operating system's
|
|
70 |
development and deployment needs, binary software delivery is
|
|
71 |
preferred over source-based build delivery.
|
|
72 |
|
|
73 |
3. Business Summary
|
|
74 |
3.1. Problem Area:
|
|
75 |
|
|
76 |
Deficits in the current packaging, patching, and installation
|
|
77 |
tool set affect potentially all parties interacting with the
|
|
78 |
historical Solaris releases and successors. Such deficits
|
|
79 |
include:
|
|
80 |
|
|
81 |
- lack of support for dependency-based retrieval during package
|
|
82 |
installation, from one or more network repositories,
|
|
83 |
|
|
84 |
- coarse and incorrect dependencies, limiting use for
|
|
85 |
construction of appliances or other specific-purpose systems,
|
|
86 |
|
|
87 |
- lack of versioning and control over change,
|
|
88 |
|
|
89 |
- forced interactivity,
|
|
90 |
|
|
91 |
- integration with virtualized systems, particularly patching
|
|
92 |
performance,
|
|
93 |
|
|
94 |
- reliance of the installer on hidden information, limiting
|
|
95 |
participation in system upgrade scenarios,
|
|
96 |
|
|
97 |
- lack of safety, specifically around package completeness and
|
|
98 |
alternate package contexts,
|
|
99 |
|
|
100 |
- high developer costs around package and patch creation and
|
|
101 |
maintenance,
|
|
102 |
|
|
103 |
- lack of support for unprivileged package and patch
|
|
104 |
installation,
|
|
105 |
|
|
106 |
- lack of awareness of ZFS and smf(5),
|
|
107 |
|
|
108 |
- late or no correctness checking, and
|
|
109 |
|
|
110 |
- minimal ease of use.
|
|
111 |
|
|
112 |
Additionally, the absence of a portable and efficient
|
|
113 |
cross-platform software delivery system places additional costs
|
|
114 |
upon teams that must deliver software for multiple platforms,
|
|
115 |
such as enterprise middleware vendors.
|
|
116 |
|
|
117 |
3.2. Market/Requester:
|
|
118 |
|
|
119 |
Distribution providers and software content providers have
|
|
120 |
requested substantial changes to the legacy packaging system.
|
|
121 |
The requested changes focus on reducing maintenance costs and
|
|
122 |
increasing development efficiencies.
|
|
123 |
|
|
124 |
Various customers of the historical Solaris release have asked
|
|
125 |
for substantial capabilities not present in the legacy packaging
|
|
126 |
system.
|
|
127 |
|
|
128 |
Finally, multiplatform packaging capabilities are of interest to
|
|
129 |
a number of software content providers.
|
|
130 |
|
|
131 |
3.3. Business Justification:
|
|
132 |
|
|
133 |
See 3.1 and 3.2.
|
|
134 |
|
|
135 |
3.4. Competitive Analysis:
|
|
136 |
|
|
137 |
Every major operating system vendor--and most upcoming new
|
|
138 |
vendors--offers a form of networked software delivery and
|
|
139 |
updates. Well known companies with such technologies are
|
|
140 |
Microsoft, Red Hat, Apple, and Canonical; new companies include
|
|
141 |
rPath. Non-profit entities with equivalent technology include
|
|
142 |
the Debian Project.
|
|
143 |
|
|
144 |
3.5. Opportunity Window/Exposure:
|
|
145 |
|
|
146 |
In order for OpenSolaris-based systems to remain competitive in
|
|
147 |
software delivery functionality, Solaris 10 should be the last
|
|
148 |
Minor release with a packaging system that fails to meet the
|
|
149 |
needs stated in 3.1 and 3.2.
|
|
150 |
|
|
151 |
3.6. How will you know when you are done?:
|
|
152 |
|
|
153 |
In terms of basic capabilities, we can examine each component.
|
|
154 |
Project completion on the retrieval side can be measured by
|
|
155 |
achieving the capability of managing mixed content from a
|
|
156 |
variety of publishers, with potentially distinct entitlement
|
|
157 |
regimes. On the publication side, completion of the initial
|
|
158 |
project is reached once the goals around dependency and
|
|
159 |
correctness checking (and failure handling) are met for both the
|
|
160 |
server and the publication client. Finally, ease of use (or
|
|
161 |
familiarity) must match or exceed that of other leading
|
|
162 |
packaging systems.
|
|
163 |
|
|
164 |
In terms of the product as a whole, we must be able to upgrade,
|
|
165 |
with some statement about limitations on fidelity, a system
|
|
166 |
installed using the legacy packaging components such that it can
|
|
167 |
be further updated using the image packaging system.
|
|
168 |
|
|
169 |
4. Technical Description:
|
|
170 |
4.1. Details:
|
|
171 |
|
|
172 |
pkg(5) is a network-oriented binary packaging system. Although
|
|
173 |
it will have on-disk representations for versioned packages, the
|
|
174 |
primary expected use for installation of software will be
|
|
175 |
between an intelligent client and one or more relatively simple
|
|
176 |
servers.
|
|
177 |
|
|
178 |
The project defines a client-server publication mechanism. The
|
|
179 |
publication client offers up transactions on packages. The
|
|
180 |
server evaluates transactions from the publication client.
|
|
181 |
Transations that are deemed to be complete and/or safe by the
|
|
182 |
server are then made available to the retrieval client.
|
|
183 |
|
|
184 |
The initial transport will be HTTP and HTTPS, protocols around
|
|
185 |
which most sites have developed mature access policies. Support
|
|
186 |
for most common HTTP/HTTPS load-balancing, redirection, and
|
|
187 |
proxying techniques will be implemented, making the system easy
|
|
188 |
to deploy in a variety of scenarios. Additional transports may
|
|
189 |
be investigated during the course of the project or as future
|
|
190 |
work.
|
|
191 |
|
|
192 |
The project does not define a default mechanism for building
|
|
193 |
software as part of the packaging process. The project team
|
|
194 |
believes strongly that software builds are a separate function,
|
|
195 |
and probably also agrees that different kinds of software may
|
|
196 |
require different build techniques.
|
|
197 |
|
|
198 |
More controversially, the project, in an attempt to increase
|
|
199 |
system safety and to reduce developer burden, removes the notion
|
|
200 |
of arbitrary context scripting from packaging. (This removal
|
|
201 |
means that the legacy packaging system must remain on the system
|
|
202 |
for long-term compatibility.) Empirical evidence from the
|
|
203 |
prototype phase has so far borne out this decision.
|
|
204 |
|
|
205 |
4.2. Bug/RFE Number(s):
|
|
206 |
|
|
207 |
As an example of the kinds of defects and RFEs intended to be
|
|
208 |
resolved by this project, we present the following selection of
|
|
209 |
bug IDs from the past 15 years:
|
|
210 |
|
|
211 |
1105830 pkgadd and pkgrm should be able to handle dependency ordering
|
|
212 |
1149607 Package dependencies hidden within a cluster.
|
|
213 |
1165888 RFE: allow non-root users to install software using the package mechanis
|
|
214 |
1184238 patches should be fully managed by package utilites
|
|
215 |
1208431 pkgrm with no arguments defaults to all
|
|
216 |
1249015 pkgadd requires root access
|
|
217 |
4202113 pkginfo command is ridiculously slow
|
|
218 |
4240078 pkgadd should not allow an intel package to install on Sparc and visa-ve
|
|
219 |
4385316 RFE Support pkgadd of clusters
|
|
220 |
4480153 Improvements desired for pkg management
|
|
221 |
4762470 pkgadd: soft dependencies
|
|
222 |
4795539 pkgadd should check dependencies of all packages provided on the command
|
|
223 |
4847723 rem_drv in preremove scripts should have consistent usage model
|
|
224 |
4939605 grep-friendly pkgchk -l variant desired
|
|
225 |
5012345 request for tool to list package dependencies
|
|
226 |
6208580 pkgadd/pkgrm should be smarter about dependancies
|
|
227 |
6246595 Sun's package management needs improvement
|
|
228 |
6491381 Create audit log for packaging and patch commads
|
|
229 |
|
|
230 |
In contrast,
|
|
231 |
|
|
232 |
1181241 wants to split large binary across multiple floppies with pkgmk
|
|
233 |
|
|
234 |
will not be addressed by this proposal.
|
|
235 |
|
|
236 |
4.3. In Scope:
|
|
237 |
|
|
238 |
Package-service delivery and containment relationships.
|
|
239 |
|
|
240 |
Package installation behaviour in virtualized environments.
|
|
241 |
|
|
242 |
4.4. Out of Scope:
|
|
243 |
|
|
244 |
Specific operational scenarios for repositories operated by Sun
|
|
245 |
Microsystems.
|
|
246 |
|
|
247 |
Provision of a GUI/BUI for package management.
|
|
248 |
|
|
249 |
Specific package contents and manifests.
|
|
250 |
|
|
251 |
4.5. Interfaces:
|
|
252 |
|
|
253 |
pkg(5) will present a substantial set of new and modified interfaces
|
|
254 |
to the core system. In particular, documented definitions of
|
|
255 |
|
|
256 |
- retrieval client CLI,
|
|
257 |
|
|
258 |
- publication client CLI,
|
|
259 |
|
|
260 |
- administrative and server CLIs,
|
|
261 |
|
|
262 |
- client metadata representations,
|
|
263 |
|
|
264 |
- server metadata representations,
|
|
265 |
|
|
266 |
- retrieval and publication protocol operations,
|
|
267 |
|
|
268 |
- a dynamic language API to access packaging functions,
|
|
269 |
|
|
270 |
- an on-disk package format,
|
|
271 |
|
|
272 |
- package metadata conventions,
|
|
273 |
|
|
274 |
- available package constituents ("actions"), and
|
|
275 |
|
|
276 |
- package naming and versioning conventions,
|
|
277 |
|
|
278 |
will be presented as interfaces introduced by this project.
|
|
279 |
|
|
280 |
It is possible that some of the nominally private interfaces
|
|
281 |
associated with legacy packaging will be affected; at a minimum,
|
|
282 |
files previously delivered via legacy packaging will no longer
|
|
283 |
be tracked by the legacy system. This outcome could result in a
|
|
284 |
correctly functioning system that presents very differently in
|
|
285 |
terms of file-package membership when interrogated using the
|
|
286 |
legacy packaging API.
|
|
287 |
|
|
288 |
Various components of the project will be introduced at each
|
|
289 |
stability and/or commitment level. The components are being
|
|
290 |
engineered such that the public interfaces can be evolved
|
|
291 |
compatibly, once the initial development is complete. (In fact,
|
|
292 |
the prototype is expected to support this evolution during the
|
|
293 |
development phase.)
|
|
294 |
|
|
295 |
The components are currently implemented in Python
|
|
296 |
(PSARC/2005/532, PSARC/2005/555, PSARC/2006/666).
|
|
297 |
|
|
298 |
4.6. Doc Impact:
|
|
299 |
|
|
300 |
The project expects to provide reference manual pages for each
|
|
301 |
of the groups of interface identified above. Furthermore, the
|
|
302 |
project expects to provide a Developer's Guide to replace the
|
|
303 |
current Application Packager's Guide.
|
|
304 |
|
|
305 |
4.7. Admin/Config Impact:
|
|
306 |
|
|
307 |
Substantial new capabilities in software installation will
|
|
308 |
become available.
|
|
309 |
|
|
310 |
A related project to produce a package manipulation GUI is being
|
|
311 |
pursued.
|
|
312 |
|
|
313 |
4.8. HA Impact:
|
|
314 |
|
|
315 |
None known; dependent on specific impacts of legacy packaging on
|
|
316 |
these capabilities.
|
|
317 |
|
|
318 |
4.9. I18N/L10N Impact:
|
|
319 |
|
|
320 |
Commands will require localization, as will any publically
|
|
321 |
committed library or equivalent APIs.
|
|
322 |
|
|
323 |
4.10. Packaging & Delivery:
|
|
324 |
|
|
325 |
All package, cluster, and metacluster boundaries will be
|
|
326 |
examined in the course of the project.
|
|
327 |
|
|
328 |
The primary upgrade mechanism for operating systems using pkg(5)
|
|
329 |
will be achieved via pkg(5) components; this mechanism is
|
|
330 |
expected to replace the current standalone upgrade and
|
|
331 |
LiveUpgrade paths. The replacement is expected, in concert with
|
|
332 |
the SnapUpgrade project, to present capabilities that equal or
|
|
333 |
exceed those of LiveUpgrade.
|
|
334 |
|
|
335 |
4.11. Security Impact:
|
|
336 |
|
|
337 |
In the current implementation, the protocol is built atop access
|
|
338 |
to HTTP and/or HTTPS. Accordingly, the server side will
|
|
339 |
potentially listen on ports associated with those services.
|
|
340 |
|
|
341 |
The server and client side will require access to key and
|
|
342 |
certificate management interfaces.
|
|
343 |
|
|
344 |
A mechanism for signing repository catalogs and package
|
|
345 |
manifests will be a part of the publication interface. A
|
|
346 |
corresponding mechanism for verifying signed catalogs and
|
|
347 |
manifests will be implemented for the installation client.
|
|
348 |
These mechanisms will apply to both packages in a network
|
|
349 |
package repository and packages in their on-disk representation.
|
|
350 |
|
|
351 |
4.12. Dependencies:
|
|
352 |
|
|
353 |
The project is dependent on SnapUpgrade for coherent collection,
|
|
354 |
organization, and activation of filesystem snapshots.
|
|
355 |
|
|
356 |
5. Reference Documents:
|
|
357 |
Project site:
|
|
358 |
|
|
359 |
http://opensolaris.org/os/project/pkg/
|
|
360 |
|
|
361 |
Project team members have written a number of informal essays on
|
|
362 |
various goals--problems to solve, outcomes to avoid, hopes to
|
|
363 |
realize--on aspects of the project:
|
|
364 |
|
|
365 |
- General observations:
|
|
366 |
|
|
367 |
http://blogs.sun.com/sch/entry/observations_on_packaging
|
|
368 |
|
|
369 |
- On testability and complexity costs with the current patching
|
|
370 |
methods:
|
|
371 |
|
|
372 |
http://blogs.sun.com/barts/entry/rethinking_patching
|
|
373 |
|
|
374 |
- Eliminating scripting in a packaging system
|
|
375 |
|
|
376 |
http://blogs.sun.com/sch/entry/pkg_1_a_no_scripting
|
|
377 |
|
|
378 |
- Keep software builds separate from software delivery:
|
|
379 |
|
|
380 |
http://blogs.sun.com/sch/entry/pkg_leaving_the_build_system
|
|
381 |
|
|
382 |
- Keeping critical metadata back the packaging system, rather
|
|
383 |
than in the installer:
|
|
384 |
|
|
385 |
http://blogs.sun.com/sch/entry/pkg_no_more_installer_magic
|
|
386 |
|
|
387 |
Related efforts in the Caiman project:
|
|
388 |
|
|
389 |
- Snap Upgrade,
|
|
390 |
http://opensolaris.org/os/project/caiman/Snap_Upgrade/
|
|
391 |
|
|
392 |
- Distribution Constructor,
|
|
393 |
http://opensolaris.org/os/project/caiman/Constructor/
|
|
394 |
|
|
395 |
6. Resources and Schedule:
|
|
396 |
6.1. Projected Availability:
|
|
397 |
|
|
398 |
2008
|
|
399 |
|
|
400 |
6.2. Cost of Effort:
|
|
401 |
|
|
402 |
Unable to estimate at present time.
|
|
403 |
|
|
404 |
6.4. Product Approval Committee requested information:
|
|
405 |
6.4.1. Consolidation or Component Name:
|
|
406 |
|
|
407 |
ON
|
|
408 |
|
|
409 |
6.5. ARC review type:
|
|
410 |
|
|
411 |
Standard.
|
|
412 |
|
|
413 |
6.6. ARC Exposure: open
|
|
414 |
6.6.1. Rationale: Part of OpenSolaris
|
|
415 |
|
|
416 |
7. Prototype Availability:
|
|
417 |
|
|
418 |
7.1. Prototype Availability:
|
|
419 |
|
|
420 |
Prototype exit criteria are: ability to support multiple transports,
|
|
421 |
some access control capability, constrained dependency support,
|
|
422 |
bulk of OpenSolaris-specific actions.
|
|
423 |
|
|
424 |
7.2. Prototype Cost:
|
|
425 |
|
|
426 |
Unable to estimate.
|
|
427 |
|