components/openssl/README
author jenny.yung@oracle.com <jenny.yung@oracle.com>
Mon, 25 Aug 2014 14:20:21 -0700
branchs11u2-sru
changeset 3285 78460de60ed1
parent 3217 6c32d6df504a
child 3490 026457ca5815
permissions -rw-r--r--
17799549 libcrypto openssl incorrect size for libcrypto.so.1.0.0`_sparcv9_random

#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
# Copyright (c) 2009, 2014, Oracle and/or its affiliates. All rights reserved.
#


Build Layout
---

OpenSSL build is run four times. Once for regular dynamic 1.0.1 non-fips, once 
for static 1.0.1 bits to link with standalone wanboot binary, once for 1.0.1
fips-140, and once for 1.0.1 FIPS-140 canister (in the openssl-fips component)
needed to build 1.0.1 FIPS-140 certified libraries. All builds apart from 
static libraries for wanboot are done for 32 and 64 bits. So, in total, OpenSSL
is built seven times. OpenSSL for wanboot is only build on sparc.

See also comments in all the Makefiles for more information.

OpenSSL Version
---

For non-FIPS build, we currently deliver OpenSSL 1.0.1 with some updates
from OpenSSL 1.0.2 to make T4 instructions embedded in the OpenSSL
upstream code.  As of April 2013, 1.0.2 is not yet released, and therefore,
we have decided to patch the code.
The following files/code are copied in from 1.0.2.
added:
   components/openssl/openssl-1.0.1/inline-t4/aest4-sparcv9.pl
   components/openssl/openssl-1.0.1/inline-t4/dest4-sparcv9.pl
   components/openssl/openssl-1.0.1/inline-t4/md5-sparcv9.pl
   components/openssl/openssl-1.0.1/inline-t4/sparc_arch.h
   components/openssl/openssl-1.0.1/inline-t4/sparct4-mont.pl
   components/openssl/openssl-1.0.1/inline-t4/sparcv9_modes.pl
   components/openssl/openssl-1.0.1/inline-t4/sparcv9-gf2m.pl
   components/openssl/openssl-1.0.1/inline-t4/vis3-mont.pl
   components/openssl/openssl-1.0.1/patches/openssl-t4-inline.sparc-patch


The non-fips Build.
---

The non-fips build is the main build of OpenSSL and includes the regular
binaries, libraries, man pages, and header files.

Patches
---

08-6193522.patch
Give CA.pl better defaults. See 6193522 for more information.

11-6546806.patch
Make sure the HMAC_CTX_init(3) man page gets delivered. See 6546806 for
more information.

14-manpage_openssl.patch
Force openssl to install man pages into man[1357]openssl instead of man[1357].

15-pkcs11_engine-0.9.8a.patch
Patch which adds the pkcs11 engine. See also the engines/pkcs11
sub-directory. 

18-compiler_opts.patch
Adds five Solaris specific configurations (both 32bit and 64bit for both sparc
and x86, plus 64bit sparc for wanboot) to Configure which are then explicitly
used by the Makefiles. Wanboot configuration is special in that it doesn't link
with libc and uses -xF=%all to put functions in separate sections, so that
unused code can be discarded.

Care should be taken if modifying this patch as changes to compile-time options
can change the ABI. One example of this is the use of RC4_INT vs RC4_CHAR.

20-remove_rpath.patch
Prevent build binaries having an unnecessary runpath (/lib).

23-noexstack.patch
Build with non-executable stacks and non-executable data (x86).

27-6978791.patch
Modifies Makefile.shared so that libssl is built with -znodelete.

28-enginesdir.patch
Adds a new "enginesdir" option to the Configure script which allows a user to
specify the engines directory.

30_wanboot.patch:
Wanboot specific patches.
- modified Makefiles not to build in engines apps test tools
- not using vfprintf for error print in crypto/cryptlib.c
- not using ERR_load_DSO_strings() in crypto/err/err_all.c
- not using EVP_read_pw_string() in crypto/evp/evp_key.c
    - reading password is implemented in disabled DES library
- avoid select() in crypto/rand/rand_unix.c
- direct reading of IP to avoid sscanf() in crypto/x509v3/v3_utl.c
- using functions from libsock in e_os.h
- by-passing version of sparc detection in crypto/sparcv9cap.c
    - results in not using FPU for big numbers multiplication
    - should be ok - original detection seems broken, FPU gets never used
- implementation of atoi()
- avoid using ssl_fill_hello_random() in s3_clnt.c

36_evp_leak.patch:
Solaris-specific fix for mem leak caused by EVP_EncryptFinal_ex()
and EVP_DecryptFinal_ex() not cleaning up properly.

37_openssl-t4-inline.patch
SPARC-only patch.
Add patch to support inline T4 instruction in OpenSSL upstream code until
OpenSSL 1.0.2 is released.

38_remove_illegal_instruction_calls.patch
SPARC patch. Solaris-only patch.
Fix and/or remove illegal instructions from sparcv9cap.c.

opensslconf.patch
Modifies opensslconf.h so that it is suitable for both 32bit and 64bit installs.
OpenSSL either builds for 32bit or 64bit - it doesn't allow for combined 32bit
and 64bit builds.

The fips Build
---

We are now shipping FIPS-140 certified OpenSSL 1.0.1 with S12 and S11.2.
The admin may choose to activate 'openssl-fips' implementation using 'pkg mediator'.
The change will come soon.


Patches
---

All the patches from 1.0.1 (non-fips) are used in 1.0.1(fips) as well aside from
14-manpage_openssl.patch which is not needed since we do not deliver 1.0.1(fips) man
pages.  Once we make fips version public, we should deliver man page.
(coming soon)

The wanboot Build
----

There are some significant differences when building OpenSSL for wanboot.

Some additional Configuration options are needed:
-DNO_CHMOD		chmod not available in stand-alone environment
-DBOOT			guard for wanboot specific patches
-DOPENSSL_NO_DTLS1	to avoid dtls1_min_mtu() - DTLS not used anyway

List of object files for wanboot-openssl.o
----

At this moment, object files for wanboot-openssl.o need to be listed explicitly.
This is cumbersome and relatively tedious with respect to upgrading to higher
version of openssl. 

In future, it would be nice, if this could be performed automatically by the
linker. The required interface for wanboot is already defined in a mapfile and
linker option '-zdiscard-unused=sections,files' is already used to discard
unused code. 
But sadly, at this moment when the linker is given all the object files, it
correctly discards some unused files, but references to undefined symbols from
the discarded files don't get discarded along. Later, these undefined references
cause wanboot linking failure. 

In order to determine which openssl object files are required for wanboot,
first build static standalone openssl bits in Userland. As a site effect,
static libraries libssl.a and libcrypto.a are created in build/sparcv9-wanboot.

    $ cd $USERLAND/components/openssl/openssl-1.0.1 ; gmake build

Next, collect some information from linking wanboot static libraries in ON.
This can be done by the following hack.

    $ cd $ON/usr/src/psm/stand/boot/sparcv9/sun4
    $ touch wanboot.o
    $ LD_OPTIONS="-Dfiles,symbols,output=ld.dbg \
        -L$USERLAND/components/openssl/openssl-1.0.1/build/sparcv9-wanboot " \
        WAN_OPENSSL=" -lwanboot -lssl -lcrypto" dmake all

The following sort of information ends up in ld.dbg (note that the debugging
output from the link-editor is not considered a 'stable interface' and may
change in the future):

    debug: file=/builds/tkuthan/ul-wanboot-rebuilt/components/openssl/openssl-1.0.1/build/sparcv9-wanboot/libcrypto.a(sparcv9cap.o)  [ ET_REL ]
    debug:
    debug: symbol table processing; file=/builds/tkuthan/ul-wanboot-rebuilt/components/openssl/openssl-1.0.1/build/sparcv9-wanboot/libcrypto.a(sparcv9cap.o)  [ ET_REL ]
    debug: symbol[1]=sparcv9cap.c
    ....

Now run the following script in Userland:

    #!/bin/bash
 
    # set to workspace paths:
    USERLAND=/builds/tkuthan/ul-wanboot-rebuilt
    ON=/builds/tkuthan/on11u1-wanboot-rti
 
    BUILD=$USERLAND/components/openssl/openssl-1.0.1/build/sparcv9-wanboot
    LD_DBG=$ON/usr/src/psm/stand/boot/sparcv9/sun4/ld.dbg
 
    for i in `find $BUILD/crypto $BUILD/ssl -name '*.o'`
    do
            f=`basename $i`
            if grep -q "^debug: file.*\<$f\>" $LD_DBG
            then
                    echo $i | sed "s#$BUILD/##"
            fi
    done

to get the list of required object files.

Additionally, you can format the list for including to Makefile by:
    sort | tr '\n' ' ' | fold -s -w74 | sed -e 's/^/    /' -e 's/$/\\/'

Linking with wanboot
----

When linking with wanboot please pay attention to following pitfalls.

Correct openssl header files need to be included. This is done in
$ON/usr/src/stand/lib/wanboot/Makefile
Make sure CPPFLAGS point to the right directories.

EXTREME CAUTION needs to be employed, if WANBOOT GREW IN SIZE because of the
changes!
Wanboot is a statically linked standalone binary and it is loaded on a fixed
address before execution. This address is defined in 
$ON/usr/src/psm/stand/boot/sparc/common/mapfile:

     27 LOAD_SEGMENT text {
     28 	FLAGS = READ EXECUTE;
     29 	VADDR = 0x130000;
     30 	ASSIGN_SECTION {
     31 		TYPE = PROGBITS;
     32 		FLAGS = ALLOC !WRITE;
     33 	};
     34 };

This address (VADDR) NEEDS TO BE GREATER THEN 
    size of wanboot binary + 0x4000

The reason for this is in how wanboot is loaded by OpenBoot Prom:
1) user initiates boot from network - "boot net"
2) obp loads wanboot binary at address 0x4000
3) obp parses ELF header, reads virtual address where to load wanboot to
4) obp mem-copies .text section to this address
5) obp copies .data section behind .text
6) obp starts executing wanboot at entry address

If the given address is too small, obp overwrites part of .data with
instructions from .text in step 4. resulting in .data being corrupted.
Initialized variables get bogus values and failure is inevitable.
This is very hard to troubleshoot.


Testing wanboot with new openssl
----

With every upgrade of OpenSSL, it is necessary to make sure wanboot builds and
works well with the new bits.

Provided you have a freshly built ON workspace, you can link wanboot with new
OpenSSL bits by redefining WAN_OPENSSL macro:

    # copy wanboot-openssl.o to ON build machine
    cp wanboot-openssl.o /var/tmp/

    # prepare to rebuild wanboot
    cd $ON
    bldenv developer.sh
    cd usr/src/psm/stand/boot/sparcv9/sun4

    # hack to force a rebuild
    touch wanboot.o

    # link new OpenSSL to wanboot
    WAN_OPENSSL=/var/tmp/wanboot-openssl.o dmake all

Wanboot should build without warning.

If there is something like this in the output:

    Undefined                       first referenced
     symbol                             in file
    CRYPTO_ccm128_setiv                 /var/tmp/wanboot-openssl.o
    SSL_get_srtp_profiles               /var/tmp/wanboot-openssl.o
    ssl_parse_clienthello_use_srtp_ext  /var/tmp/wanboot-openssl.o
    CRYPTO_gcm128_setiv                 /var/tmp/wanboot-openssl.o
    ...
    cmac_pkey_meth                      /var/tmp/wanboot-openssl.o
    ld: fatal: symbol referencing errors. No output written to wanboot
    *** Error code 1
    dmake: Fatal error: Command failed for target `wanboot'

some additional work has to be done in OpenSSL to either satisfy the function 
references listed in the linker error message, or to remove the calls to these
functions.

Finally, resulting wanboot binary shall be deployed on some install server and
wanbooting from this server shall be tested.