components/openssl/README
changeset 763 45da4d38492e
parent 745 09fd85317532
child 1267 3d7359ef8168
child 2674 4801864231c8
equal deleted inserted replaced
762:d87f60e41094 763:45da4d38492e
    27 OpenSSL build is run four times. Once for regular dynamic 1.0.0 non-fips, once 
    27 OpenSSL build is run four times. Once for regular dynamic 1.0.0 non-fips, once 
    28 for static 1.0.0 bits to link with standalone wanboot binary, once for 0.9.8
    28 for static 1.0.0 bits to link with standalone wanboot binary, once for 0.9.8
    29 fips-140, and once for 0.9.8 FIPS-140 canister (in the openssl-fips component)
    29 fips-140, and once for 0.9.8 FIPS-140 canister (in the openssl-fips component)
    30 needed to build 0.9.8 FIPS-140 certified libraries. All builds apart from 
    30 needed to build 0.9.8 FIPS-140 certified libraries. All builds apart from 
    31 static libraries for wanboot are done for 32 and 64 bits. So, in total, OpenSSL
    31 static libraries for wanboot are done for 32 and 64 bits. So, in total, OpenSSL
    32 is built seven times.
    32 is built seven times. OpenSSL for wanboot is only build on sparc.
    33 
    33 
    34 For more details on OpenSSL for wanboot see openssl-1.0.0-wanboot/README.
    34 See also comments in all the Makefiles for more information.
    35 See also comments in all three Makefiles for more information.
       
    36 
    35 
    37 The non-fips Build.
    36 The non-fips Build.
    38 ---
    37 ---
    39 
    38 
    40 The non-fips build is the main build of OpenSSL and includes the regular
    39 The non-fips build is the main build of OpenSSL and includes the regular
    56 15-pkcs11_engine-0.9.8a.patch
    55 15-pkcs11_engine-0.9.8a.patch
    57 Patch which adds the pkcs11 engine. See also the pkcs11-engine/
    56 Patch which adds the pkcs11 engine. See also the pkcs11-engine/
    58 sub-directory. 
    57 sub-directory. 
    59 
    58 
    60 18-compiler_opts.patch
    59 18-compiler_opts.patch
    61 Adds four Solaris specific configurations (both 32bit and 64bit for both sparc
    60 Adds five Solaris specific configurations (both 32bit and 64bit for both sparc
    62 and x86) to Configure which are then explicitly used by the Makefiles.
    61 and x86, plus 64bit sparc for wanboot) to Configure which are then explicitly
       
    62 used by the Makefiles. Wanboot configuration is special in that it doesn't link
       
    63 with libc and uses -xF=%all to put functions in separate sections, so that
       
    64 unused code can be discarded.
    63 
    65 
    64 Care should be taken if modifying this patch as changes to compile-time options
    66 Care should be taken if modifying this patch as changes to compile-time options
    65 can change the ABI. One example of this is the use of RC4_INT vs RC4_CHAR.
    67 can change the ABI. One example of this is the use of RC4_INT vs RC4_CHAR.
    66 
    68 
    67 20-remove_rpath.patch
    69 20-remove_rpath.patch
    78 specify the engines directory.
    80 specify the engines directory.
    79 
    81 
    80 29-devcrypto_engine.patch
    82 29-devcrypto_engine.patch
    81 Modifies engines/Makefile so that the devcrypto engine will be built in the
    83 Modifies engines/Makefile so that the devcrypto engine will be built in the
    82 "engines" directory. 
    84 "engines" directory. 
       
    85 
       
    86 30_wanboot.patch:
       
    87 Wanboot specific patches.
       
    88 - modified Makefiles not to build in engines apps test tools
       
    89 - not using vfprintf for error print in crypto/cryptlib.c
       
    90 - not using ERR_load_DSO_strings() in crypto/err/err_all.c
       
    91 - not using EVP_read_pw_string() in crypto/evp/evp_key.c
       
    92     - reading password is implemented in disabled DES library
       
    93 - avoid select() in crypto/rand/rand_unix.c
       
    94 - direct reading of IP to avoid sscanf() in crypto/x509v3/v3_utl.c
       
    95 - using functions from libsock in e_os.h
       
    96 - by-passing version of sparc detection in crypto/sparcv9cap.c
       
    97     - results in not using FPU for big numbers multiplication
       
    98     - should be ok - original detection seems broken, FPU gets never used
       
    99 - implementation of atoi()
       
   100 
    83 
   101 
    84 openssl-1.0.0d-aesni-v4.i386-patch
   102 openssl-1.0.0d-aesni-v4.i386-patch
    85 X86-only patch.
   103 X86-only patch.
    86 Add a built-in engine, aesni, to support X86 AES-NI instructions, along with
   104 Add a built-in engine, aesni, to support X86 AES-NI instructions, along with
    87 files engines/aesni/aesni-x86[_64].pl.
   105 files engines/aesni/aesni-x86[_64].pl.
   116 Fixing a bug introduces in 0.9.8q and fixed in 0.9.8r.
   134 Fixing a bug introduces in 0.9.8q and fixed in 0.9.8r.
   117 
   135 
   118 sparc-01-ccwrap.patch
   136 sparc-01-ccwrap.patch
   119 Workaround so that fingerprinting the canister during runtime and comparing it
   137 Workaround so that fingerprinting the canister during runtime and comparing it
   120 with the saved fingerprint works correctly.
   138 with the saved fingerprint works correctly.
       
   139 
       
   140 The wanboot Build
       
   141 ----
       
   142 
       
   143 There are some significant differences when building OpenSSL for wanboot.
       
   144 
       
   145 Some additional Configuration options are needed:
       
   146 -DNO_CHMOD		chmod not available in stand-alone environment
       
   147 -DBOOT			guard for wanboot specific patches
       
   148 -DOPENSSL_NO_DTLS1	to avoid dtls1_min_mtu() - DTLS not used anyway
       
   149 
       
   150 List of object files for wanboot-openssl.o
       
   151 ----
       
   152 
       
   153 At this moment, object files for wanboot-openssl.o need to be listed explicitly.
       
   154 This is cumbersome and relatively tedious with respect to upgrading to higher
       
   155 version of openssl. 
       
   156 
       
   157 In future, it would be nice, if this could be performed automatically by the
       
   158 linker. The required interface for wanboot is already defined in a mapfile and
       
   159 linker option '-zdiscard-unused=sections,files' is already used to discard
       
   160 unused code. 
       
   161 But sadly, at this moment when the linker is given all the object files, it
       
   162 correctly discards some unused files, but references to undefined symbols from
       
   163 the discarded files don't get discarded along. Later, these undefined references
       
   164 cause wanboot linking failure. 
       
   165 
       
   166 In order to determine which openssl object files are required for wanboot,
       
   167 first build static standalone openssl bits in Userland. As a site effect,
       
   168 static libraries libssl.a and libcrypto.a are created in build/sparcv9-wanboot.
       
   169 
       
   170     $ cd $USERLAND/components/openssl/openssl-1.0.0 ; gmake build
       
   171 
       
   172 Next, collect some information from linking wanboot static libraries in ON.
       
   173 This can be done by the following hack.
       
   174 
       
   175     $ cd $ON/usr/src/psm/stand/boot/sparcv9/sun4
       
   176     $ touch wanboot.o
       
   177     $ LD_OPTIONS="-Dfiles,symbols,output=ld.dbg \
       
   178         -L$USERLAND/components/openssl/openssl-1.0.0/build/sparcv9-wanboot " \
       
   179         WAN_OPENSSL=" -lwanboot -lssl -lcrypto" dmake all
       
   180 
       
   181 The following sort of information ends up in ld.dbg (note that the debugging
       
   182 output from the link-editor is not considered a 'stable interface' and may
       
   183 change in the future):
       
   184 
       
   185     debug: file=/builds/tkuthan/ul-wanboot-rebuilt/components/openssl/openssl-1.0.0/build/sparcv9-wanboot/libcrypto.a(sparcv9cap.o)  [ ET_REL ]
       
   186     debug:
       
   187     debug: symbol table processing; file=/builds/tkuthan/ul-wanboot-rebuilt/components/openssl/openssl-1.0.0/build/sparcv9-wanboot/libcrypto.a(sparcv9cap.o)  [ ET_REL ]
       
   188     debug: symbol[1]=sparcv9cap.c
       
   189     ....
       
   190 
       
   191 Now run the following script in Userland:
       
   192 
       
   193     #!/bin/bash
       
   194  
       
   195     # set to workspace paths:
       
   196     USERLAND=/builds/tkuthan/ul-wanboot-rebuilt
       
   197     ON=/builds/tkuthan/on11u1-wanboot-rti
       
   198  
       
   199     BUILD=$USERLAND/components/openssl/openssl-1.0.0/build/sparcv9-wanboot
       
   200     LD_DBG=$ON/usr/src/psm/stand/boot/sparcv9/sun4/ld.dbg
       
   201  
       
   202     for i in `find $BUILD/crypto $BUILD/ssl -name '*.o'`
       
   203     do
       
   204             f=`basename $i`
       
   205             if grep -q "^debug: file.*\<$f\>" $LD_DBG
       
   206             then
       
   207                     echo $i | sed "s#$BUILD/##"
       
   208             fi
       
   209     done
       
   210 
       
   211 to get the list of required object files.
       
   212 
       
   213 Additionally, you can format the list for including to Makefile by:
       
   214     sort | tr '\n' ' ' | fold -s -w74 | sed -e 's/^/    /' -e 's/$/\\/'
       
   215 
       
   216 Linking with wanboot
       
   217 ----
       
   218 
       
   219 When linking with wanboot please pay attention to following pitfalls.
       
   220 
       
   221 Correct openssl header files need to be included. This is done in
       
   222 $ON/usr/src/stand/lib/wanboot/Makefile
       
   223 Make sure CPPFLAGS point to the right directories.
       
   224 
       
   225 EXTREME CAUTION needs to be employed, if WANBOOT GREW IN SIZE because of the
       
   226 changes!
       
   227 Wanboot is a statically linked standalone binary and it is loaded on a fixed
       
   228 address before execution. This address is defined in 
       
   229 $ON/usr/src/psm/stand/boot/sparc/common/mapfile:
       
   230 
       
   231      27 LOAD_SEGMENT text {
       
   232      28 	FLAGS = READ EXECUTE;
       
   233      29 	VADDR = 0x130000;
       
   234      30 	ASSIGN_SECTION {
       
   235      31 		TYPE = PROGBITS;
       
   236      32 		FLAGS = ALLOC !WRITE;
       
   237      33 	};
       
   238      34 };
       
   239 
       
   240 This address (VADDR) NEEDS TO BE GREATER THEN 
       
   241     size of wanboot binary + 0x4000
       
   242 
       
   243 The reason for this is in how wanboot is loaded by OpenBoot Prom:
       
   244 1) user initiates boot from network - "boot net"
       
   245 2) obp loads wanboot binary at address 0x4000
       
   246 3) obp parses ELF header, reads virtual address where to load wanboot to
       
   247 4) obp mem-copies .text section to this address
       
   248 5) obp copies .data section behind .text
       
   249 6) obp starts executing wanboot at entry address
       
   250 
       
   251 If the given address is too small, obp overwrites part of .data with
       
   252 instructions from .text in step 4. resulting in .data being corrupted.
       
   253 Initialized variables get bogus values and failure is inevitable.
       
   254 This is very hard to troubleshoot.
       
   255 
       
   256 
       
   257 Testing wanboot with new openssl
       
   258 ----
       
   259 
       
   260 With every upgrade of OpenSSL, it is necessary to make sure wanboot builds and
       
   261 works well with the new bits.
       
   262 
       
   263 Provided you have a freshly built ON workspace, you can link wanboot with new
       
   264 OpenSSL bits by redefining WAN_OPENSSL macro:
       
   265 
       
   266     # copy wanboot-openssl.o to ON build machine
       
   267     cp wanboot-openssl.o /var/tmp/
       
   268 
       
   269     # prepare to rebuild wanboot
       
   270     cd $ON
       
   271     bldenv developer.sh
       
   272     cd usr/src/psm/stand/boot/sparcv9/sun4
       
   273 
       
   274     # hack to force a rebuild
       
   275     touch wanboot.o
       
   276 
       
   277     # link new OpenSSL to wanboot
       
   278     WAN_OPENSSL=/var/tmp/wanboot-openssl.o dmake all
       
   279 
       
   280 Wanboot should build without warning.
       
   281 
       
   282 If there is something like this in the output:
       
   283 
       
   284     Undefined                       first referenced
       
   285      symbol                             in file
       
   286     CRYPTO_ccm128_setiv                 /var/tmp/wanboot-openssl.o
       
   287     SSL_get_srtp_profiles               /var/tmp/wanboot-openssl.o
       
   288     ssl_parse_clienthello_use_srtp_ext  /var/tmp/wanboot-openssl.o
       
   289     CRYPTO_gcm128_setiv                 /var/tmp/wanboot-openssl.o
       
   290     ...
       
   291     cmac_pkey_meth                      /var/tmp/wanboot-openssl.o
       
   292     ld: fatal: symbol referencing errors. No output written to wanboot
       
   293     *** Error code 1
       
   294     dmake: Fatal error: Command failed for target `wanboot'
       
   295 
       
   296 some additional work has to be done in OpenSSL to either satisfy the function 
       
   297 references listed in the linker error message, or to remove the calls to these
       
   298 functions.
       
   299 
       
   300 Finally, resulting wanboot binary shall be deployed on some install server and
       
   301 wanbooting from this server shall be tested.