Summary

For the impatient...

svn checkout https://svn.FreeBSD.org/base/head /usr/src
fetch -o ~/patch-D15791 'https://reviews.freebsd.org/D15791?download=true'
fetch -o ~/openssl-1.1.1-pre8.tar.gz https://www.openssl.org/source/openssl-1.1.1-pre8.tar.gz
cd /usr/src
patch -p0 < ~/patch-D15791
mv /usr/src/crypto/openssl /usr/src/crypto/openssl-1.0.2
tar xf ~/openssl-1.1.1-pre8.tar.gz -C /usr/src/crypto
mv /usr/src/crypto/openssl-1.1.1-pre8 /usr/src/crypto/openssl
cd /usr/src/secure/lib/libcrypto
pkg install perl5
rm amd64/*
mv /usr/obj/usr/src/amd64.amd64/secure/lib/libcrypto/*.S amd64/
make man-makefile-update
rm man/*
make man-update

Preliminary steps

1. Move current version of OpenSSL

$ mv /usr/src/crypto/openssl /usr/src/crypto/openssl-1.0.2o

2. Download, verify

$ fetch https://www.openssl.org/source/openssl-1.1.1-pre8.tar.gz
openssl-1.1.1-pre8.tar.gz                     100% of 8139 kB   41 MBps 00m00s
$ fetch https://www.openssl.org/source/openssl-1.1.1-pre8.tar.gz.asc
openssl-1.1.1-pre8.tar.gz.asc                 100% of  488  B 5178 kBps 00m00s
$ gpg --verify openssl-1.1.1-pre8.tar.gz.asc openssl-1.1.1-pre8.tar.gz
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/documentation/faqs.html for more information
gpg: Signature made Wed Jun 20 16:48:22 2018 CEST using RSA key ID 0E604491
gpg: Good signature from "Matt Caswell <matt@openssl.org>"
gpg:                 aka "Matt Caswell <frodo@baggins.org>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 8657 ABB2 60F0 56B1 E519  0839 D9C4 D26D 0E60 4491

3. Unpack, remove version from directory

$ tar xf ~/openssl-1.1.1-pre8-tar.gz -C /usr/src/crypto
$ mv /usr/src/crypto/openssl-1.1.1-pre8 /usr/src/crypto/openssl

4. Update /usr/src/secure/lib/libcrypto/Makefile.inc

...
# OpenSSL version used for manual page generation
OPENSSL_VER=    1.1.1.p8
OPENSSL_DATE=   2018-06-20
...

5. Required tools: Perl We need pod2man from perl and perl to generate assembly from the perl scripts

$ pkg install perl5
...

Generate `.h` files

Complication here, as there are different opensslconf.h files for various arch. For now I generated an opensslconf.h like so

$ uname -a
FreeBSD freebsd12 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r334983: Tue Jun 12 01:42:40 UTC 2018     root@releng3.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
$ ./config -v --openssldir=/usr/local/openssl --prefix=/usr/local enable-ec_nistp_64_gcc_128 no-sm2 no-sm3
./config -v --openssldir=/usr/local/openssl --prefix=/usr/local enable-ec_nistp_64_gcc_128 no-sm2 no-sm3
Operating system: amd64-whatever-freebsd
/usr/bin/env __CNF_CPPDEFINES='' __CNF_CPPINCLUDES='' __CNF_CPPFLAGS='' __CNF_CFLAGS='' __CNF_CXXFLAGS='' __CNF_LDFLAGS='' __CNF_LDLIBS='' /usr/local/bin/perl5 ./Configure BSD-x86_64 '--openssldir=/usr/local/openssl' '--prefix=/usr/local' 'enable-ec_nistp_64_gcc_128' 'no-sm2' 'no-sm3'
Configuring OpenSSL version 1.1.1-pre8 (0x10101008L) for BSD-x86_64
Using os-specific seed configuration
Creating configdata.pm
Creating Makefile

**********************************************************************
***                                                                ***
***   If you want to report a building issue, please include the   ***
***   output from this command:                                    ***
***                                                                ***
***     perl configdata.pm --dump                                  ***
***                                                                ***
**********************************************************************

Now we can run make build_generated which will generate the files we want

$ make build_generated
/usr/local/bin/perl5 "-I." -Mconfigdata "util/dofile.pl"  "-oMakefile" crypto/include/internal/bn_conf.h.in > crypto/include/internal/bn_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/dofile.pl"  "-oMakefile" crypto/include/internal/dso_conf.h.in > crypto/include/internal/dso_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/dofile.pl"  "-oMakefile" include/openssl/opensslconf.h.in > include/openssl/opensslconf.h

Merge the diff into opensslconf-x86.h.in TODO: Generate proper opensslconf.h files for all archs
Same applies to crypto/include/internal/bn_conf.h
This is a hint from the OpenSSL Makefile

GENERATED_MANDATORY=crypto/include/internal/bn_conf.h crypto/include/internal/dso_conf.h include/openssl/opensslconf.h

and from the make output

/usr/local/bin/perl5 "-I." -Mconfigdata "util/dofile.pl"  "-oMakefile" crypto/include/internal/bn_conf.h.in > crypto/include/internal/bn_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/dofile.pl"  "-oMakefile" crypto/include/internal/dso_conf.h.in > crypto/include/internal/dso_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/dofile.pl"  "-oMakefile" include/openssl/opensslconf.h.in > include/openssl/opensslconf.h

1.1.1 introduces yet another generate header, apps/progs.h

Generate assembly `.S` files

From secure/libcrypto/Makefile.asm:

# $FreeBSD: head/secure/lib/libcrypto/Makefile.asm 307976 2016-10-26 20:02:22Z jkim $
# Use this to help generate the asm *.S files after an import.  It is not
# perfect by any means, but does what is needed.
# Do a 'make -f Makefile.asm all' and it will generate *.S.  Move them
# to the arch subdir, and correct any exposed paths and $ FreeBSD $ tags.

Let's do that!

make -f Makefile.asm all
[Creating objdir /usr/obj/usr/src/amd64.amd64/secure/lib/libcrypto...]
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from aes-x86_64.pl. */' ; env CC=cc perl /usr/src/crypto/openssl/crypto/aes/asm/aes-x86_64.pl elf ) > aes-x86_64.S
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from aesni-mb-x86_64.pl. */' ; env CC=cc perl /usr/src/crypto/openssl/crypto/aes/asm/aesni-mb-x86_64.pl elf ) > aesni-mb-x86_64.S
...
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from sha512-x86_64.pl. */' ; cat sha256-x86_64.s ) > sha256-x86_64.S
env CC=cc perl /usr/src/crypto/openssl/crypto/sha/asm/sha512-x86_64.pl elf sha512-x86_64.s
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from sha512-x86_64.pl. */' ; cat sha512-x86_64.s ) > sha512-x86_64.S

TODO: Verify that we have all necessary .S files
The files processed are defined in the Makefile, which is thus version-specific.

Update Makefile.asm `.pl` sources

OpenSSL 1.1 added some perl files that generate ASM. From ${WRKSRC}/Configurations/00-base-templates.conf we can find out what OpenSSL's current list of assembly files is per architecture. Quite some additional files to be added. Chacha20/Poly1305 for all archs, some other for ARM(64).

Manpage generation

The structure in the source has changed, there's no distinction between the libs and/or apps. All we have is the docs directory containing sub-directories man1 man3 man5 man7. The man-update and man-makefile-update targets in secure/lib/libcrypto/Makefile.inc must be updated to reflect that.

Additionally, the scripts to convert the (Perl) .pod files to man-pages were changed.

All docs are now located in the secure/lib/libcrypto/man dir.

`extract-names.pl` and `process_docs.pl`

extract-names.pl is the old and process_docs is the new script

`extract-names.pl`

Looks like all it does is read the =head1 NAME section of the .pod files and extract the comma-separated entries contained.
Example from ASN1_ITEM_lookup.pod:

=head1 NAME

ASN1_ITEM_lookup, ASN1_ITEM_get - lookup ASN.1 structures

Results in a Perl hash-table

ASN1_ITEM_lookup
ASN1_ITEM_get

NOTE: The manpage here is ASN1_ITEM_lookup.3 so somewhere the perl script filters out that link to itself.
This should lead to a single sym-link ASN1_ITEM_get.3 -> ASN1_ITEM_lookup.3

`process_docs.pl`

Looks like the part we're interested in is https://github.com/openssl/openssl/blob/master/util/process_docs.pl#L169|line 169 onwards (This link is broken).

This uses a feature from OpenSSL::Util::Pod which isn't directly accessible.

An alternative in sed is

sed '/=head1 NAME/,/=head1 /! d;/=head1.*/d;/^\s*$/d;s/ - .*$//;s/,//g' BIO_ADDR.pod || tr ' ' '\n'

Update secure/lib/libcrypto/Makefile

Sources to be checked is crypto/*/build.info which defines the sources to be used.
Helper to extract SRCS+= from a dir

cd /usr/src/crypto/openssl/crypto
for file in */build.info ; do
   echo -e "\n# ${file%/build.info}"
   sed -E '/^SOURCE/,/^($|INCLUDE.*)/! d;/^INCLUDE.*/d;s/^SOURCE.*=//;s/\\//g;s/\{.*\}//g;s/[[:space:]]+/ /g;s/^ //;s/ $//;/^$/d' $file \
   | tr ' ' '\n' | sort | tr '\n' ' ' | fold -s -w70 \
   | sed 's/$/\\/;s/^/  /;1s/^/SRCS+=/;$s/ \\//'
   echo
done

copy-paste.