Process to update OpenSSL in base
(for 1.1.0h WIP see OpenSSL/Base/Update110)


For the impatient...

svn checkout /usr/src
fetch -o ~/patch-D15791 ''
fetch -o ~/openssl-1.1.1-pre8.tar.gz
cd /usr/src
patch -p0 < ~/patch-D15791
mv /usr/src/crypto/openssl /usr/src/crypto/openssl-1.0.2
tar xf ~/openssl-1.1.1-pre8.tar.gz -C /usr/src/crypto
mv /usr/src/crypto/openssl-1.1.1-pre8 /usr/src/crypto/openssl
cd /usr/src/secure/lib/libcrypto
pkg install perl5
rm amd64/*
mv /usr/obj/usr/src/amd64.amd64/secure/lib/libcrypto/*.S amd64/
make man-makefile-update
rm man/*
make man-update

Preliminary steps

1. Move current version of OpenSSL

$ mv /usr/src/crypto/openssl /usr/src/crypto/openssl-1.0.2o

2. Download, verify

$ fetch
openssl-1.1.1-pre8.tar.gz                     100% of 8139 kB   41 MBps 00m00s
$ fetch
openssl-1.1.1-pre8.tar.gz.asc                 100% of  488  B 5178 kBps 00m00s
$ gpg --verify openssl-1.1.1-pre8.tar.gz.asc openssl-1.1.1-pre8.tar.gz
gpg: WARNING: using insecure memory!
gpg: please see for more information
gpg: Signature made Wed Jun 20 16:48:22 2018 CEST using RSA key ID 0E604491
gpg: Good signature from "Matt Caswell <>"
gpg:                 aka "Matt Caswell <>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 8657 ABB2 60F0 56B1 E519  0839 D9C4 D26D 0E60 4491

3. Unpack, remove version from directory

$ tar xf ~/openssl-1.1.1-pre8-tar.gz -C /usr/src/crypto
$ mv /usr/src/crypto/openssl-1.1.1-pre8 /usr/src/crypto/openssl

4. Update /usr/src/secure/lib/libcrypto/

# OpenSSL version used for manual page generation
OPENSSL_VER=    1.1.1.p8
OPENSSL_DATE=   2018-06-20

5. Required tools: Perl We need pod2man from perl and perl to generate assembly from the perl scripts

$ pkg install perl5

Generate `.h` files

Complication here, as there are different opensslconf.h files for various arch. For now I generated an opensslconf.h like so

$ uname -a
FreeBSD freebsd12 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r334983: Tue Jun 12 01:42:40 UTC 2018  amd64
$ ./config -v --openssldir=/usr/local/openssl --prefix=/usr/local enable-ec_nistp_64_gcc_128 no-sm2 no-sm3
./config -v --openssldir=/usr/local/openssl --prefix=/usr/local enable-ec_nistp_64_gcc_128 no-sm2 no-sm3
Operating system: amd64-whatever-freebsd
/usr/bin/env __CNF_CPPDEFINES='' __CNF_CPPINCLUDES='' __CNF_CPPFLAGS='' __CNF_CFLAGS='' __CNF_CXXFLAGS='' __CNF_LDFLAGS='' __CNF_LDLIBS='' /usr/local/bin/perl5 ./Configure BSD-x86_64 '--openssldir=/usr/local/openssl' '--prefix=/usr/local' 'enable-ec_nistp_64_gcc_128' 'no-sm2' 'no-sm3'
Configuring OpenSSL version 1.1.1-pre8 (0x10101008L) for BSD-x86_64
Using os-specific seed configuration
Creating Makefile

***                                                                ***
***   If you want to report a building issue, please include the   ***
***   output from this command:                                    ***
***                                                                ***
***     perl --dump                                  ***
***                                                                ***

Now we can run make build_generated which will generate the files we want

$ make build_generated
/usr/local/bin/perl5 "-I." -Mconfigdata "util/"  "-oMakefile" crypto/include/internal/ > crypto/include/internal/bn_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/"  "-oMakefile" crypto/include/internal/ > crypto/include/internal/dso_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/"  "-oMakefile" include/openssl/ > include/openssl/opensslconf.h

Merge the diff into TODO: Generate proper opensslconf.h files for all archs
Same applies to crypto/include/internal/bn_conf.h
This is a hint from the OpenSSL Makefile

GENERATED_MANDATORY=crypto/include/internal/bn_conf.h crypto/include/internal/dso_conf.h include/openssl/opensslconf.h

and from the make output

/usr/local/bin/perl5 "-I." -Mconfigdata "util/"  "-oMakefile" crypto/include/internal/ > crypto/include/internal/bn_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/"  "-oMakefile" crypto/include/internal/ > crypto/include/internal/dso_conf.h
/usr/local/bin/perl5 "-I." -Mconfigdata "util/"  "-oMakefile" include/openssl/ > include/openssl/opensslconf.h

1.1.1 introduces yet another generate header, apps/progs.h

Generate assembly `.S` files

From secure/libcrypto/Makefile.asm:

# $FreeBSD: head/secure/lib/libcrypto/Makefile.asm 307976 2016-10-26 20:02:22Z jkim $
# Use this to help generate the asm *.S files after an import.  It is not
# perfect by any means, but does what is needed.
# Do a 'make -f Makefile.asm all' and it will generate *.S.  Move them
# to the arch subdir, and correct any exposed paths and $ FreeBSD $ tags.

Let's do that!

make -f Makefile.asm all
[Creating objdir /usr/obj/usr/src/amd64.amd64/secure/lib/libcrypto...]
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from */' ; env CC=cc perl /usr/src/crypto/openssl/crypto/aes/asm/ elf ) > aes-x86_64.S
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from */' ; env CC=cc perl /usr/src/crypto/openssl/crypto/aes/asm/ elf ) > aesni-mb-x86_64.S
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from */' ; cat sha256-x86_64.s ) > sha256-x86_64.S
env CC=cc perl /usr/src/crypto/openssl/crypto/sha/asm/ elf sha512-x86_64.s
( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from */' ; cat sha512-x86_64.s ) > sha512-x86_64.S

TODO: Verify that we have all necessary .S files
The files processed are defined in the Makefile, which is thus version-specific.

Update Makefile.asm `.pl` sources

OpenSSL 1.1 added some perl files that generate ASM. From ${WRKSRC}/Configurations/00-base-templates.conf we can find out what OpenSSL's current list of assembly files is per architecture. Quite some additional files to be added. Chacha20/Poly1305 for all archs, some other for ARM(64).

Manpage generation

The structure in the source has changed, there's no distinction between the libs and/or apps. All we have is the docs directory containing sub-directories man1 man3 man5 man7. The man-update and man-makefile-update targets in secure/lib/libcrypto/ must be updated to reflect that.

Additionally, the scripts to convert the (Perl) .pod files to man-pages were changed.

All docs are now located in the secure/lib/libcrypto/man dir.

`` and `` is the old and process_docs is the new script


Looks like all it does is read the =head1 NAME section of the .pod files and extract the comma-separated entries contained.
Example from ASN1_ITEM_lookup.pod:

=head1 NAME

ASN1_ITEM_lookup, ASN1_ITEM_get - lookup ASN.1 structures

Results in a Perl hash-table


NOTE: The manpage here is ASN1_ITEM_lookup.3 so somewhere the perl script filters out that link to itself.
This should lead to a single sym-link ASN1_ITEM_get.3 -> ASN1_ITEM_lookup.3


Looks like the part we're interested in is|line 169 onwards (This link is broken).

This uses a feature from OpenSSL::Util::Pod which isn't directly accessible.

An alternative in sed is

sed '/=head1 NAME/,/=head1 /! d;/=head1.*/d;/^\s*$/d;s/ - .*$//;s/,//g' BIO_ADDR.pod || tr ' ' '\n'

Update secure/lib/libcrypto/Makefile

Sources to be checked is crypto/*/ which defines the sources to be used.
Helper to extract SRCS+= from a dir

cd /usr/src/crypto/openssl/crypto
for file in */ ; do
   echo -e "\n# ${file%/}"
   sed -E '/^SOURCE/,/^($|INCLUDE.*)/! d;/^INCLUDE.*/d;s/^SOURCE.*=//;s/\\//g;s/\{.*\}//g;s/[[:space:]]+/ /g;s/^ //;s/ $//;/^$/d' $file \
   | tr ' ' '\n' | sort | tr '\n' ' ' | fold -s -w70 \
   | sed 's/$/\\/;s/^/  /;1s/^/SRCS+=/;$s/ \\//'


OpenSSL/Base/Update111 (last edited 2020-10-29T03:24:57+0000 by SashaVigole)