logo
Apache Lounge
Webmasters

 

About Forum Index Downloads Search Register Log in RSS X


Keep Server Online

If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.

or

Bitcoin

A donation makes a contribution towards the costs, the time and effort that's going in this site and building.

Thank You! Steffen

Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
Post new topic   Forum Index -> Building & Member Downloads View previous topic :: View next topic
Reply to topic   Topic: OpenSSL details
Author
tdonovan
Moderator


Joined: 17 Dec 2005
Posts: 611
Location: Milford, MA, USA

PostPosted: Tue 09 Oct '07 15:56    Post subject: OpenSSL details Reply with quote

When you build Apache 2.2 (including mod_ssl and mod_deflate) with Visual C++, you must
* download OpenSSL from http://www.openssl.org/ into srclib\openssl
* download Zlib from http://www.zlib.net/ into srclib\zlib
and build them both first.

If you use mod_ssl for for non-commercial use which is not performance-sensitive, this is the simplest way to build OpenSSL:
EDIT: you can ignore the several "%OSVERSION%" and "Compilation failed" warnings which display when you run ms\do_ms.bat.
Code:
PERL Configure VC-WIN32
ms\do_ms.bat
NMAKE -f ms\ntdll.mak


If you are building OpenSSL for commercial use, or for distribution, there are a few extra things to consider. Some of them are technical and some are legal.

Assembler

A throughput increase of approximately 10% can be obtained by using some assembler code instead of C code for selected parts of the cipher calculations. Note this is a 10% increase in encryption speed only - don't expect all of Apache to be 10% faster.

MASM - If you have any Visual C++ 2005 Edition except the Express Edition, MASM is included and you can use it for commercial purposes.

If you have the free Express Edition you can download a free MASM V8 which works fine, but note that the license says that the binaries produced with this MASM cannot be used for commercial purposes. This is an odd restriction because binaries produced by the Visual C++ 2005 Express Edition C compiler can be freely used for commercial purposes. It is only the Express Edition MASM binaries which have this restriction.

The non-Microsoft assembler NASM imposes no restrictions.

To build OpenSSL using the MASM assembler, use these commands:
Code:
PERL Configure VC-WIN32
ms\do_Masm.bat
NMAKE -f ms\ntdll.mak

    Starting with OpenSSL 0.9.8f - if you use MASM version 6, you will need version 6.15 or higher to assemble the Pentium SSE2 instructions.
    If you have Visual C++ 6.0, you can download the Visual C++ 6.0 Processor Pack to upgrade MASM to version 6.15.
    MASM 6 users will need to edit the file crypto\sha\asm\sha512-sse2.asm after running ms\do_Masm.bat and change every occurence of XMMWORD to QWORD.

    MASM 8 users should not make any changes.

To use the NASM assembler:
Code:
PERL Configure VC-WIN32
ms\do_Nasm.bat
NMAKE -f ms\ntdll.mak



The IDEA cipher

The IDEA cipher in OpenSSL is patented for commercial use. If you build OpenSSL/Apache for distribution or for commercial use, you should disable the IDEA algorithm:
Code:
PERL Configure VC-WIN32  disable-idea
...
Since no browsers use the IDEA cipher, this change will not be noticed - but it will keep you from violating the patent by distributing the IDEA cipher to potentially commercial users.

The Camellia cipher

This cipher is recommended by the European Union NESSIE project, the Japanese CRYPTREC project, and was added to the SSL/TLS cipher list by RFC 4132. The Camellia algorithm will be in FireFox 3. It is not enabled by default in OpenSSL.

The Cameliia home site mentions that there are export (from Japan) restrictions which may make Japanese OpenSSL distributors cautious, but these are general restrictions on all strong (64+ bit) cryptography. There is nothing camellia-specific about these Japanese export restrictions, so adding Camellia does not change the Japanese export situation.

If you build OpenSSL for distribution to Japan or Europe, adding camellia is recommended:
Code:
PERL Configure VC-WIN32  enable-camellia
...

Other ciphers

All the other default ciphers in OpenSSL are not (or are no longer) patented, or else they allow unrestricted commercial use.
The RC5 cipher is patented by RSA Security, Inc., but it is disabled by default starting with OpenSSL 0.9.8.
The IBM patent for the seldom-used MDC2 hash function expired on August 28, 2007, although MDC2 is still disabled by default in OpenSSL 0.9.8.

Compression

The TLS spec allows for compression over a SSL/TLS connection. There are two methods defined - "deflate" and "lzs". Since version 0.9.8 OpenSSL is able to use the "deflate" method - but before you decide to do so consider:
    * OpenSSL has no enable/disable switch or API calls to trigger compression.
    If both sides of the connection have compression enabled - compression is always used for all data transferred.

    * All data on the connection (which may be multiple HTTP requests/responses) is compressed.

    * The request data gets compressed, unlike mod_deflate which only compresses response data.
    This could be beneficial for very large uncompressed (e.g. text) file uploads or POST requests.

    * Any data which is already compressed (.zip, .jar, .jpg, .gif, .png, etc.) will be re-compressed which does not make it any smaller, and may actually make it slightly larger.
    This is wasteful of CPU cycles for images, which are common in modern web sites.
    This is very wasteful of CPU cycles for large .zip or .tgz transfers.

    * Compression is unlikely to happen in practice.
    No current browsers support TLS compression (FireFox 2, IE6, IE7, Opera 9, Konqueror).

    * TLS compression is only likely to happen in an Apache-to-Apache https proxy where both sides have compression enabled.
In light of this, I don't think enabling automatic compression for TLS is a good idea. Using mod_deflate instead gives you better control over what gets compressed. If you want to enable TLS compression anyway, here is how to do it.

There are three ways to enable compression in OpenSSL. For all of them you must build Zlib before you build OpenSSL (of course):
    1. Compression is enabled and built in, zlib1.dll is not needed at runtime:
    Code:
    PERL Configure VC-WIN32  zlib --with-zlib-lib=../zlib/zlib.lib --with-zlib-include=../zlib

    2. Compression is enabled and requires zlib1.dll at runtime:
    Code:
    PERL Configure VC-WIN32  zlib  --with-zlib-lib=../zlib/zdll.lib --with-zlib-include=../zlib

    3. Dynamic (the presence of zlib1.dll determines whether compression is enabled at runtime):
    Code:
    PERL Configure VC-WIN32  zlib-dynamic  --with-zlib-lib=../zlib/zdll.lib --with-zlib-include=../zlib

If you download Apache from http://httpd.apache.org/ it has compression enabled by method 2. above. Someone at ASF had enough interest in TLS compression to add a doc note about it. One of the Apache developers said this about TLS compression in a mailing list post
Quote:
"... we throw away the zlib-dynamic stubs (and eliminate some race and processing-time performance hits)"
but I'm not sure what he means. Maybe a caution against using method 3. zlib-dynamic.


Testing

After you build OpenSSL, it is a good idea to test it. You can do this by:
Code:
cd out32dll
..\ms\test.bat
This will run through a test of all the facilities of OpenSSL.

If you have disabled the IDEA cipher, you will need to comment out these tests from ms\test.bat lines 18 through 20 so it will not try to test the IDEA cipher:
Code:
        REM echo ideatest
        REM ideatest
        REM if errorlevel 1 goto done

You should also comment out all the "idea-" tests in ms\testenc.bat at lines 54-65.

If you have enabled the Camellia cipher, you can add these Camellia tests to ms\testenc.bat:
Code:
call tenc.bat camellia-128-ecb
if errorlevel 1 goto err

call tenc.bat camellia-128-cbc
if errorlevel 1 goto err

call tenc.bat camellia-192-ecb
if errorlevel 1 goto err

call tenc.bat camellia-192-cbc
if errorlevel 1 goto err

call tenc.bat camellia-256-ecb
if errorlevel 1 goto err

call tenc.bat camellia-256-cbc
if errorlevel 1 goto err


Summary

In my opinion, the best way to build the current version of OpenSSL (openssl 0.9.8g) for commercial use or distribution is with NASM (if you have VS8 Express Edition), with Camellia, without compression, and without IDEA:
Code:
PERL Configure VC-WIN32  enable-camellia  disable-idea
ms\do_Nasm
nmake -f ms\ntdll.mak
Remember to fix ms\test.bat before running the tests so it doesn't try to test the IDEA cipher.

-tom-


Last edited by tdonovan on Fri 19 Oct '07 19:25; edited 10 times in total
Back to top
glsmith
Moderator


Joined: 16 Oct 2007
Posts: 2268
Location: Sun Diego, USA

PostPosted: Tue 16 Oct '07 3:33    Post subject: Reply with quote

Thanks for the info. I will have to rebuild OpenSSL 098f and 097m (for 1.3.39) with disable idea and enable Camilla.

However, I'd like to point out a few experiences I have come across.

Apache 2.0.61 & 2.2.6 building with zlib 1.2.3 will simply not compile (vc++ 6).
You can however compile mod_deflate separately with no problem, pull the zlib directory out of srclib and then build Apache and all works lovely.

Note: You have to patch 2 files in 2.0.x source to get mod_deflate built. I do not have a patch for 2.0.61, and the one written and widely available for 2.0.59 doesn't work, I assume line numbers have changed in the Apache source. One can however read the patch file and just manually do it in a few minutes.

There seems to be a bug I have run into with OpenSSL 0.9.8F and the VC-Win32.pl file it produces. It wants to go into the WinCE subroutine on WinXP which then produces an error about the OSVERSION environment variable which as far as I can tell is not set in WinXP. Edit the file and remove the entire CE subroutine (since we want Win32 anyway) and all is well.

Edit: OK, it's not a subroutine since it is all inside an elsif statement, but I digress.

I also seem not to be able to ms/do_masm on this version. I assume ms/do_nasm would run into the same problem. During the preceding nmake I get some error about some .asm file (unfortunately I do not remember right now) and this brings the compiling to a halt. ms/do_ms works fine however so I'll take the 10% hit on this go-around.

I'd post my way to get Apache 2.0.x compiled with OpenSSL 0.9.8x, but I'll save that for another thread if asked.

If you take any exception/s to this post, floggings will be graciously accepted provided I learn something from them Mr. Green

again, thanks for the info!
Back to top
tdonovan
Moderator


Joined: 17 Dec 2005
Posts: 611
Location: Milford, MA, USA

PostPosted: Wed 17 Oct '07 19:42    Post subject: Reply with quote

I do not see this problem building OpenSSL 0.9.8f - although there is an unrelated problem with OpenSSL 0.9.8f.

re: "There seems to be a bug I have run into with OpenSSL 0.9.8F and the VC-Win32.pl file it produces."
I don't have any VC-Win32.pl file. Do you mean the file util\pl\VC-32.pl?
This file should build the correct platform if you entered VC-WIN32 as the first argument to the PERL Configure command. Note that this argument is case-sensitive.

Also - I had no problems with ms/do_masm.bat. Can you tell us your exact error message?

-tom-
Back to top
glsmith
Moderator


Joined: 16 Oct 2007
Posts: 2268
Location: Sun Diego, USA

PostPosted: Wed 17 Oct '07 21:59    Post subject: Reply with quote

sure ... Since I am now trying it with your disable idea and enable camilla, here is the specidic error I get when using do_masm

P.S. I know it is case sensitive, just like everything perl.
and yes again, util\pl\VC-32.pl ... I was going off of a couple days old memory when I posted this since I first compiled it on the 13th

using perl util\mkdef.pl crypto ssl update has no effect on the problems

H:\build\openssl-0.9.8f>perl configure VC-WIN32 --prefix=/openssl disable-idea enable-camellia
-all is good-

H:\build\openssl-0.9.8f>ms\do_masm
-all looks good-

error during nmake /f ms\ntdll.mak after using ms\do_masm

.\crypto\sha\asm\sha512-sse2.asm(29) : error A2006: undefined symbol : XMMWORD

I get 26 of them, I assume the (##) is the line numbers in the .asm file, that is the first one.

specific errors when trying to do ms\do_ms

H:\build\openssl-0.9.8f>ms\do_ms
H:\build\openssl-0.9.8f>perl util\mkfiles.pl 1>MINFO
H:\build\openssl-0.9.8f>perl util\mk1mf.pl no-asm VC-WIN32 1>ms\nt.mak
H:\build\openssl-0.9.8f>perl util\mk1mf.pl dll no-asm VC-WIN32 1>ms\ntdll.mak
H:\build\openssl-0.9.8f>perl util\mk1mf.pl no-asm VC-CE 1>ms\ce.mak
%OSVERSION% is not defined at util/pl/VC-32.pl line 41.
Compilation failed in require at util\mk1mf.pl line 138.
H:\build\openssl-0.9.8f>perl util\mk1mf.pl dll no-asm VC-CE 1>ms\cedll.mak
%OSVERSION% is not defined at util/pl/VC-32.pl line 41.
Compilation failed in require at util\mk1mf.pl line 138.

Line 41 in util/ms/VC-32.pl is inside the WinCE elsif statement. Now if I remove all the elsif to make it drop into "else" (for Win32) all works good and everything compiles during the preceding nmake.

I see I was a little off on my original post.
I've never run into this with 0.9.7x and did not run into this on 098d&e so I was a little taken back this time around.

I read your report on the unrelated bug and do not have 1 single line as you had stated in any error log. I however specifically recompiled Apache 2.2.6 against 098f and did update the 2 dll files to the 098f build in %System%. Since this particular vulnerability did have nothing to do with Apache 2.2 I may have not even tried this, however I was unsure exactly what all this would affect on the 13th and I got to the point of no return anyway so I just hammered it out.
Back to top
glsmith
Moderator


Joined: 16 Oct 2007
Posts: 2268
Location: Sun Diego, USA

PostPosted: Wed 17 Oct '07 22:27    Post subject: Reply with quote

quick note:

I'm not new to compiling OpenSSL, I have done it many times successfully and some times not. I am new *hangs head* to actually tring to understand what is going on when it is doing it.

I have no clue where to really begin on the masm error but for the do_ms one I think I found at least my problem.

in ms\do_ms.bat lines 5 & 6 are the culpret
perl util\mk1mf.pl no-asm VC-CE >ms\ce.mak
perl util\mk1mf.pl dll no-asm VC-CE >ms\cedll.mak


Changing them from VC-CE to VC-WIN32 and it all worked wonderfully.
I do not believe ce.mak nor cedll.mak get used during compile so removing them may work as well.

I did start with fresh source and used
perl util\mkdef.pl crypto ssl update
right at the beginning before configure.
Back to top
tdonovan
Moderator


Joined: 17 Dec 2005
Posts: 611
Location: Milford, MA, USA

PostPosted: Thu 18 Oct '07 1:23    Post subject: Reply with quote

re: the MASM errors
Starting with 0.9.8f, OpenSSL requires a version of MASM to compile the SSE2 instructions for recent Pentium processors using the XMMWORD directive.

MASM v8 will compile this directive - but apparently MASM v6 will not. I am surprised that this has not been previously reported as an OpenSSL bug. I entered OpenSSL bug 1592 for this. I doubt they intended to use a MASM-v8-only directive.

If you want to work around this problem with OpenSSL 0.9.8f and build it with MASM v6.15+, you could edit the file crypto\sha\asm\sha512-sse2.asm prior to running NMAKE and change every occurrence of XMMWORD to QWORD.

Any recent version of NASM works OK with ms\do_Nasm.bat. Note that the NASM executable must be named nasmw.exe on Windows, and must be in your PATH.

The "%OSVERSION%" and "Compilation failed" warnings from ms\do_ms.bat are benign unless you are building for Windows-CE.
I updated the build instructions above to note this. You can continue on to NMAKE -f ms\ntdll.mak and OpenSSL will build OK.

-tom-


Last edited by tdonovan on Fri 19 Oct '07 15:22; edited 2 times in total
Back to top
glsmith
Moderator


Joined: 16 Oct 2007
Posts: 2268
Location: Sun Diego, USA

PostPosted: Thu 18 Oct '07 3:32    Post subject: Reply with quote

note to self, ignore those

Thanks Tom
Back to top
tdonovan
Moderator


Joined: 17 Dec 2005
Posts: 611
Location: Milford, MA, USA

PostPosted: Sun 21 Oct '07 18:01    Post subject: Reply with quote

OpenSSL Bug 1592 which caused problems building with MASM v6 has been fixed for the next release of OpenSSL 0.9.8.

The OpenSSL developers note in the bug that NASM is preferred over MASM for building OpenSSL.
Only NASM will be supported in the next major release - OpenSSL 0.9.9.

If you don't already have NASM, you might want to consider installing it and using ms\do_Nasm.bat to build OpenSSL.

-tom-
Back to top
VoodooMill



Joined: 11 Jan 2007
Posts: 60

PostPosted: Wed 31 Oct '07 19:55    Post subject: Reply with quote

Terrific write up Tom!
Back to top


Reply to topic   Topic: OpenSSL details View previous topic :: View next topic
Post new topic   Forum Index -> Building & Member Downloads