MD5 hash optimization option if you are using 8.6.2 Cryptographic library

I thought I would share an option that is available for those using AbtMD5Engine to compute hashes often enough that it is dominating the runtime.

If you do like this option, I would encourage you to use the profiler (From the ST: Performance Workbench feature) first to determine if this is necessary since we all know that a 20x performance increase in 0.001% of your runtime costs will do nothing.

Background:
Over the years in VAST, we have been incorporating more and more SUnit test suites for product testing.  There are some test suites which require external dlls to run.  Some examples are tests for our Platform Functions, EsEntryPoints and Asynchronous calls (and things like OpenSSL).

We are working on full test automation and I've been developing a new SUnit TestResource subclass called ExternalLibraryTestResource.

Essentially, it uses ENVY Library Objects to store zip-compressed shared libraries decorated with meta-data like os type, architecture, size, timestamps and an md5 hash on the unzipped file contents.

During an SUnit test that requires this resource...if necessary, it will reify the LibraryObject stored shared library to disk and perform necessary PlatformLibrary mappings.

Part of this process is to verify what was originally stored...and what was reified...are exactly the same.

This is also used to determine if an existing shared library on disk is the expected version...in which case...nothing will be reified during the setup of an sunit test.

Since management of LibraryObjects is important here...I have utilities to import/export all shared libraries to disk in a known directory structure.  If I need to update the shared libraries...I just replace them in the various directories on disk and import the root directory structure back in.

So, the workflow is roughly

Part of import, export and disk reification during a test is to compare the stored md5 hash value to the one on disk.

I was using the AbtMD5Engine to produce a printable hash (in hex), but profiling showed that this was accounting for >= 55% of the total time to perform these utility functions.  I would be ok with this for an import/export utility that wouldn't be run all that often...but this also occurs during test suite runs.

The Crypto library from 8.6.2 has multiple digest options...one of them being md5.  I would expect these to run quite a bit quicker since the actual computations occur in C.

However, there is a rub.  You want to be sure that the crypto feature is loaded...and even then...you need the OpenSSL dlls available and working.

So I constructed the following method to dynamically choose the md5 object that will be performing the computation.

Doing this shaved quite a bit of time off import/export and put the time where I would expect it to be (according to the profiler).

Here are some timing results

Total Time for Importing Shared Libraries into LibraryObjects

AbtMD5Engine version:     7391 ms

OSSslDigest md5 version: 860  ms

Total Time for Exporting Shared Libraries from LibraryObjects

-- Seth