Recently I was trying to set up an OpenVPN server on pfSense using an Ubuntu 11.10.  Following the guides http://doc.pfsense.org/index.php/VPN_Capability_OpenVPN I found I ran into a problem when generating the “server.key” file using the easy-rsa toolset that comes with OpenVPN.  It was producing a server.key file with the following headers and footers :

-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----

pfSense v.1.2.3a OpenVPN server.key setup was expecting it to read as follows :

-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----

This —-BEGIN PRIVATE KEY—- results in a error message in pfSense when it tries to use save the settings, and before you maybe tempted to try, you cannot simply rename the header and footer by adding “RSA” as this results in invalid key errors when the vpn is initiated, these are found in the pfSense error logs.

It seems it is something to do with the version of OpenSSL I was running on the rig.  On my older 10.04 Ubuntu installation I had OpenSSL 0.98a on my newer 11.10 rig I have OpenSSL 1.0.0e 6 Sep 2011.  Creating the server.key on 0.98a by default created the RSA PRIVATE KEY, creating the server.key on 1.0.0e did not create a RSA Private Key, but an encrypted PRIVATE KEY.

Running the following :

openssl rsa -in server.key -out decryptedserver.key

generated the RSA decrypted server.key file that can simply be cut and pasted into the pfSense v1.2.3a OpenVPN server configuration page.  Solved!