Skip to content

Enabling IPSec on SmartOS

Jump to bottom
Jan Poctavek edited this page Dec 10, 2017 · 14 revisions

Enabling IPSec on SmartOS is similar for both - global zone and non-global zone. However, when enabling IPSec in global zone, you have to ensure that changes persist between reboots.

To enable IPSec communication between the two SmartOS systems you need to create/edit following files:

  • /etc/inet/ike/config
  • /etc/inet/secret/ike.preshared
  • /etc/inet/secret/ipsecinit.conf

Following SMF services need to be enabled/restarted after the configuration is done:

svc:/network/ipsec/ike:default
svc:/network/ipsec/ipsecalgs:default
svc:/network/ipsec/policy:default

ipsec/policy is dependent on ipsec/ipsecalgs. And ipsec/ike is disabled by default.

Tools that you will encounter during setup:

  • in.iked - managed by SMF service svc:/network/ipsec/ike:default
    ** reads /etc/inet/ike/config, /etc/inet/secret/ike.preshared
  • ikeadm - used for inspecting and administration of the in.iked
    ** ikeadm set debug 0x3ff -> will enable all debug flags

IKE - Internet Key Exchange

IKE is used to negotiate SA (Security Association) between two hosts. First of all, to define overall configuration of the IPSec, you need to edit /etc/inet/ike/config. Based on this file the keys for IPSec encryption are exchanged. Here you specify the phase 1 and 2 parameters.

Furthermore, this is where you define:

  • whether IPSec will be using certificates or pre-shared keys
  • whether main or aggressive mode should be used

The file operates by having 4 types of parameters:

  1. global params
  2. IKE phase 1 transform defaults
  3. IKE rules defaults
  4. IKE rules

The following configuration sets global phase 1 and phase 2 configuration, and then more specific for the two hosts listed by local_addr and remote_addr. local_addr and remote_addr can also define specific subnet to which the IKE rule will apply.

p1_xform
  { auth_method preshared oakley_group 5 auth_alg sha encr_alg 3des }
p2_pfs 2
{ label "test1-test2" local_addr 10.0.60.185 remote_addr 10.0.60.188 p1_xform { auth_method preshared oakley_group 5 auth_alg sha256 encr_alg aes } p2_pfs 5 }

The above defines following: default P1 transform will be using pre-shared keys as authorization method, and Oakley DH group for deriving keys for SA (security association) with number 5, which based on values on man page is MODP 1356-bit. The authorization algorithm is SHA and encryption algorithm is 3des. For P2 we define PFS (perfect forward secrecy) to be oakley group 1024-bit. Between curly braces, starting with label... is the IKE rule for two specific hosts. What is different are authentication algorithm (SHA256) and encryption algorithm (AES), and for PFS we use oakley group 1536-bit.

Next we need to define our pre-shared keys for IKE. This howto will not tell you how to do authentication using certificates (PKI), but it can be found in man page of ikecert. But as a general guideline you need to edit/provide following files:

/etc/inet/secret/ike.privatekeys/
/etc/inet/ike/publickeys/
/etc/inet/ike/crls/

And of course, you need to reconfigure /etc/inet/ike/config to be aware that it should use certificates, instead of pre-shared keys.

Lets look at the configuration of the pre-shared key for the above IKE config in /etc/inet/secret/ike.preshared:

{
    localidtype IP
    localid 10.0.60.185
    remoteidtype IP
    remoteid 10.0.60.188
    key "<replace-your-secret-key-here>"
}

You need to specify local and remote hosts' IPs, that will use this pre-shared key. If you don't use quotes, the value will be interpreted as hex string.

Finally, you need to configure IPSec policies using /etc/inet/ipsecinit.conf file. You can think of this file as IP filter rules file. This is where you define which traffic gets encrypted by IPSec. The file enables very precise definition of what gets encrypted, but here is the simple example configuration:

{ rport 4789 } ipsec {encr_algs aes encr_auth_algs sha512 sa shared}
{ lport 4789 } ipsec {encr_algs aes encr_auth_algs sha512 sa shared}
{ raddr 10.0.60.0/24
    dir both
} bypass {}

# {laddr 10.0.60.185 raddr 10.0.60.188} ipsec {encr_algs aes encr_auth_algs sha512 sa shared}

The first line says that all outgoing traffic with destination port 4789 is subject to IPSec policy. Second line does the same, but for incoming traffic on port 4789. Third rule says, that any traffic (both directions) with remote address from the defined subnet will bypass the IPSec. Bypass rules come before other rules, so this would mean that servers will communicate without IPSec on 10.0.60.0/24 network. The last line, commented out here, (# is used for comments) is example how to encrypt communication between two hosts explicitly.

To display order in which rules will be applied use -l or -L with ipsecconf command:

ipsecconf -l

Using ipsecconf without any arguments will display current set of rules loaded by ipsec:policy SMF. For the above configuration you would see something like this:

[root@smartos ~]# ipsecconf
#INDEX 90 
{ rport 4789 } ipsec {encr_algs aes encr_auth_algs sha512 sa shared}

#INDEX 94 
{ lport 4789 } ipsec {encr_algs aes encr_auth_algs sha512 sa shared}

#INDEX 96 
{ raddr 10.0.60.0/24
    dir both
} bypass {}

After this restart IPsec SMF services:

svc:/network/ipsec/ike:default
svc:/network/ipsec/policy:default

Debugging IKE configuration

ikeadm

[root@smartos ~]# ikeadm set debug

Following are available:

           Description          Flag    Nickname
      -------------------------------------------
      Certificate management   0x0001   cert
      Key management           0x0002   key
      Operational              0x0004   op
      Phase 1 SA creation      0x0008   phase1
      Phase 2 SA creation      0x0010   phase2
      PF_KEY interface         0x0020   pfkey
      Policy management        0x0040   policy
      Proposal construction    0x0080   prop
      Door interface           0x0100   door
      Config file processing   0x0200   config
      All debug flags          0x3ff    all

Resources

Homepage | User Guide | API Reference | Wiki

Clone this wiki locally