Configuring High Availability (HA)

Configure high availability (HA) for flow policy by preparing the primary and standby hosts, distributing SSH access, uploading the required files, and scheduling the HA check.

There are two HA patterns:

  • api, workspace, and safe HA: use an external load balancer.
  • Flow policy HA: use this guide to configure failover between a primary host and a standby host.

Prerequisites

  1. Identify the primary host that currently runs the flow policy service.
  2. Identify the standby host that should take over when the primary host is unavailable.
  3. Sign in to both hosts with a user that can copy files, edit crontab, and restart the flow policy service.
  4. Confirm SSH access from the standby host to the primary host.
  5. Prepare the HA files that must be uploaded to dmax.

Configure flow policy HA

  1. Sign in to the standby host as the dmax user.

    ssh dmax@STANDBY_HOST
  2. Create an SSH key on the standby host.

    ssh-keygen -t ed25519 -f ~/.ssh/flow_policy_ha -C "flow-policy-ha"
  3. Copy the standby host public key to the primary host.

    ssh-copy-id -i ~/.ssh/flow_policy_ha.pub dmax@PRIMARY_HOST
  4. Verify that the standby host can connect to the primary host without a password prompt.

    ssh -i ~/.ssh/flow_policy_ha dmax@PRIMARY_HOST

    A successful connection prints the primary host name

  5. Upload the HA files to dmax on the secondary host.

    📘

    Important

    On version 1.40.3+ the files are already on the dmax and there is no need to upload them

    HA files

  6. Make the HA script executable on the standby host.

    chmod +x /home/dmax/DefenderMax/HA/remote-file-watcher.sh
    chmod +x /home/dmax/DefenderMax/HA/website-failover.sh
  7. Add the HA check to the standby host crontab.

    crontab -e

    Add this line to run the check every minute:

    * * * * * /home/dmax/DefenderMax/HA/website-failover.sh -c /home/dmax/DefenderMax/HA/website-failover.sh -o
    * * * * * /home/dmax/DefenderMax/HA/remote-file-watcher.sh -c /home/dmax/DefenderMax/HA/remote-file-watcher.conf -o
    

Verify HA

  1. Check the HA log on the standby host.

    tail -f /home/dmax/DefenderMax/HA/remote-file-watcher.log
    tail -f /home/dmax/DefenderMax/HA/website-failover.log
  2. Test failover during a maintenance window by shutting down the primary unit

Troubleshooting

IssueCauseFix
SSH still asks for a passwordThe public key was not installed for dmax on the primary host.Run ssh-copy-id again and verify the target user and host.
Permission denied when cron runsThe HA script is not executable or the HA user cannot read it.Run chmod +x DMAX_PATH/HA_SCRIPT and confirm file ownership.