v1.4.2

Release 1.4.2

• Bug fixes

Component Updates and Bug Fixes

• github.com/openziti/ziti: v1.4.1 -> v1.4.2
  • Issue #2860 - router healtcheck with invalid address logs error but still doesn't listen

v1.4.1

Release 1.4.1

What's New

• Bug fixes

Component Updates and Bug Fixes

• github.com/openziti/ziti: v1.4.0 -> v1.4.1
  • Issue #2825 - create config should reference client chain
  • Issue #2845 - 1.4.x Router - don't fail on invalid cert for heathchecks

v1.4.0

Release 1.4.0

What's New

• Changes to backup/restore and standalone to HA migrations
• Use cluster consistently for cluster operations
• Event Doc and Consistency
• ziti ops verify changes
  • Moved ziti ops verify-network to ziti ops verify network
  • Moved ziti ops verify traffic to ziti ops verify traffic
  • Added ziti ops verify ext-jwt-signer oidc to help users with configuring OIDC external auth
  • Added ziti ops verify ext-jwt-signer oidc to help users with configuring OIDC external auth
• Router Controller Endpoint Changes
• Bug fixes

Config Changes

NOTE: For HA configuration, the raft: config stanza is now named cluster:.

Example:

cluster:
  dataDir: ./data

Event Doc and Consistency

The event types are now exhaustively documented as part of the OpenZiti Reference Documentation.

During documentation, some inconsistencies were found the following changes were made.

Namespace Cleanup

Namespaces have been cleaned up, with the following changes:

• edge.apiSessions -> apiSession
• fabric.circuits -> circuit
• edge.entityCount -> entityCount
• fabric.links -> link
• fabric.routers -> router
• services -> service
• edge.sessions -> session
• fabric.terminators -> terminator
• fabric.usage -> usage

Note that service events used services in the config file, but service.events in the event itself.
The old namespaces still work. If the old event type is used in the config file, the old namespace will be in the events as well

Timestamp field

The following event types now have a timestamp field:

• service
• usage

This timestamp is the time the event was generated.

Event Source ID

All event types now have a new field: event_src_id. This field is the id of the controller
which emitted the event. This may be useful in HA environments, during event processing.

Cluster Operations Naming

The CLI tools under ziti fabric raft are now found at ziti ops cluster.

The Raft APIs available in the fabric management API are now namespaced under Cluster instead.

Backup/Restore/HA Migrations

What restoring from a DB snapshot has in common with migrating from a standalone setup to
a RAFT enabled one, is that the controller is changing in a way that the router might not
notice.

Now that routers have a simplified data model, they need know if the controller database
has gone backwards. In the case of a migration to an HA setup, they need to know that
the data model index has changed, likely resetting back to close to zero.

To facilitate this, the database now has a timeline identifier. This is shared among
controllers and is sent to routers along with the data state. When the controller
restores to a snapshot of previous state, or when the the controller moves to a
raft/HA setup, the timeline identifier will change.

When the router requests data model changes, it will send along the current timeline
identifier. If the controller sees that the timeline identifier is different, it knows
to send down the full data state.

Implementation Notes

In general this is all handled behind the scenes. The current data model index and
timeline identifier can be inspected on controllers and routers using:

ziti fabric inspect router-data-model-index

Example

$ ziti fabric inspect router-data-model-index
Results: (3)
ctrl1.router-data-model-index
index: 25
timeline: MMt19ldHR

vEcsw2kJ7Q.router-data-model-index
index: 25
timeline: MMt19ldHR

ctrl2.router-data-model-index
index: 25
timeline: MMt19ldHR

Whenever we create a database snapshot now, the snapshot will contain a flag indicating
that the timeline identifier needs to be changed. When a standalone controller starts
up, if that flag is set, the controller changes the timeline identifier and resets the flag.

When an HA cluster is initialized using an existing controller database it also changes the
timeline id.

HA DB Restore

There's a new command to restore an HA cluster to an older DB snapshot.

ziti agent controller restore-from-db </path/to/database.file>

Note that when a controller is already up and running and receives a snapshot to apply, it
will move the database into place and then shutdown, expecting to be restarted. This is
because there is caching in various places and restartingi makes sure that everything is
coherent with the changes database.

Router Controller Endpoint Updates

Endpoints File Config

The config setting for controller the endpoints file location has changed.

It was:

ctrl:
  dataDir: /path/to/dir

The endpoints file would live in that directory but always be called endpoints.

This is replaced by a more flexible endpointsFile.

ctrl:
  endpointsFile: /path/to/endpoints.file

The default location is unchanged, which is a file named endpoints in the same
directory as the router config file.

Enrollment

The router enrollment will now contain the set of known controllers at the time
the router as enrollled. This also works for standalone controllers, as long as
the advertiseAddress settings is set.

Example

ctrl:
  listener: tls:0.0.0.0:6262
  options:
    advertiseAddress: tls:ctrl1.ziti.example.com

This means that the controller no longer needs to be set manually in the config
file, enrollment should handle initializing the value appropriately.

Component Updates and Bug Fixes

• github.com/openziti/agent: v1.0.23 -> v1.0.26

• github.com/openziti/channel/v3: v3.0.26 -> v3.0.37

  • Issue #168 - Add DisconnectHandler to reconnecting channel

• github.com/openziti/edge-api: v0.26.38 -> v0.26.41

• github.com/openziti/foundation/v2: v2.0.56 -> v2.0.58

• github.com/openziti/identity: v1.0.94 -> v1.0.100

• github.com/openziti/metrics: v1.2.65 -> v1.2.69

• github.com/openziti/runzmd: v1.0.59 -> v1.0.65

• github.com/openziti/sdk-golang: v0.23.44 -> v0.24.1

  • Issue #673 - Add license check to GH workflow
  • Issue #663 - Add API to allow controlling proxying connections to controllers and routers.
  • Issue #659 - E2E encryption can encounter ordering issues with high-volume concurrent writes

• github.com/openziti/secretstream: v0.1.28 -> v0.1.31

• github.com/openziti/storage: v0.3.15 -> v0.4.5

  • Issue #94 - Snapshots aren't working correctly

• github.com/openziti/transport/v2: v2.0.159 -> v2.0.165

• github.com/openziti/xweb/v2: v2.1.3 -> v2.2.1

  • Issue #18 - verify advertised host/ip has a certificate defined in the identity block

• github.com/openziti/ziti: v1.3.3 -> v1.4.0

  • Issue #2807 - Cache ER/T terminator ids in the router for faster restarts
  • Issue #2288 - Edge router/tunneler hosting Chaos Test
  • Issue #2821 - Add --human-readable and --max-depth options to ziti ops db du
  • Issue #2742 - Add event when non-member peer connects and doesn't join
  • Issue #2738 - Cluster operations should return 503 not 500 if there's no leader
  • Issue #2712 - /version is missing OIDC API
  • Issue #2785 - Peer data model state not always updated
  • Issue #2737 - OIDC issue mismatch with alt server certs
  • Issue #2774 - API Session Certificate SPIFFE IDs fail validation in Routers
  • Issue #2672 - [Bug] Posture check PUT method doesn't update nested structures but works fine with PATCH
  • Issue #2668 - [Feature Request] Filterable field for posture check type
  • Issue #2681 - Support specifying which token to use on external jwt signers
  • Issue #2756 - Remove ziti agent cluster init-from-db command
  • Issue #2723 - attempts to probe advertise address on startup to ensure the SANS is correct
  • Issue #2722 - router: check advertised address on startup
  • Issue #2745 - Remove cluster initialMembers config
  • Issue #2746 - Move agent controller commands to agent cluster
  • Issue #2743 - Agent and rest cluster command names should match
  • [Issue #2731](https://github.com/openziti/zi...

v1.3.3

Release 1.3.3

What's New

• Bug Fixes

Component Updates and Bug Fixes

• github.com/openziti/ziti: v1.3.2 -> v1.3.3
  • Issue #2694 - Router should use router data model if it has more than one controller configured, regardless of controller configuration

v1.3.2

Release 1.3.2

What's New

• Bug Fixes

Component Updates and Bug Fixes

• github.com/openziti/ziti: v1.3.1 -> v1.3.2
  • Issue #2692 - Routers get empty ctrl list on upgrade
  • Issue #2689 - OIDC authentication with form data requires "id" in form data, authReqeustID in query string is ignored

v1.3.1

Release 1.3.1

What's New

• Bug Fixes

Component Updates and Bug Fixes

• github.com/openziti/ziti: v1.3.0 -> v1.3.1
  • Issue #2682 - HA Controller panics when bootstrapping by setting the db variable in the configuration
  • Issue #2683 - Controller fails to save peer configuration on a fresh install
  • Issue #2684 - Controller emits duplicate cluster events on startup

v1.3.0

Release 1.3.0

What's New

• Router Data Model enabled by default
• Bug fixes
• Controller Health Check HA Update (from @nenkoru)

Router Data Model

As part of the controller HA work, a stripped down version of the data model can now be distributed to the routers,
allowing routers to make some authorization/authentication decisions. This code has existed for some time, but
after testing and validation, is now enabled by default.

It can still be disabled at the controller level using new configuration. Note that the router data model is required
for HA functionality, so if the controller is running in HA mode, it cannot be disabled.

routerDataModel:
  # Controls whether routers are told to enable functionality dependent on the router data model
  # Defaults to true
  enabled: true 

  # How many model changes to buffer so that routers can be updated iteratively. If a router requests
  # data that's no longer available, it will receive the full data model
  logSize: 10000

HA Changes

Routers no longer require the ha: enabled flag be set in the configuration. Routers should work correctly
whether connecting to HA or non-HA controllers.

NOTE: If the controller a router is connected changes modes, specifically if the controller goes from
supporting the router data model to not, or vice-versa, the router will shutdown so that it can
restart with the correct mode.

Controller Health Check HA Update

This feature was contributed by @nenkoru.

The controller health check can now optionally return information about raft and leadership when the /controller/raft path is provided.

$ curl -k https://localhost:1280/health-checks/controller/raft
{
    "data": {
        "checks": [
            {
                "healthy": true,
                "id": "bolt.read",
                "lastCheckDuration": "0s",
                "lastCheckTime": "2025-01-14T19:42:13Z"
            }
        ],
        "healthy": true
    },
    "meta": {},
    "raft": {
        "isLeader": true,
        "isRaftEnabled": true
    }
}

Note the raft section, which indicates if raft is enabled and if the queried controller is currently the leader. If the
controller/raft path isn't present in the request, the result should be unchanged from previous releases.

When querying the controller/raft health, if raft is enabled but the controller is not the leader, the check will
return an HTTP status of 429.

Component Updates and Bug Fixes

• github.com/openziti/agent: v1.0.20 -> v1.0.23

• github.com/openziti/channel/v3: v3.0.16 -> v3.0.26

• github.com/openziti/edge-api: v0.26.35 -> v0.26.38

  • Issue #138 - management api deletes were generally not mapping 404 properly

• github.com/openziti/foundation/v2: v2.0.52 -> v2.0.56

• github.com/openziti/identity: v1.0.90 -> v1.0.94

• github.com/openziti/metrics: v1.2.61 -> v1.2.65

• github.com/openziti/runzmd: v1.0.55 -> v1.0.59

• github.com/openziti/secretstream: v0.1.26 -> v0.1.28

• github.com/openziti/storage: v0.3.8 -> v0.3.15

  • Issue #91 - Support dashes in identifier segments after the first dot

• github.com/openziti/transport/v2: v2.0.153 -> v2.0.159

• github.com/openziti/ziti: v1.2.2 -> v1.3.0

  • Issue #2674 - 404 not found on well-known OIDC configuration with default ports/localhost
  • Issue #2669 - Router api session tracker leaks memory.
  • Issue #2659 - OIDC Login Panics On Unsupported Media Type
  • Issue #2582 - An endpoint to determine whether a node is a raft leader
  • Issue #2619 - Add source id to all events
  • Issue #2644 - enhance mismapped external identity logging
  • Issue #2636 - Enable HA smoketest
  • Issue #2586 - Ziti Controller in HA mode doesn't update binding address in a bolt database after config changed
  • Issue #2639 - Change cluster events namespace from fabric.cluster to cluster
  • Issue #2184 - Add Event(s) For Controller Leader Connection State
  • Issue #2548 - Generate a log message if the cluster is without a leader for some configurable period of time
  • Issue #2624 - Remove uri/params from connect events
  • Issue #2596 - Add DisableRouterDataModel config flag to controller
  • Issue #2599 - Routers should only stream model data from one controller
  • Issue #2232 - Standardized REST API Error For Mutation on Non-Consensus Controller
  • Issue #2566 - Remove HA config flag from router
  • Issue #2550 - Router Data Model Chaos Test
  • Issue #2625 - edge sessions for an ERT may not be cleaned up when the ER/T is deleted
  • Issue #2591 - Split Edge APIs can cause ziti edge login to fail

v1.1.16

Release 1.1.16

What's New

Update golang.org/x dependencies, specifically golang.org/x/crypto, for latest security fixes

v1.2.2

Release 1.2.2

What's New

• Bug fixes and continuing progress on controller HA

Component Updates and Bug Fixes

• github.com/openziti/secretstream: v0.1.25 -> v0.1.26

• github.com/openziti/storage: v0.3.6 -> v0.3.8

  • Issue #87 - negative URL filter returns incorrect results

• github.com/openziti/ziti: v1.2.1 -> v1.2.2

  • Issue #2559 - expired JWTs are allowed to enroll
  • Issue #2543 - Support adding adding an uninitialized node to a cluster (rather than the reverse)

v1.2.1

Release 1.2.1

What's New

• Bug fixes and continuing progress on controller HA

Component Updates and Bug Fixes

• github.com/openziti/agent: v1.0.19 -> v1.0.20
• github.com/openziti/channel/v3: v3.0.10 -> v3.0.16
• github.com/openziti/foundation/v2: v2.0.50 -> v2.0.52
• github.com/openziti/identity: v1.0.88 -> v1.0.90
• github.com/openziti/metrics: v1.2.59 -> v1.2.61
• github.com/openziti/runzmd: v1.0.53 -> v1.0.55
• github.com/openziti/storage: v0.3.2 -> v0.3.6
• github.com/openziti/transport/v2: v2.0.150 -> v2.0.153
• github.com/openziti/ziti: v1.2.0 -> v1.2.1
  • Issue #2543 - Support adding adding an uninitialized node to a cluster (rather than the reverse)
  • Issue #2541 - Add cluster id, to prevent merging disparate clusters
  • Issue #2532 - When adding an existing HA cluster member, remove/add if suffrage has changed
  • Issue #2217 - Controller list is empty until peers connect
  • Issue #2533 - Handle concurrent raft connections
  • Issue #2534 - Ziti ID with leading hyphen causes command-line parameter ambiguity
  • Issue #2528 - Updated router costs are not use when evaluating current path cost in the context of smart rerouting