[Discuss] Support providing hints for breaking changes

[jsoriano]

There are situations where breaking changes may be unavoidable. Currently packages can indicate this in changelogs and/or bumping their major versions, but from a user perspective upgrading to versions with breaking changes looks the same as any other upgrade. After the upgrade they may be missing fields or having other kinds of issues.

It should be possible to include hints in packages to handle breaking changes. There can be two kinds of hints:

• Hints for Fleet, in case it can do anything during the upgrade.
• Hints for users, in case they need to be aware of some change or perform some manual action.

These hints should help with the migration path to versions with breaking changes.

There are some places where the hints could appear:

• In breaking change notes in the changelog, a new field could be added with additional information. We should ensure that breaking changes are displayed to users in the UI when upgrading ([Fleet] Detect breaking changes in integration change logs and flag them to users during integration upgrades kibana#187481), and if possible also in API responses.
• In field definitions, we could keep definitions of removed fields, along with additional information to indicate if it was completely removed, moved or whatsoever.
• In READMEs we could also add some notes, but this can be difficult to enforce and users may not read it.

We should also assess frequent reasons for breaking changes.

Try to find opportunities to improve the situation here.

Related issues

• Discuss: guidelines for a package with breaking changes #229
• [Fleet] Detect breaking changes in integration change logs and flag them to users during integration upgrades kibana#187481
• Add assistance to detect common backwards compatibilty issues elastic-package#579. In this issue we help preventing breaking changes, but what to do next after detecting a breaking change if the change cannot be avoided?
• Discuss: deprecation path for packages #192 and Support package deprecation #227 contain discussions about deprecations, an special case of breaking changes.

[Oddly]

Not sure if this is the right place, but deprecation hints when using elastic-package to build packages would be very welcome for those of us who build our own packages.

[jlind23]

@kpollich @jamiehynds @nimarezainia instead of over engineering this could we imagine an easy solution which would be to display a pop up warning the users whenever there is major version bumped in the package ready to be installed? This would avoid us to implement any flags or any logic in the package spec itself. Any thoughts?

[kpollich]

Do we enforce a major version bump in the integration's version when its targeted stack/Kibana version is bumped by a major? I think this could be a simple way to get some of this behavior, but I don't know that we have any kind of enforcement on package maintainers that would guarantee we trigger the pop-up when it's intended.

[ebeahan]

I don't know that we have any kind of enforcement on package maintainers that would guarantee we trigger the pop-up when it's intended.

The changelog spec includes a breaking-change type maintainers should use for any breaking changes.

Maybe the upgrade banner includes a warning if a newer changelog entry contains a breaking-change?