Enable PGMQ to run without installing an extension

[axelfontaine]

This pull request enables the installation of PGMQ using a simple psql command:

psql < pgmq.sql

It does NOT affect the existing functionality, nor does it impact PGMQ's ability to be installed as an extension.

This addresses the following issues directly and inderectly:

• Question: Does this work on hosted providers? #205
• Don't assume to be installed as an extension #347
• Add pgmq extension to Microsoft Azure Postgres flexible server (upvote link) #373
• GCP Support #378

To achieve this the following was changed:

• The pgmq schema gets created if it doesn't already exist
• The actual implementation of send and send_batch are moved higher up the SQL file so they can be found when referenced by the convenience functions with less arguments
• A new _extension_exists function is introduced to check the existance of arbitrary extensions
• The implementation of _ensure_pg_partman_installed has been retrofitted to use the new _extension_exists function.
• All references to the pgmq extension have been guarded with _extension_exists('pgmq')

[ChuckHend]

This is awesome, thank you for doing this @axelfontaine. I will look it over in the next few days.

[ChuckHend]

image builds are going to fail for PR coming from an external repo (we still need a better way to configure that in CI), so we can ignore those failures.

[v0idpwn]

This is great. I think we should have tests run on it, though.

[ChuckHend]

This is great. I think we should have tests run on it, though.

Tests would be great. At first I was thinking of maybe adding another workflow like the existing extension_upgade.yml, except for the stand-alone mode. Except thinking that through, I'm not sure what upgrading looks like. Do we need to track the current version in some table when in stand-alone mode, then execute the migrate files? I'll think this through a bit more, maybe there is a simpler way.

[axelfontaine]

@ChuckHend Versioning is not something PGMQ should be concerned with. I use Flyway, but other similar tools will also do the job.

Upgrading means copying the PGMQ upgrade script into one's project and giving it the next version according to Flyway's (or whatever other tool's) naming convention.

Or simply applying it manually using psql.

PGMQ should exclusively deal with queueing. Let other specialized tools handle the versioning.

[ChuckHend]

Understood. I was thinking of this from the perspective of providing a similar experience to what pgmq has as a Postgres extension, but as stand-alone functionality. It is helpful for me, when troubleshooting, to know which version of the pgmq project folks are running, and versioning is already part of our release process. We don't need to block this PR on versioning though.

I do think we need to have some sort of test in place for executing the migration sql files for stand-alone pgmq though, so that we don't break it for all the non extension users. @axelfontaine what do you think?

[axelfontaine]

It is helpful for me, when troubleshooting, to know which version of the pgmq project folks are running, and versioning is already part of our release process.

This can be tackled by providing a pgmq.version() function that returns a hardcoded string with the version. Simple and effective, without getting tangled up in the complexity web of a full migration solution. I worked on Flyway for 10 years before selling it in 2019. Trust me, I know what I am talking about 😉

I do think we need to have some sort of test in place for executing the migration sql files for stand-alone pgmq though, so that we don't break it for all the non extension users. @axelfontaine what do you think?

Agreed. I actually think all tests should be decoupled from GHA and also easily runnable locally at least on Linux, but ideally on all platforms (Win/Mac/Linux). I see that as being separate from this PR as well though.

[ChuckHend]

Can confirm this did not break any existing functionality. I also ran the python and Rust client tests against this branch.