-
Notifications
You must be signed in to change notification settings - Fork 924
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use meson to generate docs (html + pdf) #15169
Open
omoerbeek
wants to merge
9
commits into
PowerDNS:master
Choose a base branch
from
omoerbeek:rec-meson-gen-docs
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
ce5610a
rec: generate html-docs using meson compile html-docs
omoerbeek 707a2c7
Generate tarball and pdf and introduce all-docs target
omoerbeek 5f58af5
dnsdist docs (html-docs, tarball and pdf)
omoerbeek 379b8d7
And also do html-docs, tarball and pdf for auth
omoerbeek 112d834
Only run python if found
omoerbeek 1fb0df5
Add all '*.rst' files in docs as a dependency.
omoerbeek 7ffce91
Fix dependencies for docs, using a Sphinx extension to generate a dep…
omoerbeek fe1a594
Ignore docs/depfile.py for spell checking
omoerbeek 6fd135e
Update .github/actions/spell-check/excludes.txt
omoerbeek File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| """Generate docs using sphinx in a venv.""" | ||
|
|
||
| import argparse | ||
| import glob | ||
| import itertools | ||
| import os | ||
| import subprocess | ||
| import sys | ||
| import venv | ||
| from pathlib import Path | ||
|
|
||
|
|
||
| def main(): | ||
| """Start the script.""" | ||
| args = create_argument_parser() | ||
|
|
||
| source_root = args.source_root | ||
| build_root = args.build_root | ||
|
|
||
| # Create the venv. | ||
| venv_directory = build_root.joinpath(args.venv_name) | ||
| venv.create( | ||
| venv_directory, | ||
| with_pip=True, | ||
| clear=True, | ||
| upgrade_deps=True, | ||
| prompt=args.venv_name, | ||
| ) | ||
|
|
||
| # Install some stuff into the venv. | ||
| requirements_file = source_root.joinpath(args.requirements_file) | ||
| pip = venv_directory.joinpath("bin").joinpath("pip") | ||
| subprocess.run([pip, "install", "-U", "pip", "setuptools", "wheel"], check=True) | ||
| subprocess.run([pip, "install", "-r", requirements_file], check=True) | ||
|
|
||
| # Run sphinx to generate the docs | ||
| source_directory = source_root.joinpath(args.source_directory) | ||
| target_directory = build_root.joinpath(args.target_directory) | ||
| files = [glob.glob(str(source_root.joinpath(pat))) for pat in args.files] | ||
| files = list(itertools.chain.from_iterable(files)) | ||
| sphinx_build = venv_directory.joinpath("bin").joinpath("sphinx-build") | ||
|
|
||
| if args.pdf_name: | ||
| build_args = [ | ||
| sphinx_build, | ||
| "-M", | ||
| "latexpdf", | ||
| source_directory, | ||
| '.' | ||
| ] | ||
| else: | ||
| build_args = [ | ||
| sphinx_build, | ||
| "-b", | ||
| "html", | ||
| source_directory, | ||
| target_directory, | ||
| ] | ||
| subprocess.run( | ||
| build_args + files, # if files is empty, it means do all files | ||
| check=True | ||
| ) | ||
| if args.pdf_name: | ||
| os.rename(build_root.joinpath('latex').joinpath(args.pdf_name), args.pdf_name) | ||
|
|
||
| def create_argument_parser(): | ||
| """Create command-line argument parser.""" | ||
| parser = argparse.ArgumentParser( | ||
| description="Create a virtualenv from a requirements file" | ||
| ) | ||
| parser.add_argument( | ||
| "--build-root", | ||
| type=Path, | ||
| required=True, | ||
| help="Build root", | ||
| ) | ||
| parser.add_argument( | ||
| "--source-root", | ||
| type=Path, | ||
| required=True, | ||
| help="Source root", | ||
| ) | ||
| parser.add_argument( | ||
| "--venv-name", | ||
| type=str, | ||
| required=True, | ||
| help="Name for the virtualenv", | ||
| ) | ||
| parser.add_argument( | ||
| "--requirements-file", | ||
| type=Path, | ||
| required=True, | ||
| help="Package requirements file relative to the source root", | ||
| ) | ||
| parser.add_argument( | ||
| "--source-directory", | ||
| type=Path, | ||
| required=True, | ||
| help="Docs directory relative to the source root (contains conf.py)", | ||
| ) | ||
| parser.add_argument( | ||
| "--target-directory", | ||
| type=Path, | ||
| required=True, | ||
| help="Target directory for man-pages relative to the build root", | ||
| ) | ||
| parser.add_argument( | ||
| "--pdf-name", | ||
| type=Path, | ||
| required=False, | ||
| help="Generate pdf instead of html", | ||
| ) | ||
| parser.add_argument( | ||
| "files", | ||
| type=Path, | ||
| nargs="*", | ||
| help="Input files relative to the source root", | ||
| ) | ||
| return parser.parse_args() | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../../../docs/generate-docs.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../../../docs/generate-docs.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This won't be updated when new files are added. The QEMU project faced this challenge already, and wrote a neat trick to solve it:
https://gitlab.com/qemu-project/qemu/-/blob/master/docs/sphinx/depfile.py
This allows you to use
depfile: ...and have sphinx report what files it used and which should trigger further rebuilds.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I mentioned the drawback in the commit message. I'll look into it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, Meson isn't a fan of this run_command method... because it's better done by depfiles any time you don't explicitly need the input files as command line arguments! :)
(When you do need them as command-line arguments, for example with
executable()sources, I tend to recommend running a generator script as a pre-commit hook, then usingfs.read('sources.list').strip().split('\n')instead.fs.readwill produce reconfigure dependencies, much like an automake include.)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-->
ninja -d listWe use depfiles for all *.o outputs as well. Ninja reads them in after the command finishes for build nodes that declare they provide depfiles, and compiles them into a binary database (
.ninja_deps) for, as you guessed, optimization purposes. No need to read tons of tiny files in on every startup. :)Hence also why keeping the file around is one of the options exposed in the debugging modes. Though, I don't know why it is necessary to delete them after reading them, but maybe the ninja authors figured the file is useless after reading it, so might as well compact the build directory size a tiny bit.