SPEC 13: Recommended targets and naming conventions

[tupui]

Topic raised during the summit.

Discussion: https://discuss.scientific-python.org/t/spec-13-recommended-targets-and-naming-conventions/1226

[tupui]

cc @Carreau

[Carreau]

See https://learn.scientific-python.org/development/guides/packaging-simple/#extras

[pllim]

??

Suggested change

	pyproj.toml
	pyproject.toml

[bsipocz]

I would also add some defaults for declaring optional dependencies: all, test, and docs.

[pllim]

test -> tests? 🤔

[bsipocz]

never mind, should have read all the comments below before commenting, as this was already mentioned.

[pllim]

dev would also be nice.

[bsipocz]

test --> tests, bike shedding, but I'm 👎 on this. As it's test-dependencies in my view while the other recommendation is about a directory that contains the tests

[pllim]

Just hard to remember that "s" is for testing dir (pytest tests/) but no "s" for install (pip install .[test]).

[bsipocz]

astropy does it this way since forever 😄

[henryiii]

Not too fond of [dev], this makes people think it's okay to install all your development dependencies into a single environment, which is not true. Also not fond of [all], same sort of reason. I've seen [all] used well for projects that have several optional dependencies and then provide an [all], but it's not useful to add things like [test] dependencies.

[Carreau]

We also limited the discussion to test[s] and doc[s] and did not discuss other target to now grow the spec too much.

We can extend to other naming conventions later.

[henryiii]

FYI, almost every package I work on (regardless of whether I set it up) uses [test] and [docs] (both four letters), not tests. I've even been looking at possibly helping some tools automatically find the [test] extra. Has current popularity been checked before selecting one over the other? For folder names, tests/ and docs/ are more popular, but I think extras tend to be [test] and [docs].

[henryiii]

Quick numbers on a somewhat older copy of PyPI indicates [test] is over 3x more popular than [tests]:

select COUNT(DISTINCT name) from deps where extra == 'tests';

COUNT(DISTINCT name)
2362

sqlite> select COUNT(DISTINCT name) from deps where extra == 'test';

COUNT(DISTINCT name)
7573

The reverse is true for [docs] (3638) vs. [doc] (1050).

[tupui]

I mean we had a vote here 😅 We will discuss this once more here with everyone.

[Carreau]

I think we were roughly aware that [test] is more frequent, and /tests/ more frequent, but one of the discussion we had is we believe it's better to be consistant and not test in some places and tests in other.

[henryiii]

I think you should have a very good reason to go against a 3x more popular choice. It's much harder to suggest the opposite of what everyone does and hope people change than it is to encourage a self-adopted standard. Maybe matching the folder name is good enough reason, but folder names are not the same things as extras, so not sure.

[bsipocz]

but folder names are not the same things as extras

💯

[tupui]

It takes less time for people to change things than writing a comment to argue honestly.

[Carreau]

We can also punt for now on test/tests and focus on doc/docs where there seem to be a consensus.

I personally can live with extra and folders being different as long as there is some consistency across projects.

[henryiii]

It takes less time for people to change things than writing a comment to argue honestly.

This is not remotely helpful or realistic. All CI files, most cibuildwheel configs, task runners, etc. all have to change. Then CI has to run. If everything passes, then it has to be merged and released. Then all third party packaging that uses these may also have to change. This is for every package - I maintain over 40. How is minimizing this saying "less time for people to change things" at all remotely considerate of other people's time? Aren't SPECs supposed to be community consensus? Aren't they supposed to be discussed?

[tupui]

Sorry for the joke there. I see it can have been read in a different way and here we had quite some fun making this draft. Anyway, I was only saying that besides very active people like you, this would not be that difficult for most people who barely have a single repo to handle. Also as with all the specs there is endorsement vs adoption.

[henryiii]

Okay, it rubbed me the wrong way. You have to remember this is also being proposed as a rule for repo-review, which is not just a suggestion in a document!

PS: To be clear, I'm not against the idea, and don't really care other than wanting there to be some sort of standard - [test] is just currently 3x closer to that than [tests] is. But I think punting on test vs. tests for a bit is a good idea, and it's worth seeing if we can get interest in the broader Python community to recommend one vs. the other.

If something does become standard, then automatically picking up on it if defined can simply some tooling, for example.

[Carreau]

The repo-review one is just me finding this spec as an excuse to try to understand the code of repo-review. I think it makes sens is this gets adopted to be added to repo-review (and reflected in packaging guide), but the submission to repo-review was not a global decision.

[henryiii]

I agree on that, but I think that's why it's important make sure the SPEC has the correct recommendation. Once it's in the SPEC, it would eventually come to sp-repo-review.

[Carreau]

I pushed some modifications to clarify that we punt for now on test vs tests for extra, but that in general targets for runner should be tests (spin tests, nox -s tests...)

[henryiii]

FYI, hatch test is called test. Though if you can choose, this seems fine. I do try to do nox -s tests, though a few projects I work on use nox -s test (maybe just 1-2).

Only a "preference" for a tests folder over a test folder? There's a standard library module called test, I thought that would be enough to make it a strong preference for tests...

[ofek]

https://packaging.python.org/en/latest/specifications/core-metadata/#provides-extra-multiple-use

Two feature names test and doc are reserved to mark dependencies that are needed for running automated tests and generating documentation, respectively.

I think it's okay to recommend a specific naming convention but historically my view is this: https://discuss.python.org/t/providing-a-way-to-specify-how-to-run-tests-and-docs/15016/49

Basically, I am very wary of standardization for environment manager configuration. As long as we don't start advocating for extra stuff then it seems relatively innocuous to me 🙂

[henryiii]

Wow, I didn't realize "test" and "doc" were already reserved as the testing and docs extras. Especially since "docs" is 3x more popular. Thanks!

I recommend we follow Python's standards then for extras, at least for test.

[henryiii]

I've opened pypa/hatch#1553 to suggest hatch pick up on the test extra by default.

[bsipocz]

I would like to think that adding a recommendation for all and dev are non-controversial, too?

[henryiii]

I strongly recommend no recommendation on [dev]. Having an "all in one" environment makes the environment hard to solve, sometimes impossible to solve. Solvers are very sensitive to the number of dependencies in an environment. There was a time when you couldn't install black and tensorflow in the same environment due to dependency conflicts. I've seen solvers choke for minutes¹ trying to solve updates on massive "all-in-one" environments. And you never need to install your code formatter, or pre-commit, etc. in the same environment as your package, they are stand-alone tools!

And [all] sometimes means all true optional dependencies, not testing and development dependencies. Like validate-pyproject[all].

uv is currently working on a solution for project-level "pipx"-like installs. And there is an effort to make dependency groups a standard in PEP 735. How about avoiding any extra recommendation until PEP 735 is accepted or rejected, and we look into seeing if there's interest in changing the official [doc] extra recommendation to [docs]?

Footnotes

1. On average. The record I've seen was 25 hours. ↩

[bsipocz]

OK, no strong preference for [dev], but definitely having a true, user facing dependencies into [all], so it should have no testing or documentation ones in there.

[stefanv]

I didn't know what the SPEC was about, and found the term "target" quite confusing; would "Recommended naming conventions" capture the whole scope well enough?

[henryiii]

My recommendation: Avoid mentioning extras for now. I don't think a SPEC should make a recommendation in direct opposition to the official guidelines lightly, so we can open a discussion about [doc] vs. [docs] on discuss.python.org. Hatch will have native support for the one that is in the standard, so there's an incentive to follow the standard; let's see if people are interested in changing the standard to match common usage. I want to rerun my numbers against a more recent copy of PyPI before opening an issue. Others feel free to if you can first.

Long term, having the recommendations on extras is good, but let's not hastily add something that contracts a 7+ year old official guideline. The most important thing it to have a standard, so tools can automatically optimize if you follow it.

I'd replace the word "target" everywhere with "task", as it's referring to task runner's unit of work, which I think "task" conveys in a tool-agnostic way. Nox calls them sessions. And I think "Recommended naming conventions" is a better SPEC name, too.

[mdhaber]

Suggested change

	- Folders containing tests be names `tests`.
	- Folders containing tests be named `tests`.

[mdhaber]

"cursory" means hasty and without attention to detail, but suggests that it wasn't done well. Readers might not think it's appropriate to base a SPEC on a "cursory" survey. "Informal" is also true but may carry less of a negative connotation.

Isn't scientific python used in the lower case sense here?

Suggested change

	From a cursory survey in the Scientific Python ecosystem, we discover some
	frustration from contributors and maintainer when moving from one project to
	another and believe that consistency will make it both easier for existing
	maintainer to contribute to project as well a decrease the confusion of new
	developers when contributing or creating new projects.
	From an informal of scientific Python developers, we discover some
	frustration from contributors and maintainers when moving from one project to
	another. We believe that consistency will make it both easier for existing
	maintainers to contribute to projects as well as decrease the confusion of new
	developers when creating or contributing to projects.

Or consider "creating new projects or contributing to existing ones". (I found it easy to misread "contributing or creating new".)

[mdhaber]

Suggested change

	There seem to be a strong consensus with preference for `docs` in favor of
	`docs`, and a preference for `tests` in favor of `test`.
	There seem to be a strong consensus with preference for `docs` over
	`doc`, and a preference for `tests` over `test`.

"preference for tests" means that "tests" is preferred, but "in favor of test" means that "test" is preferred, so it was not clear to me which was preferred.

[mdhaber]

Suggested change

	For the tie being we will not pronounce ourselves on the optional extra
	`extra` dependency for `pyproject.toml`.
	For the time being, we will not specify a name of the optional extra
	`extra` dependency for `pyproject.toml`.

And I'd suggest moving this to the end. It might be OK to precede this section with one about "Scope", but as-is, it looks like this is just one example of something that was not decided.

[mdhaber]

Suggested change

	- Targets related to documentations be named `docs` (and not `doc`). For example
	- Targets related to documentation be named `docs` (and not `doc`). For example

(It was mentioned during the discussion that we say "docs" even though "documentation" is the full word.)

[mdhaber]

Might mention what the aliases are for. Probably not aliases for folders?