Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makes DPI configurable via default_config.yaml #527

Merged
merged 3 commits into from
Apr 17, 2023
Merged

Makes DPI configurable via default_config.yaml #527

merged 3 commits into from
Apr 17, 2023

Conversation

ns-rse
Copy link
Collaborator

@ns-rse ns-rse commented Apr 14, 2023

Closes #526

The Images class always had dpi as an option but it was not accessible from run_topostats as no options were present in the configuration file and therefore didn't cascade through. Dots Per Inch (DPI) can now be specified in the configuration file, its default is set to 100.0 (the default used by savefig in Matplotlib). Validation is performed and a test introduced that sets dpi = 300 for testing to check the value cascades through and plots are made using this value (this also required setting the parameter for the pytest-mpl fixture too, otherwise it would save the figure with the default value).

Closes #526

The `Images` class always had `dpi` as an option but it was not accessible from `run_topostats` as no options were
present in the configuration file and therefore didn't cascade through. Dots Per Inch (DPI) can now be specified in the
configuration file, its default is set to `100.0` (the default used by `savefig` in Matplotlib). Validation is performed
and a test introduced that sets `dpi = 300` for testing to check the value cascades through and plots are made using
this value (this also required setting the parameter for the `pytest-mpl` fixture too, otherwise it would save the
figure with the default value).
@ns-rse ns-rse added the Images Issues pertaining to the Images class label Apr 14, 2023
@codecov-commenter
Copy link

codecov-commenter commented Apr 14, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.04 ⚠️

Comparison is base (703f2ef) 80.08% compared to head (27be2fb) 80.05%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #527      +/-   ##
==========================================
- Coverage   80.08%   80.05%   -0.04%     
==========================================
  Files          19       19              
  Lines        2792     2792              
==========================================
- Hits         2236     2235       -1     
- Misses        556      557       +1     
Impacted Files Coverage Δ
topostats/validation.py 100.00% <ø> (ø)
topostats/run_topostats.py 84.67% <100.00%> (ø)
topostats/utils.py 97.36% <100.00%> (+0.10%) ⬆️

... and 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Collaborator

@SylviaWhittle SylviaWhittle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor typo.

Edit: I was mistaken, all good

Just to check...

I tried running topostats with dpi=1 and it produced exactly the same image as dpi=1000. Is this normal?

@@ -262,6 +262,9 @@ def validate_config(config: dict, schema: Schema, config_type: str) -> None:
),
),
"histogram_bins": lambda n: n > 0,
"dpi": Or(
lambda n: n > 0, "figure", error="Invalid valud in config for 'dpi', valid values are 'figure' or > 0."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
lambda n: n > 0, "figure", error="Invalid valud in config for 'dpi', valid values are 'figure' or > 0."
lambda n: n > 0, "figure", error="Invalid value in config for 'dpi', valid values are 'figure' or > 0."

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You weren't mistaken, I corrected it.

In adding `dpi` as an option I forgot to update the individual plotting dictionaries with this new option. To avoid such
errors in the future I've changed how these dictionaries are updated to expand all options (bar `plot_dict` which would
be recursive and `run` which is not required when plotting individual images).
@ns-rse
Copy link
Collaborator Author

ns-rse commented Apr 17, 2023

Just to check...

I tried running topostats with dpi=1 and it produced exactly the same image as dpi=1000. Is this normal?

That isn't what I'd expect, and sorry for not checking.

I've investigated and realised that I'd forgotten how we pass options to each image (via the plot_dict key/value of plotting_config). In doing so we update each of these nested dictionaries with the plotting options and I hadn't added dpi to this list.

I've therefore modified how this is done in the update_plotting_config() function so that in the future new options that are added won't require explicit inclusion, they will be passed through automatically thanks to **kwargs expansion (see 27be2fbb for the specific change if curious).

Thanks for catching this, a glaring omission on my behalf 😊 but hopefully an improved solution.

@ns-rse ns-rse requested a review from SylviaWhittle April 17, 2023 11:02
Copy link
Collaborator

@SylviaWhittle SylviaWhittle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to work for me. Thanks for smoothing out and future-proofing the config update function 😄

@MaxGamill-Sheffield
Copy link
Collaborator

I just wanted to clarify behaviour for this. So the dpi was originally put into the images class in preparation for DNAtracing to produce visible single pixel traces with the intention to set these at the plottingdict level.

So my question is if this is now a config parameter, have we given any thought to handling different images separately? On my cats branch where I've implemented this it takes a while to produce a single plot at 1000 dpi (to show representitive skeletons / traces although I've not seen how low of a dpi I could go), so if all 20 plots are set globally like this are we concerned with runtime?

@SylviaWhittle
Copy link
Collaborator

I just wanted to clarify behaviour for this. So the dpi was originally put into the images class in preparation for DNAtracing to produce visible single pixel traces with the intention to set these at the plottingdict level.

So my question is if this is now a config parameter, have we given any thought to handling different images separately? On my cats branch where I've implemented this it takes a while to produce a single plot at 1000 dpi (to show representitive skeletons / traces although I've not seen how low of a dpi I could go), so if all 20 plots are set globally like this are we concerned with runtime?

@MaxGamill-Sheffield Good point. I suppose that this is the base functionality, and in a new issue / PR, we could add the ability to only let it affect the two main output plots?

@ns-rse
Copy link
Collaborator Author

ns-rse commented Apr 17, 2023

I just wanted to clarify behaviour for this. So the dpi was originally put into the images class in preparation for DNAtracing to produce visible single pixel traces with the intention to set these at the plottingdict level.

So my question is if this is now a config parameter, have we given any thought to handling different images separately? On my cats branch where I've implemented this it takes a while to produce a single plot at 1000 dpi (to show representitive skeletons / traces although I've not seen how low of a dpi I could go), so if all 20 plots are set globally like this are we concerned with runtime?

This is something of a can of 🪱 depending on your point of view.

In my, opinionated, view plotting should be done in Notebooks. There is too much variation and tweaks that might be required to get everything perfect across a potential bunch of different scans. The iterative nature of developing the "optimal" image is therefore best done interactively on a per scan basis for the image that is desired and Notebooks are most suited to that, rather than run_topostats > Edit config file to change paramaters > run_topostats > Edit config file to change parameters ad nauseum

On the flip side batch processing should perhaps not be too much of a worry, you set it running and go and do something else in the mean time. However this cyclical approach can be counter productive as it promotes context switching which can be counter productive.

A middle ground might be to set DPI in the topostats/plotting_dictionary.yaml to be specific to the plot being generated. But then that would make it considerably more complicated for users to modify and run (although the pain could be eased to some extent by adding more options to --create-plotting-dictionary and --plotting-dictionary so that users can specify their own plotting dictionary).

However, I've already been thinking about this in a related vein because as the number of options starts to grow having additional flags or individual commands can be handled in a better manner. The "swiss army" approach used by commands such as git [command] <options> and pre-commit [command] <options> would be better suited as its more modular and easier to extend/maintain code (see #517).

One of the over-riding aspects I took on board early on was the desire to not have configuration files too complicated for users as its over-whelming and puts people off. However, if the desire is to have highly customised/configurable options that is a necessary evil.

I'm a big fan of teaching people how to achieve what they want and its why I advocate for the Notebook approach to plotting and have spent time developing Notebooks that show how to do some (but not yet all) aspects of this.

@ns-rse ns-rse added this pull request to the merge queue Apr 17, 2023
Merged via the queue into main with commit bd06b70 Apr 17, 2023
@ns-rse ns-rse deleted the ns-rse/526-dpi branch May 9, 2023 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Images Issues pertaining to the Images class
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make DPI configurable
4 participants