Add PED-ANOVA as the first option for importance evaluator #811
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #811 +/- ##
=======================================
Coverage 69.53% 69.53%
=======================================
Files 35 35
Lines 2360 2367 +7
=======================================
+ Hits 1641 1646 +5
- Misses 719 721 +2 ☔ View full report in Codecov by Sentry. |
c-bata
reviewed
Feb 22, 2024
optuna_dashboard/_importance.py
Outdated
Comment on lines
34
to
36
| except Exception as e: | ||
| _logger.warning(f"Skipping to use PedAnovaImportanceEvaluator due to {e}.") | ||
| PedAnovaImportanceEvaluator = None # type: ignore |
There was a problem hiding this comment.
Suggested change
| except Exception as e: | |
| _logger.warning(f"Skipping to use PedAnovaImportanceEvaluator due to {e}.") | |
| PedAnovaImportanceEvaluator = None # type: ignore |
[nit] I would say these lines can be removed since PedAnovaImportanceEvaluator is a pure Python module.
c-bata
reviewed
Feb 22, 2024
* Apply black * Add type ignore to PedAnova import
nabenabe0928
force-pushed
the
feat/replace-fanova-with-ped-anova
branch
from
51aba76 to
3efee74
Compare
|
@c-bata Thank you for your review, I addressed your suggestion and removed the error in CI. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Contributor License Agreement
This repository (
optuna-dashboard) and Goptuna share common code.This pull request may therefore be ported to Goptuna.
Make sure that you understand the consequences concerning licenses and check the box below if you accept the term before creating this pull request.
Reference Issues/PRs
This PR aims to add PED-ANOVA, a quick f-ANOVA calculation algorithm, to the first choice of the importance calculation of Optuna Dashboard to enhance UX.
For PED-ANOVA, please check the following PR:
What does this implement/fix? Explain your changes.
This PR introduces PED-ANOVA to Optuna Dashboard if the Optuna version in the environment is 3.6 or later.
By using PED-ANOVA, the visualization becomes very quick, so users will not experience very lagged dashboard behavior anymore due to the importance evaluation.
The current Optuna Dashboard would not give anything after several minutes when
n_trialsis huge (e.g.,n_trails > 10000).Meanwhile, if we use PED-ANOVA, Optuna Dashboard gives the importance plot in a few seconds even with
n_trails > 10000.