Skip to content

v0.7.2 - All your batch are belong to us - Micro Update

Compare
Choose a tag to compare
@GilesStrong GilesStrong released this 11 Mar 13:50
· 176 commits to master since this release

v0.7.2 - All your batch are belong to us - Micro Update

Important changes

  • Fixed bug in Model.set_mom which resulted in momentum never being set (affects e.g. OneCycle and CyclicalMom)
  • Model.fit now shuffles the fold indices for training folds prior to each epoch rather than once per training; removes the periodicity in training loss which was occasionally apparent.
  • Bugs found in OneCycle:
    • When training multiple models, the initial LR for subsequent models was the end LR of the previous model (list in partial was being mutated)
    • The model did not stop training at end of cycle
    • Momentum was never altered in the optimiser

Breaking

Additions

  • Mish activation function
  • Model.fit_params.val_requires_grad to control whether to compute validation epoch with gradient, default zero, built some losses might require it in the future
  • ParameterisedPrediction now stores copies of values for parametrised features in case they change, or need to be changed locally during prediction.
  • freeze_layers and unfreeze_layers methods for Model
  • PivotTraining callback implementing Learning to Pivot Louppe, Kagan, & Cranmer, 2016
    • New example reimplementing paper's jets example
  • TargReplace callback for replacing target data in BatchYielder during training
  • Support for loss functions being fastcore partialler objects
  • train_models now has arguments to:
    • Exclude specific fold indices from training and validation
    • Train models on unique folds, e.g. when training 5 models on a file with 10 folds, each model would be trained on their own unique pair of folds
  • Added discussion of core concepts in LUMIN to the docs

Removals

Fixes

  • Cases in which a NaN in the metric during training could spoil plotting and SaveBest
  • Bug in Model.set_mom which resulted in momentum never being set (affects e.g. OneCycle and CyclicalMom)
  • Bug in MetricLogger.get_results where tracking metrics could be spoilt by NaN values
  • Bug in train when not passing any metrics
  • Bug in FoldYielder when loading output pipe from Path
  • Bugs found in OneCycle:
    • When training multiple models, the initial LR for subsequent models was the end LR of the previous model (list in partial was being mutated)
    • The model did not stop training at end of cycle
    • Momentum was never altered in the optimiser

Changes

  • Model.fit now shuffles the fold indices for training folds prior to each epoch rather than once per training; removes the periodicity in training loss which was occasionally apparent.
  • Validation and prediction forwards passes now performed without gradient tracking to save memory and time
  • MetricLogger now records loss values on batch end rather than on forwards end
  • on_batch_end now always called regardless of model state

Depreciations

Comments