Added a neural network architecture that allows the model to learn long-term dependencies

[honkkis]

For long-term dependencies and for low-alias 96kHz training it would be great to have an option with slightly larger network and larger receptive field.

Therefore this pull request adds an architecture with "dilations": [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024], instead of "dilations": [1, 2, 4, 8, 16, 32, 64, 128, 256, 512], as in Architecture.standard. This does not seem to create huge CPU impact at inference time.

This is useful for 96kHz training to get the same effective receptive field in real time compared to the standard architecture in 48kHz. Could be also useful for modeling long-term effects in 48kHz.

This option is automatically reflected by the GUI, while the default is still standard.

[honkkis]

Quite big effect in 96kHz training of a high gain amp. Both are trained 600 epochs with the same settings except the better ESR one is using the 1024 dilation defined in the HD model in this pull request!

[sdatkinson]

Thanks for the PR.

This is a cool idea, but I'd like to see more evidence of its usefulness before merging it in as one of the pre-configured architectures.

It's important to me to keep the list of models that users pick from as small as possible and that they all have a solid value proposition. I'd be happy to keep an eye out on this if you can demonstrate that it does something cool, but the bar for introducing it into the standardized trainer is quite high since this also would open up a second dimension for users to make a choice on (where the only one right now is CPU usage vs accuracy).