Skip to content

Direct dynamic setting or change of ccov #349

@nikohansen

Description

@nikohansen

Similar to #346, we want to be able to set or change "$c_\text{cov}$".

Review possible ways:

  • make option 'CMA_on' (and 'CMA_rankone' and 'CMA_rankmu') versatile to be read in dynamically. They are already used to (re-)set the learning rate before the update.
  • call weights.finalize after self.sm.parameters(), i.e. after resetting the learning rate, (possibly conditioned to any of the multipliers have change)
  • The "correct" way to change $c_\text{cov}$, say by $\alpha$, is by re-computing $c_1, c_\mu, c_c$ from $\alpha\mu_w$ where $\alpha < 1 / \mu_w$ would need to be checked carefully?
  • ...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions