RmspropOptimizer

class RmspropOptimizer(val featureSize: Int, val learningRate: ScalarExpr, val rho: Double = 0.9, val epsilon: Double = 1.0E-8, concurrency: Concurrency = Concurrency.None) : Optimizer(source)

RMSProp: exponential moving average of squared gradients with decay rho; effective per-coord learning rate is lr / sqrt(emaG2[i] + epsilon).

Constructors

constructor(featureSize: Int, learningRate: ScalarExpr, rho: Double = 0.9, epsilon: Double = 1.0E-8, concurrency: Concurrency = Concurrency.None)

Numerical-stability epsilon.

open override val featureSize: Int

Number of weight coordinates this optimizer manages.

Base learning-rate schedule.

EMA decay for the squared gradient.

open override fun advance()

Advance per-update counters (Adam step t, etc.). Called once per stat update.

open override fun computeDelta(coordIndex: Int, gradient: Double, observationWeight: Double): Double

Per-coordinate update. Reads/writes any auxiliary state owned by the optimizer, then returns the delta the caller should add to w[coordIndex].

open override fun reset()

Reset all internal state to its initial values.

constructor(featureSize: Int, learningRate: ScalarExpr, rho: Double = 0.9, epsilon: Double = 1.0E-8, concurrency: Concurrency = Concurrency.None)(source)