kumulant

com.eignex.kumulant/stat/regression/AdagradOptimizer

AdagradOptimizer

class AdagradOptimizer(val featureSize: Int, val learningRate: ScalarExpr, val epsilon: Double = 1.0E-10, concurrency: Concurrency = Concurrency.None) : Optimizer(source)

Adagrad: accumulates squared gradients per coordinate; the effective per-coord learning rate is lr / sqrt(sumG2[i] + epsilon). Adapts faster on rare features.

Constructors

AdagradOptimizer

constructor(featureSize: Int, learningRate: ScalarExpr, epsilon: Double = 1.0E-10, concurrency: Concurrency = Concurrency.None)(source)

Properties

epsilon

val epsilon: Double(source)

Numerical-stability epsilon added under the square root.

featureSize

open override val featureSize: Int(source)

Number of weight coordinates this optimizer manages.

learningRate

val learningRate: ScalarExpr(source)

Base learning-rate schedule.

Functions

advance

open override fun advance()(source)

Advance per-update counters (Adam step t, etc.). Called once per stat update.

computeDelta

open override fun computeDelta(coordIndex: Int, gradient: Double, observationWeight: Double): Double(source)

Per-coordinate update. Reads/writes any auxiliary state owned by the optimizer, then returns the delta the caller should add to w[coordIndex].

reset

open override fun reset()(source)

Reset all internal state to its initial values.