kumulant
kumulant/com.eignex.kumulant.stat.regression/AdamOptimizer

AdamOptimizer

class AdamOptimizer(val featureSize: Int, val learningRate: ScalarExpr, val beta1: Double = 0.9, val beta2: Double = 0.999, val epsilon: Double = 1.0E-8, concurrency: Concurrency = Concurrency.None) : Optimizer(source)

Adam with bias-corrected first and second moments. Default hyperparameters beta1=0.9, beta2=0.999, epsilon=1e-8 follow Kingma & Ba 2015.

Constructors

Link copied to clipboard
constructor(featureSize: Int, learningRate: ScalarExpr, beta1: Double = 0.9, beta2: Double = 0.999, epsilon: Double = 1.0E-8, concurrency: Concurrency = Concurrency.None)

Properties

Link copied to clipboard
val beta1: Double

First-moment EMA decay.

Link copied to clipboard
val beta2: Double

Second-moment EMA decay.

Link copied to clipboard
val epsilon: Double

Numerical-stability epsilon.

Link copied to clipboard
open override val featureSize: Int

Number of weight coordinates this optimizer manages.

Link copied to clipboard
val learningRate: ScalarExpr

Base learning-rate schedule.

Functions

Link copied to clipboard
open override fun advance()

Advance per-update counters (Adam step t, etc.). Called once per stat update.

Link copied to clipboard
open override fun computeDelta(coordIndex: Int, gradient: Double, observationWeight: Double): Double

Per-coordinate update. Reads/writes any auxiliary state owned by the optimizer, then returns the delta the caller should add to w[coordIndex].

Link copied to clipboard
open override fun reset()

Reset all internal state to its initial values.

AdamOptimizer

constructor(featureSize: Int, learningRate: ScalarExpr, beta1: Double = 0.9, beta2: Double = 0.999, epsilon: Double = 1.0E-8, concurrency: Concurrency = Concurrency.None)(source)

advance

open override fun advance()(source)

Advance per-update counters (Adam step t, etc.). Called once per stat update.

beta1

val beta1: Double(source)

First-moment EMA decay.

beta2

val beta2: Double(source)

Second-moment EMA decay.

computeDelta

open override fun computeDelta(coordIndex: Int, gradient: Double, observationWeight: Double): Double(source)

Per-coordinate update. Reads/writes any auxiliary state owned by the optimizer, then returns the delta the caller should add to w[coordIndex].

epsilon

val epsilon: Double(source)

Numerical-stability epsilon.

featureSize

open override val featureSize: Int(source)

Number of weight coordinates this optimizer manages.

learningRate

val learningRate: ScalarExpr(source)

Base learning-rate schedule.

reset

open override fun reset()(source)

Reset all internal state to its initial values.