kumulant

EpsilonGreedy

class EpsilonGreedy(val epsilon: Double = 0.1, val decay: Double = 0.0, priorMean: Double = 0.0, priorWeight: Double = 0.02) : SampleCountingPolicy<WeightedVarianceResult> (source)

Epsilon-greedy: with probability eps pick a uniformly random arm (explore), otherwise pick the arm with the highest mean (exploit). The simplest exploration scheme that actually works; no math machinery.

The exploration probability is eps(n) = min(1, epsilon / n^decay) over the total sample count n. At the default decay = 0 it is the fixed epsilon; typical good values are 0.05-0.2. A positive decay anneals exploration toward zero as samples accumulate, exploring aggressively early and converging to greedy once the arms are well separated; decay = 0.5 with epsilon around 2.0 gives Auer et al.'s sqrt(T) regret bound.

Constructors

EpsilonGreedy

constructor(epsilon: Double = 0.1, decay: Double = 0.0, priorMean: Double = 0.0, priorWeight: Double = 0.02)(source)

Properties

arm

open override val arm: NormalArm(source)

Per-arm cumulator spec; determines the prior pseudo-counts, value encoding, and result shape that evaluate consumes.

decay

Decay exponent applied to the total sample count; 0.0 keeps exploration fixed.

epsilon

Exploration probability at decay = 0; otherwise the scale of the annealed probability.

Functions

createPolicy

open override fun createPolicy(): EpsilonGreedy(source)

Fresh policy carrying the same configuration but no accumulated global state. A replica bandit must not share counters with the bandit it was created from, so any policy holding mutable state has to override this; the default returns this because a stateless policy is safe to share.

evaluate

open override fun evaluate(snapshot: WeightedVarianceResult, step: Long, rng: Random): Double(source)

Score an arm given its current snapshot. Higher scores are preferred by the bandit. step is the global update count (for time-dependent exploration schedules); rng is the bandit's shared com.eignex.kumulant.bandit.Bandit.random (consumed by sampling policies).

Link copied to clipboard
open override fun addArm(snapshot: WeightedVarianceResult)

Hook called when a new arm joins the population. Lets stateful policies fold the new arm's snapshot into their global counters (UCB's total-samples). Default no-op.

Link copied to clipboard
open fun createArm(concurrency: Concurrency = Concurrency.None): SeriesStat<WeightedVarianceResult>

Allocate a fresh per-arm accumulator from the arm spec with the given concurrency. Default delegates to arm.createStat(concurrency); override only if the policy needs a non-standard variant.

Link copied to clipboard
open override fun removeArm(snapshot: WeightedVarianceResult)

Hook called when an arm leaves the population. Inverse of addArm; lets stateful policies remove the departing arm's contribution from their global counters. Default no-op.

Link copied to clipboard
open override fun update(stat: SeriesStat<WeightedVarianceResult>, value: Double, weight: Double = 1.0)

Fold an observed reward value (with optional weight) into the per-arm stat. Default applies arm.encode first; policies with global counters (UCB families) override to update their counter alongside the stat update.