com.eignex.kumulant/bandit/univariate/Boltzmann

Boltzmann

class Boltzmann(val initialTau: Double = 1.0, val minTau: Double = 0.001, val decay: Double = 1.0, priorMean: Double = 0.0, priorWeight: Double = 0.02) : BanditPolicy<WeightedMeanResult> (source)

Boltzmann exploration (softmax bandit): plays arm a with probability proportional to exp(mean(a) / tau(t)), where the temperature cools as tau(t) = max(minTau, initialTau / t^decay) over the 1-based round t; decay = 1 is Cesa-Bianchi & Fischer's classical recipe and decay = 0 a fixed temperature. High temperature flattens play toward uniform exploration; low temperature sharpens it toward greedy exploitation.

The softmax draw is the argmax of mean / tau plus independent standard Gumbel noise per arm, which samples the softmax distribution exactly, so the policy scores arms independently inside MultiArmedBandit. A non-finite mean scores as logit 0, so a corrupt arm stays playable without dominating.

Use cases: stationary scalar-reward problems where smooth probabilistic exploration is preferable to UCB's deterministic confidence bounds; any setting where a tunable cooling schedule is convenient.

Randomness: one rng.nextDouble() per arm per evaluation.

Constructors

Boltzmann

constructor(initialTau: Double = 1.0, minTau: Double = 0.001, decay: Double = 1.0, priorMean: Double = 0.0, priorWeight: Double = 0.02)(source)

Properties

arm

open override val arm: MeanArm(source)

Per-arm cumulator spec; determines the prior pseudo-counts, value encoding, and result shape that evaluate consumes.

decay

val decay: Double(source)

Cooling decay exponent: tau(t) = initialTau / t^decay. 0.0 is fixed-temperature.

initialTau

val initialTau: Double(source)

Initial temperature; the schedule cools from here.

minTau

val minTau: Double(source)

Floor on the temperature so the softmax never collapses to a delta.

Functions

evaluate

open override fun evaluate(snapshot: WeightedMeanResult, step: Long, rng: Random): Double(source)

Score an arm given its current snapshot. Higher scores are preferred by the bandit. step is the global update count (for time-dependent exploration schedules); rng is the bandit's shared com.eignex.kumulant.bandit.Bandit.random (consumed by sampling policies).

addArm

open fun addArm(snapshot: WeightedMeanResult)

Hook called when a new arm joins the population. Lets stateful policies fold the new arm's snapshot into their global counters (UCB's total-samples). Default no-op.

createArm

open fun createArm(concurrency: Concurrency = Concurrency.None): SeriesStat<WeightedMeanResult>

Allocate a fresh per-arm accumulator from the arm spec with the given concurrency. Default delegates to arm.createStat(concurrency); override only if the policy needs a non-standard variant.

createPolicy

open fun createPolicy(): BanditPolicy<WeightedMeanResult>

Fresh policy carrying the same configuration but no accumulated global state. A replica bandit must not share counters with the bandit it was created from, so any policy holding mutable state has to override this; the default returns this because a stateless policy is safe to share.

removeArm

open fun removeArm(snapshot: WeightedMeanResult)

Hook called when an arm leaves the population. Inverse of addArm; lets stateful policies remove the departing arm's contribution from their global counters. Default no-op.

update

open fun update(stat: SeriesStat<WeightedMeanResult>, value: Double, weight: Double = 1.0)

Fold an observed reward value (with optional weight) into the per-arm stat. Default applies arm.encode first; policies with global counters (UCB families) override to update their counter alongside the stat update.