kumulant/com.eignex.kumulant.bandit.univariate/UcbV

UcbV

class UcbV(val zeta: Double = 1.2, val c: Double = 1.0, priorMean: Double = 0.0, priorWeight: Double = 0.02) : BanditPolicy<MomentsResult> (source)

UCB-V; variance-aware UCB with finite-sample honesty (Audibert, Munos, Szepesvári 2009). Score is mean + sqrt(2 * V * zeta * ln(t) / n) + 3 * c * zeta * ln(t) / n, where V is the running variance from the MomentsResult snapshot.

The third term; a bias correction scaled by c; is what distinguishes UCB-V from UCB1Tuned: the bound is honest at finite sample sizes rather than only asymptotically. Reach for it when sample sizes per arm stay small (early stopping, expensive arms) and the asymptotic tightness of UCB1Tuned / KlUcb doesn't materialise.

Audibert et al. recommend zeta in [1, 1.2]; defaults are at the upper end of that range.

Constructors

UcbV

constructor(zeta: Double = 1.2, c: Double = 1.0, priorMean: Double = 0.0, priorWeight: Double = 0.02)

Properties

arm

open override val arm: MomentsArm

Per-arm cumulator spec; determines the prior pseudo-counts, value encoding, and result shape that evaluate consumes.

val c: Double

Bias-correction term scale. Default matches the original paper.

zeta

val zeta: Double

Variance-term scale. Audibert et al. recommend zeta in [1, 1.2].

Functions

addArm

open override fun addArm(snapshot: MomentsResult)

Hook called when a new arm joins the population. Lets stateful policies fold the new arm's snapshot into their global counters (UCB's total-samples, UCB1Normal's arm count). Default no-op.

createArm

open fun createArm(): SeriesStat<MomentsResult>

Allocate a fresh per-arm accumulator from the arm spec. Default delegates to arm.createStat(); override only if the policy needs a non-standard variant.

evaluate

open override fun evaluate(snapshot: MomentsResult, step: Long, rng: Random): Double

Score an arm given its current snapshot. Higher scores are preferred by the bandit. step is the global update count (for time-dependent exploration schedules); rng is the bandit's shared com.eignex.kumulant.bandit.Bandit.random (consumed by sampling policies).

removeArm

open override fun removeArm(snapshot: MomentsResult)

Hook called when an arm leaves the population. Inverse of addArm; lets stateful policies remove the departing arm's contribution from their global counters. Default no-op.

update

open override fun update(stat: SeriesStat<MomentsResult>, value: Double, weight: Double = 1.0)

Fold an observed reward value (with optional weight) into the per-arm stat. Default applies arm.encode first; policies with global counters (UCB families) override to update their counter alongside the stat update.