kumulant/com.eignex.kumulant.bandit.univariate/ThompsonSampling

ThompsonSampling

class ThompsonSampling<R : Result>(val arm: Arm<R>, val posterior: Posterior<R>) : BanditPolicy<R> (source)

Thompson sampling: score each arm by a draw from its conjugate posterior given the snapshot. The bandit then picks the arm with the highest sample; no explicit exploration knob, the exploration falls out of posterior variance shrinking as data accumulates.

Pair an Arm with a Posterior of the same result type R:

BernoulliArm + BetaPosterior: see BetaBernoulliTS.
NormalArm + NormalGammaPosterior: see NormalTS.
LogNormalArm + LogNormalGammaPosterior: see LogNormalTS.
MeanArm + PoissonGammaPosterior / GeometricBetaPosterior / ExponentialGammaPosterior / GammaScalePosterior; see PoissonTS, GeometricTS, ExponentialTS, GammaScaleTS.

Stateless across arms; addArm / removeArm are no-ops because no global counter is involved.

Constructors

ThompsonSampling

constructor(arm: Arm<R>, posterior: Posterior<R>)

Properties

arm

open override val arm: Arm<R>

Per-arm cumulator spec; determines the prior pseudo-counts, value encoding, and result shape that evaluate consumes.

posterior

val posterior: Posterior<R>

Stateless sampler used to draw a score from each arm's snapshot.

Functions

addArm

open fun addArm(snapshot: R)

Hook called when a new arm joins the population. Lets stateful policies fold the new arm's snapshot into their global counters (UCB's total-samples, UCB1Normal's arm count). Default no-op.

createArm

open fun createArm(): SeriesStat<R>

Allocate a fresh per-arm accumulator from the arm spec. Default delegates to arm.createStat(); override only if the policy needs a non-standard variant.

evaluate

open override fun evaluate(snapshot: R, step: Long, rng: Random): Double

Score an arm given its current snapshot. Higher scores are preferred by the bandit. step is the global update count (for time-dependent exploration schedules); rng is the bandit's shared com.eignex.kumulant.bandit.Bandit.random (consumed by sampling policies).

removeArm

open fun removeArm(snapshot: R)

Hook called when an arm leaves the population. Inverse of addArm; lets stateful policies remove the departing arm's contribution from their global counters. Default no-op.

update

open fun update(stat: SeriesStat<R>, value: Double, weight: Double = 1.0)

Fold an observed reward value (with optional weight) into the per-arm stat. Default applies arm.encode first; policies with global counters (UCB families) override to update their counter alongside the stat update.