get.optimal.armor.combos()
ranks armor combinations using a single number, SCORE (and
its human-readable form, SCORE_QUALITY). This vignette
explains where that number comes from.
get.armor.tradeoffs() uses the same score: it’s the stat
maximized with metric = "SCORE", and the tie-break when
maximizing any other stat.
An armor combination is described by ten metrics: physical, strike,
slash, and thrust defense; magic, fire, and lightning defense; and
bleed, poison, and curse resistance. They vary between combinations by
very different amounts: across every combination, slash defense has a
standard deviation of about 33, and bleed resistance about 13. Added up
raw with equal weights, a point of slash would count as much as a point
of bleed, so slash’s wider spread would decide far more of which
combination comes out on top, not because the weights
argument said so. (A stat’s typical size doesn’t matter on its own:
adding the same amount to every combination’s total changes no ranking.
What matters is how far apart combinations are.)
The fix is standardization: express each metric in units of “how
unusual is this value”, not its raw number.
get.optimal.armor.combos() does this using the population
mean and standard deviation of each metric, computed once across every
possible armor combination (including every upgrade level) when the
package’s data was built - see create_rda/create_rda.R.
These live in the package’s internal data as means,
stddevs, and corrs (the population correlation
matrix across the ten metrics). Because the four slots are chosen
independently, these follow exactly from each slot’s own means and
covariances (the combination’s mean is the sum of the slots’ means, and
likewise its covariance matrix), so no combination ever needs to be
enumerated.
Given a weights vector (one weight per metric, after the
function normalizes it to sum to 1), the package computes a set of
per-metric scalars:
For a combination with combined metric values \(x_1, \dots, x_{10}\), the score is:
\[ \text{SCORE} = \sum_{i=1}^{10} \text{score.scalars}_i \cdot (x_i - \mu_i) \]
where \(\mu_i\) is that metric’s
population mean (means[i]). This is a weighted, centered
sum - but the particular denominator in
score.scalars isn’t arbitrary. It’s chosen so that
SCORE always has variance exactly 1, no matter what weights
are passed in.
To see why: let \(\Sigma\) be the
population covariance matrix implied by stddevs and
corrs (so \(\Sigma_{ij} =
\sigma_i \sigma_j \cdot \text{corr}_{ij}\)), and let \(k = \sqrt{w^\top \text{corr} \, w}\) be the
normalizing term inside score.scalars. Then
score.scalars \(= w / (\sigma
k)\) element-wise, and the variance of SCORE works out to:
\[ \text{Var}(\text{SCORE}) = \text{score.scalars}^\top \Sigma \, \text{score.scalars} = \frac{1}{k^2} w^\top \text{corr} \, w = \frac{1}{k^2} \cdot k^2 = 1 \]
This puts every choice of weights on the same scale: a
SCORE of 1.5 always means “one and a half standard
deviations above the average combination, as judged by these weights”,
whichever weights were chosen. It doesn’t make scores under
different weights interchangeable, though - see “What this does
and doesn’t let you compare” below.
tests/testthat/test-scoring.R checks this algebraic
identity directly, for a battery of random weight vectors.
The standard deviation is one choice of unit for each stat.
stat.scale offers two others:
"max": each stat in units of its highest value across
every armor combination (stat.maxes, the best set total at
full upgrade). A weight then covers a stat’s whole range, from nothing
to the best any set can reach."raw": no adjustment, so one point of any stat counts
the same. With equal weights, stats that vary more between sets count
for more: slash defense (standard deviation 33) counts about two and a
half times as much as bleed resistance (13).Whatever the scale, the weights become per-point multipliers \(c_i = w_i / u_i\), where \(u_i\) is the stat’s unit (its standard
deviation, its highest value, or 1). The score is then scaled exactly as
above: its variance is \(c^\top \Sigma
c\), so score.scalars \(=
c / \sqrt{c^\top \Sigma c}\). Under "sd" this is the
formula above. So SCORE means “standard deviations above
the average combination” under every scale. The scale changes how the
weights trade stats off against each other, not what
SCORE’s numbers mean. Equivalently, weights \(w\) under "raw" give exactly
the same scores as \(w \cdot \sigma\)
under "sd", and \(w\)
under "max" the same as \(w\) divided by each stat’s highest value
under "raw".
The ten stats fall into three groups whose stats move together: almost perfectly for the four physical defenses (average correlation 0.98), partly for the elemental defenses (0.54), and less for the resistances (0.31). So with equal weights on standardized stats, the physical defenses, nearly one stat counted four times, drive about half of what separates scores:
| Group | Share with equal stat weights | Share with equal group weights |
|---|---|---|
| Physical defenses | 48% | 30% |
| Elemental defenses | 32% | 39% |
| Resistances | 20% | 31% |
weights.mode = "grouped" takes one weight per group
instead (PHYSICAL_DEFENSES,
ELEMENTAL_DEFENSES, RESISTANCES). Each group
counts as the average of its stats, and stat.scale applies
to that average:
"sd", its own standard deviation, which follows
from the stats’ standard deviations and correlations;"max", its highest value across every combination
(group.maxes);"raw", one point.Equal group weights then give each group close to a third, as in the right-hand column above (on the Std Dev scale).
A grouped score is still a weighted sum of the ten stats: a group’s
weight \(W\) becomes \(W / (k \cdot u)\) per point on each of its
\(k\) stats, where \(u\) is the group average’s unit. So the
search, SCORE_QUALITY and the trade-off curves all work
exactly as before.
The four armor slots (head, chest, hands, legs) are scored
independently and then added together -
optimal_armor_combinations() sums
curr_head_OBJECTIVE + curr_chest_OBJECTIVE + curr_hands_OBJECTIVE + curr_legs_OBJECTIVE
(each piece’s objective, which is its score when ranking by score); see
R/optimal-combos.R, where each slot’s contribution is
computed as
score.scalars[i] * (metric_i - 0.25 * means[i]).
means[i] is the population mean of the combined
metric across a full four-piece combination, not any one slot’s mean.
Subtracting 0.25 * means[i] in each of the four slots, and
then summing all four, subtracts exactly means[i] from the
total - which is what actually needs to happen for the total score to be
centered at 0. The even four-way split is simply the easiest way to
divide that single subtraction across four separately-computed slot
scores; it doesn’t require (and isn’t affected by) whether any
individual slot’s own average metric value happens to be exactly a
quarter of the combo-level mean.
SCORE is a number of standard deviations, which isn’t
very intuitive on its own. SCORE_QUALITY turns it into a
rarity: how many of all the possible combinations at the selected
upgrade levels score at least as well. A combination that 1 in every 40
combinations matches or beats reads "Top 1 in 40"; the
single best combination at a level reads
"Top 1 in 11,930,328" - every combination there is.
Combinations in the bottom half read "Bottom 1 in N"
instead, counting the combinations that score at most as well.
Ties count toward the number in both cases.
The comparison is against every combination at the selected
regular.level/twinkling.level, regardless of
the other filters (areas, starting class, equip load, minima, and so
on), so it describes how good a result is among everything a player at
that upgrade level could wear.
There are about 11.9 million combinations at each upgrade level, but counting the ones that score at least as well as a given result doesn’t require visiting them. A combination’s score is just the sum of its four pieces’ scores (see the previous section), so the count is a question about sums of numbers drawn from four short lists:
## every head+chest+legs sum (~221 thousand values), computed once and sorted
other.sums <- sort(as.vector(outer(outer(head.scores, chest.scores, `+`), legs.scores, `+`)))
## for each hands piece, a binary search counts the sums that complete a combination at or past
## the target score
at.least <- 0
for(hands.score in hands.scores){
at.least <- at.least + sum(other.sums >= target - hands.score)
}(the package itself uses findInterval() for the binary
search, handles every result in one pass, and allows a tiny
floating-point tolerance - see R/score-quality.R). Looping
over hands, the slot with the fewest pieces, keeps the per-result cost
to about 54 binary searches; the sorted list of the other three slots’
sums is built once per call.
An earlier version of SCORE_QUALITY assumed
SCORE was normally distributed across every combination at
every upgrade level pooled together, and converted it with
pnorm(). That turned out to be off by large factors in both
directions. The score is a sum of only four independent terms (one per
slot), not of many, and its tails aren’t normal: the README example’s
top result read “1 in 318” against an exact “1 in 193”, the best
combination at +10/+5 read “1 in 702 million” when only about 11.9
million combinations exist at that level, and single-stat weights
produced numbers like “1 in 3 x 10^27”. Pooling every upgrade level also
made upgraded builds look far rarer than they are among the combinations
a player at that level can actually choose from.
Because Var(SCORE) = 1 holds for any weights,
SCORE is directly comparable across different filter and
constraint choices, including different upgrade levels, as long as
the weights, stat.scale and weights.mode are
held fixed. SCORE_QUALITY is comparable across filter
and constraint choices at the same upgrade levels -
"Top 1 in 1,000" under one set of filters really is better
than "Top 1 in 20" under another - but is always relative
to the selected upgrade levels. Scores computed with different
weights are not directly comparable to each other in the same way, since
each weight vector defines its own notion of “average” combination.