Spyda TrustScore Methodology
A reproducible evidence and TrustScore layer above existing AppSec tools. Mathematically rigorous, deterministic by construction, and operationally conditional. The v2.4 scoring register is frozen; v2.5–v2.6.4 hardened the determinism contract, reproducibility guarantee, and control interactions.
Scoring-engine version 0.2.0 • Numerics profile np-x86-64-glibc.s2 (schema 2) • Scoring constants unchanged from the v2.4 register.
Three Methodological Commitments
Bayesian evidence fusion with calibrated per-tool likelihood ratios, not arbitrary severity multipliers.
Confidence (probability of truth) vs Priority (consequential weight) are distinct, inspectable quantities.
Empirical validation protocol published before results, with reproducibility artefacts.
§10.3 Confidence Calculation
logit(C_f) = logit(π_d) + Σ_k L_k
l_i = log(LR_i) × clarity_i × source_quality_i × direction_i
n_eff,k = n_k / (1 + (n_k − 1) × ρ_k)
support_weight_k = (n_eff,k − 1) / (n_k − 1) [0 if n_k = 1]
L_k = l_primary + support_weight_k × Σ_(i≠primary) a_i × l_i
C_f = logistic(logit(C_f))
Worked example — Confidence by hand (default a_i = 1.0)
One cluster, CodeQL primary + Semgrep secondary, same CWE/location, ρ = 0.45, n = 2.
- logit(π_d=0.08) = −2.4423
- l_primary (CodeQL, LR=8.2, clarity=0.8) = 1.6833
- l_2 (Semgrep, LR=5.0, clarity=0.7) = 1.1266
- n_eff = 2 / 1.45 = 1.3793 → support_weight = 0.3793
- L_k = 1.6833 + 0.3793 × 1.1266 = 2.1106
- logit(C_f) = −0.3317 → C_f = 0.4178
Every term is a published input; the Confidence point value is reproducible by hand. The TrustScore is hand-reproducible only in standard mode (M1 scope, §10.3).
§10.9 Conservative Modes
“Conservative” is meaningless without naming the protected party. The selected mode chooses the Confidence bound C_bound used in Priority, and is printed in every evidence pack.
| Mode | Confidence bound | Use case |
|---|---|---|
| standard | Mean / median posterior | Dashboards, trend reporting (hand-computable) |
| buyer_protection | Upper 95% | Procurement, audit, high-assurance gates (compliance default) |
| release_fairness | Lower 95% | Developer release gating where false blockers are costly |
| dual_report | Both bounds | Combined buyer + vendor reporting |
Interval-dependent modes use a deterministic Monte-Carlo percentile (20k draws), verified by replay under the recorded seed and numerics profile — not by hand.
§10.6 Priority Calculation
C_bound is the Confidence value selected by the active conservative mode (§10.9): mean for standard, upper 95% for buyer_protection, lower 95% for release_fairness.
Severity Normalization
SeverityNorm = CVSS / 10
Continuous, externally validated
Reachability Channel
Orthogonalised reachability enters Priority, not Confidence — a present-but-unimported dependency contributes 0 to Confidence.
No Double-Counting
Priority enters the TrustScore, not Confidence. Each factor is counted exactly once.
§10.7 Domain Score & TrustScore
cumulative_priority_d = Σ P_f (gate-eligible, non-waived findings in d)
λ_d = ln(2) / anchor_d
DomainScore_d_raw = 100 × exp(−λ_d × cumulative_priority_d)
Where anchor_d is the half-score anchor (cumulative priority at which the raw domain score = 50).
| Domain | P*_d | Rationale |
|---|---|---|
| Vulnerabilities | 5.0 | ~5 high-priority real exploits = serious problem |
| Supply Chain | 4.0 | Dependency risk concentrates |
| Compliance | 3.0 | Compliance gaps mostly binary |
| AI Risk | 3.0 | EU AI Act exposure scales rapidly |
Why Geometric Mean?
Arithmetic mean allows one strong domain to mask catastrophic weakness. A project with {vulns: 30, compliance: 95, supply_chain: 95, ai_risk: 95} scores 78.6 under arithmetic mean but only 60.5 under geometric mean, properly surfacing the vulnerability weakness.
ε is a disclosed policy lever, not hygiene (M2)
ε (default 1e-6) prevents a collapsed domain from sending the log-sum to −∞, but it also sets the collapse-penalty magnitude: a single collapse caps an otherwise-perfect TrustScore between ~7 (ε=1e-6) and ~26 (ε=1e-2). It is declared and versioned in every evidence pack.
Collapse governance (M5): when collapsed_domain_count ≥ 1 the policy result is fail via the domain_collapse gate, and the composite is reported as context only — it MUST NOT be used to rank collapsed results. The per-domain raw scores and collapsed_domain_count are always reported alongside.
§16 Determinism & Engine Constants
pre_clip — ESS computed on stabilised, unclipped weights and governs the promotion gate (M8)2 — rank-stability band is the half-width spanned by ±2 order statistics (c4)The scoring engine ships as an immutable, content-addressed OCI image; itsengine_image_digest is recorded in every snapshot and manifest and retained in the evidence vault for durable replay (M6).
Two replay tiers: bit-identity wherever the digest-pinned image runs, and cross-class tolerance (rtol=1e-9, atol=1e-12) everywhere else, on the numerics profile np-x86-64-glibc.s2.
Lifecycle events explain.ready / explain.failed (m13) signal late materialisation or failure so consumers never fall back to polling after scan.completed.
Policy Templates (Appendix D)
Heavy vulnerability (40%) and supply chain (30%) weighting. KEV = instant FAIL (non-waivable). 24-hour threat intel staleness.
AI risk domain weighted at 40%. Model card gates. Training data provenance warnings. AI ethics board approval for waivers.
Zero KEV tolerance. SBOM completeness required. License compliance mandatory. Signed approval for all waivers.
Permissive gating for faster iteration. Higher anchors for gentler decay. Up to 100 active waivers. KEV still blocks (waivable).
§7 Threat Intelligence Pipeline
EPSS
FIRST.org • Daily
~240,000 CVEs
KEV
CISA • Weekly
~1,200 CVEs
NVD
NIST • Continuous
~280,000 CVEs
Fail-Open Design (§7.E)
When threat intel is unavailable, Spyda continues scoring with reduced fidelity rather than blocking CI/CD. This prevents a single-point-of-failure on FIRST.org/CISA infrastructure cascading into customer deployments. Findings receive data_freshness_warning annotations.
§11 Validation Methodology
The validation protocol is published in advance (pre-registration) so customers, auditors, and reviewers can assess adequacy before results are produced. The methodology is reproducible by construction; no Appendix N release-blocking condition is yet verified (0 of 40). After v2.6.4 the residual defect class is empirical, not specificational — the specification work programme is closed.
Validation Corpus
- • NIST Juliet Test Suite (~85,000 cases)
- • OWASP Benchmark (~2,700 cases)
- • BigVul, CVEFixes, DiverseVul (real-world)
- • 50-project holdout (stratified sampling)
Metrics
- • Expected Calibration Error (ECE) < 0.05
- • AUC-PR with 95% bootstrap CIs
- • Per-stratum performance (language, CWE)
- • Inter-rater reliability (Cohen's κ ≥ 0.65)
Full whitepaper available upon request for enterprise evaluations.