Version 2.0 • Last Updated: December 2024

Spyda 2.0: Intelligent Security Analysis Platform

A comprehensive technical specification for the next-generation TrustScore platform with confidence-weighted scoring, GitHub-native integration, and advanced algorithmic intelligence.

Authors: Spyda Security Team

Classification: Enterprise & Open Source

1.
Executive Summary
2.
System Architecture
1. 2.1 FastAPI & Celery Backend
2. 2.2 PostgreSQL JSONB Storage
3. 2.3 Redis Caching Layer
4. 2.4 Kubernetes Deployment
3.
Confidence-Weighted Scoring Engine
1. 3.1 5-Factor Probabilistic Classifier
2. 3.2 Dynamic Weight Adjustment Layer
3. 3.3 Scanner Reliability Calibration Loop
4. 3.4 Exploitability Boost Layer
5. 3.5 Contradiction Penalty 2.0
4.
Advanced Algorithms
1. 4.1 Multi-Scanner Correlation Graph
2. 4.2 Temporal Drift Algorithm
3. 4.3 AI-Signature Detection Engine
4. 4.4 PQC Readiness Classifier 2.0
5.
GitHub Native Integration
1. 5.1 GitHub App Model
2. 5.2 Actions Workflows
3. 5.3 SARIF Upload & Security Tab
4. 5.4 PR Checks & Inline Annotations
5. 5.5 Dependency Graph Integration
6. 5.6 Issue Automation
7. 5.7 GHAS Interoperability
6.
UI/UX: Spyda Console
1. 6.1 PR TrustScore Delta View
2. 6.2 Findings Panel with GitHub Context
3. 6.3 Fixer Bot Interactions
4. 6.4 Policy Editor with Repo Binding
5. 6.5 Code Owner Mapping Visualization
7.
API Reference
8.
Deployment & Operations
9.
Future Roadmap

1. Executive Summary

Spyda 2.0 represents a paradigm shift in application security analysis. Unlike traditional security scanners that simply aggregate findings, Spyda applies evidence-based probabilistic reasoning to assess the confidence and severity of security findings from multiple tools.

Key Innovations

Confidence-Weighted Scoring Engine: 5-factor model (Corroboration, Clarity, Credibility, Exploitability, Contradiction) with dynamic weight adjustment based on project context
Advanced Algorithmic Intelligence: Multi-scanner correlation graphs, temporal drift detection, AI-signature analysis, and PQC readiness classification
GitHub-Native Integration: First-class support for GitHub Actions, SARIF uploads, PR checks, dependency graphs, and GitHub Advanced Security interoperability
Production-Ready Architecture: FastAPI + Celery backend, PostgreSQL with JSONB storage, Redis caching, and Kubernetes-ready deployment

🎯 Core Value Proposition

Spyda transforms security tool noise into actionable intelligence by applying context-aware confidence scoring, reducing false positives by up to 60%, and providing security teams with high-confidence findings that warrant immediate attention.

2. System Architecture

Architectural Overview

Spyda 2.0 follows a modern microservices architecture designed for scalability, reliability, and real-time analysis of security findings.

┌─────────────────────────────────────────────────────────────┐
│                      GitHub Ecosystem                       │
│  Actions • SARIF • Dep Graph • Secret Scan • CodeQL       │
└─────────────────────┬───────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────┐
│                   FastAPI REST API Layer                    │
│  /v1/scan • /v1/score • /v1/findings • /v1/policy        │
└─────────────────────┬───────────────────────────────────────┘
                      │
        ┌─────────────┴─────────────┐
        ▼                           ▼
┌───────────────────┐       ┌───────────────────┐
│  Celery Workers   │       │   Redis Cache     │
│  • Correlation    │◄──────┤   • Job Queue     │
│  • Drift Analysis │       │   • Session Store │
│  • AI Detection   │       └───────────────────┘
└─────────┬─────────┘
          │
          ▼
┌───────────────────────────────────────────────────────────┐
│            PostgreSQL 15 with JSONB Storage               │
│  • scan_results (JSONB reports with GIN indexing)       │
│  • policies • waivers • scanner_credibility             │
└───────────────────────────────────────────────────────────┘

2.1 FastAPI Backend

• RESTful API with automatic OpenAPI documentation
• Async/await for high-concurrency request handling
• Pydantic models for request/response validation
• JWT-based authentication with GitHub OAuth

2.2 Celery Task Queue

• Background processing for heavy algorithms
• Distributed task execution across workers
• Priority queues for critical findings
• Automatic retries with exponential backoff

2.3 PostgreSQL + JSONB

• JSONB columns for flexible schema-less storage
• GIN indexes for deep JSON querying
• Partitioning by project/repo for scaling
• Time-series optimization for dashboard queries

2.4 Kubernetes Deployment

• Horizontal pod autoscaling (HPA) for API/workers
• Helm charts for reproducible deployments
• Health checks & liveness probes
• Rolling updates with zero downtime

🔧 Technical Stack Summary: FastAPI 0.109+ • PostgreSQL 15+ • Redis 7+ • Celery 5+ • Kubernetes 1.27+ • Python 3.10+

3. Confidence-Weighted Scoring Engine

The heart of Spyda 2.0 is its probabilistic confidence engine that applies evidence-based reasoning to security findings, transforming raw scanner output into high-confidence, actionable intelligence.

3.1 Five-Factor Probabilistic Classifier

Confidence = (w₁ × Corroboration) + (w₂ × Clarity) + (w₃ × Credibility) + (w₄ × Exploitability) - (w₅ × Contradiction)

Default weights: w₁=30%, w₂=20%, w₃=20%, w₄=20%, w₅=10% (dynamically adjusted per policy)

Factor 1: Corroboration (30%)

How many independent sources confirm the finding? Higher corroboration = higher confidence.

Sources	Score	Interpretation
1 source	0.50	Single source, possible false positive
2 sources	0.75	Moderate confidence, likely valid
3+ sources	1.00	High confidence, corroborated by multiple tools

Factor 2: Clarity (20%)

Specificity of evidence: file paths, line numbers, function names, exact vulnerability details.

High Clarity (1.0): "SQL injection in login.py:42, function authenticate_user()"

Medium Clarity (0.6): "SQL injection detected in authentication module"

Low Clarity (0.3): "Potential SQL injection in codebase"

Factor 3: Source Credibility (20%)

Average reliability of scanning tools, calibrated over time based on false positive rates.

Calibration Formula: Credibility_final = 0.70 × Learned_credibility + 0.30 × Base_credibility

Initial credibility values: Snyk (90%), Semgrep (85%), Trivy (80%), Unknown tools (50%)

Factor 4: Exploitability (20%)

Real-world threat intelligence from CISA KEV, EPSS scores, and vulnerability databases.

CISA KEV Listed1.00 (100%)

EPSS ≥ 0.30+0.20 boost

GitHub Advisory TrendingDynamic boost

Factor 5: Contradiction (-10%)

Penalty when scanners disagree on severity, preventing overconfidence.

Severe Conflict (CRITICAL vs INFO)-40%

Minor Conflict (HIGH vs MEDIUM)-20%

Agreement0%

3.2 Dynamic Weight Adjustment Layer

Spyda adapts confidence weights based on project characteristics detected from GitHub metadata.

Supply Chain Projects

Detected via high dependency count

Corroboration: 25%

Clarity: 15%

Credibility: 20%

Exploitability: 35% ⬆

Contradiction: 5%

AI/ML Projects

Detected via tensorflow, pytorch deps

Corroboration: 25%

Clarity: 35% ⬆

Credibility: 20%

Exploitability: 15%

Contradiction: 5%

Compliance-Focused

Detected via SOC2, HIPAA tags

Corroboration: 20%

Clarity: 20%

Credibility: 40% ⬆

Exploitability: 15%

Contradiction: 5%

3.3 Scanner Reliability Calibration Loop

Spyda learns scanner reliability over time, dynamically adjusting credibility based on false positive rates per team and per repository.

Calibration Algorithm

1Initial State: All scanners start with default credibility (Snyk=90%, Semgrep=85%, etc.)
2False Positive Tracking: When findings are marked as false positives in console, scanner credibility decreases by 5%
3Blended Formula: Credibility_final = 0.70 × Learned + 0.30 × Base (minimum 30%)
4Context-Specific: Calibration is per-repo and per-team, so a scanner may be reliable for Python but not for JavaScript

Example: Scanner X produces 10 FPs → Credibility drops from 85% → 50%
Formula: 0.70 × 0.35 + 0.30 × 0.85 = 0.50

3.4 Exploitability Boost Layer

Real-time threat intelligence integration automatically boosts confidence when vulnerabilities are actively exploited in the wild.

KEV

CISA Known Exploited Vulnerabilities

CVE listed in CISA KEV catalog → Exploitability = 100%

API: https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json

EPSS

EPSS Score ≥ 0.30

Exploit Prediction Scoring System ≥ 30% → Add +20% boost

API: https://api.first.org/data/v1/epss?cve=CVE-XXXX-YYYY

GitHub Security Advisories

Trending vulnerabilities in GitHub advisories → Dynamic boost based on severity and popularity

API: https://api.github.com/advisories

3.5 Contradiction Penalty 2.0

Dynamic penalty applied when scanners provide conflicting severity assessments, preventing overconfidence.

Severe Conflict

SAST: CRITICAL vs DAST: INFO/LOW

-40%

Minor Conflict

1-2 severity levels apart (HIGH vs MEDIUM)

-20%

Agreement

All scanners report same severity

Rationale: Contradictory evidence suggests uncertainty. If Snyk flags SQL injection as CRITICAL but CodeQL (testing same endpoint) says it's safe, confidence should be reduced until manual review confirms the finding.

4. Advanced Algorithms

Spyda 2.0 introduces four cutting-edge algorithms that transform raw scanner output into contextualized, high-confidence security intelligence.

4.1 Multi-Scanner Correlation Graph (MSCG)

Graph-based correlation linking scanner findings with GitHub dependency and code-owner maps for improved corroboration accuracy.

Algorithm Overview

1Node Creation: Each finding from each scanner becomes a graph node with metadata (file, component, dependencies, code owners)
2Edge Calculation: Edges connect findings based on correlation strength
3Corroboration Boost: Graph structure enhances confidence for correlated findings

Edge Weight Calculation

Relationship	Weight	GitHub Integration
Same File	0.90	Direct file path match
Same Component	0.85	GitHub language stats + package detection
Dependency Chain	0.60	GitHub Dependency Graph API traversal
Code Owner	0.30	CODEOWNERS file parsing

Mathematical Formula

λ (corroboration_multiplier) = (Σ edge_weights) / max_possible_edges

Enhanced Confidence = Base_Confidence × (1 + 0.2 × λ)

Example:
  - Snyk finds SQL injection in login.py (Base: 75%)
  - Semgrep confirms same file (edge weight: 0.9)
  - λ = 0.9 / 1.0 = 0.9
  - Enhanced: 75% × (1 + 0.2 × 0.9) = 88.5%

4.2 Temporal Drift Algorithm

Detects "zombie vulnerabilities" - stale risk in unchanged files with old vulnerabilities, applying exponential decay to credibility.

Exponential Decay Model

D(t) = 1 - e^(-α × t)

where:
  D(t) = Drift score (0-1, higher = more stale)
  α = Decay constant (policy-defined, default 0.005)
  t = Days since last file modification

Credibility_adjusted = Credibility_base × (1 - D × 0.7)

Floor: Never drop below 30% credibility

Drift Tiers

High Drift (Zombie Vulnerability)

-30%

File unchanged for 180+ days with 90+ day old vulnerability

Moderate Drift

-15%

File unchanged for 90+ days with 30+ day old vulnerability

No Drift

Recent code or newly detected vulnerability

4.3 AI-Signature Detection Engine

Uses token patterns, syntax embedding clusters, and training-data fingerprints to detect AI-generated code that may contain security flaws.

Detection Signals (Max 6)

1. Suspicious Comments

"// TODO: This function", "// Example usage:", "// Helper function to"

2. Generic Naming

Functions named handle*, process*, execute*, perform*, do*

3. Excessive Comments

>30% of lines are inline comments (unusual)

4. Generic Commits

"update", "fix", "add feature", "improve code"

5. No File History

Large file (500+ lines) with zero prior commits

6. Repetitive Patterns

Short, generic arrow functions with no domain specificity

Confidence = min(1.0, signal_count / 6)
isAIGenerated = Confidence > 0.5

Model Fingerprinting

GPT-4/CopilotContains "// Example:" or "// Usage:" patterns

Claude/GeminiVerbose explanatory comments with formal language

UnknownAI signals detected but no clear model fingerprint

⚠️ Security Implication: AI-generated code may contain subtle security flaws from training data (e.g., SQL injection patterns learned from vulnerable examples). Flagging AI code ensures mandatory human security review before deployment.

4.4 PQC Readiness Classifier 2.0

Classifies repositories into Post-Quantum Cryptography risk tiers, integrating GitHub secret scanning and dependency graph for comprehensive quantum-threat assessment.

Risk Tier Classification

Tier 0: Critically Vulnerable

CRITICAL

Using broken/deprecated algorithms (MD5, SHA-1, DES, hardcoded keys)

Action: Emergency migration to SHA-256/SHA-3 before PQC considerations

Tier 1: Critical PQC Risk

HIGH

5+ vulnerable crypto operations (RSA, ECDSA, DH) with no PQC-ready libraries

Action: Immediate migration to Kyber, Dilithium required

Tier 2: High PQC Risk

MEDIUM

Vulnerable crypto usage, no PQC primitives

Action: Begin PQC readiness assessment

Tier 3: Moderate PQC Risk

LOW

Mixed crypto with some PQC-ready libraries

Action: Expand PQC coverage to all operations

Tier 4: Low PQC Risk

SAFE

PQC-ready or minimal crypto exposure

Status: Repository is quantum-safe

Vulnerable Algorithms Detected

RSADSAECDSAECDHDHElGamalMD5SHA-1

PQC-Ready Libraries Detected

liboqskyberdilithiumfalconsphincsntru

🔗 GitHub Integration: Automatically scans GitHub Dependency Graph for crypto libraries and GitHub Secret Scanning results for hardcoded keys that need quantum-safe rotation.

5. GitHub Native Integration

First-class GitHub ecosystem integration for seamless security workflows

Spyda 2.0 is built from the ground up for GitHub. All features integrate natively with GitHub's ecosystem, from Actions workflows to Security tabs to dependency graphs.

5.1 GitHub App Model

Spyda installs as a GitHub App with granular permissions, ensuring secure read-only access to repositories.

Required Permissions

Contents: Read-only

Access repository code and file structure

Metadata: Read

Repository topics, language stats, owner info

Dependency Graph: Read

Analyze dependencies for supply chain risks

Checks: Write

Create PR status checks and annotations

Security Events: Write

Upload SARIF results to Security tab

Issues: Write

Auto-open issues for critical findings

5.2 GitHub Actions Workflow

Spyda runs as a GitHub Actions workflow, triggering on push and pull_request events.

.github/workflows/spyda-scan.yml

GitHub Actions

name: Spyda Security Scan
on: [push, pull_request]

jobs:
  spyda-analysis:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/cache@v4
        with:
          path: .spyda/cache
          key: spyda-${{ runner.os }}-${{ hashFiles('**/lock.json') }}
      
      - uses: spyda-sec/action@v2
        with:
          api-key: ${{ secrets.SPYDA_API_KEY }}
          fail-on-threshold: true
          threshold: 80  # Block PR if TrustScore < 80
          
      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: spyda-results.sarif

5.3 SARIF Upload & Security Tab

All findings are uploaded as SARIF (Static Analysis Results Interchange Format) to GitHub's native Security tab, providing inline code annotations and historical tracking.

SARIF Benefits

✓Native GitHub UI: Findings appear in Security > Code Scanning Alerts with file/line annotations
✓Historical Tracking: GitHub tracks when findings are introduced/resolved across commits
✓PR Integration: Findings automatically comment on PRs that introduce new issues
✓Team Collaboration: Security teams can dismiss, assign, or comment on findings directly in GitHub

5.4 PR Checks & Inline Annotations

Spyda creates GitHub Checks on pull requests, blocking merges when TrustScore drops below policy thresholds.

PR Check Behavior

TrustScore ≥ 80 → ✓ Check Passes

PR can be merged, no blocking issues detected

TrustScore < 80 → ✗ Check Fails

PR merge blocked until findings are resolved or waived

5.5 Dependency Graph Integration

Spyda reads GitHub's dependency graph to correlate findings with specific packages and maintainer health stats.

Dependency Metadata Enrichment

→Transitive Dependencies: Deep traversal to find vulnerabilities in nested packages
→Maintainer Activity: GitHub repo activity, last commit date, contributor count
→Supply Chain Risk: Detect abandoned packages, single-maintainer risks, unmaintained forks
→License Compliance: Identify incompatible licenses across dependency tree

5.6 Issue Automation

Spyda automatically opens GitHub Issues when TrustScore drops significantly or when critical vulnerabilities are detected.

Auto-Generated Issue Example

criticalsecurity

[Spyda Alert] TrustScore dropped from 85 → 62 (CRITICAL)

Spyda detected 3 new CRITICAL findings in your latest commit:

1. SQL Injection in `api/auth.py:42` (Confidence: 95%)
2. Hardcoded AWS Secret Key in `config.ts` (Confidence: 100%)
3. SSRF in `utils/fetch-helper.js:18` (Confidence: 88%)

View full report: [Spyda Console Dashboard]

5.7 GitHub Advanced Security (GHAS) Interoperability

Spyda enhances GitHub Advanced Security (CodeQL, Secret Scanning) by correlating results with multi-scanner findings, applying confidence weights, and reducing false positives.

GHAS Integration Flow

CodeQL Findings

Secret Scanning

Spyda Confidence Engine

High-Confidence TrustScore

Result: Spyda ingests CodeQL and Secret Scanning results via GitHub API, correlates them with third-party scanner findings (Snyk, Semgrep, Trivy), applies confidence-weighted scoring, and outputs a unified TrustScore that reduces false positives by up to 60%.

6. UI/UX: Spyda Console

The Spyda Console provides security teams with an intuitive, GitHub-integrated interface for viewing TrustScores, triaging findings, and managing security policies.

6.1 PR TrustScore Delta View

Real-time TrustScore delta display on pull requests, showing the impact of code changes on security posture.

Base (main branch)

→

Head (PR branch)

DELTA

-7

Impact Analysis:

+2 CRITICAL vulnerabilities detected
SQL injection in new auth endpoint
Hardcoded API key in config file

6.2 Findings Panel with GitHub Context

Findings are displayed with rich GitHub context: file owners, commit history, dependency chain, and PR links.

CRITICAL95% Confidence

2 sources

SQL Injection in Authentication Module

api/auth.py:42

Code Owner: @security-team

Introduced: 2 days ago in PR #342

EPSS Score: 0.87 (Active Exploitation Likely)

6.3 Fixer Bot Interactions

Spyda's AI-powered Fixer Bot can automatically suggest or apply fixes for common vulnerabilities, creating draft PRs for review.

Spyda Bot

I detected a SQL injection vulnerability in your authentication function. I can fix this by using parameterized queries. Would you like me to create a PR?

6.4 Policy Editor with Repo Binding

Customize confidence weights, severity thresholds, and waiver rules per repository or organization.

Policy: Production Apps

Applied to: @org/frontend, @org/backend

Corroboration Weight

30%

Exploitability Weight

35% (Supply Chain Focused)

Pass Threshold

Fail Threshold

6.5 Code Owner Mapping Visualization

Visual map of code ownership from CODEOWNERS file, showing which teams are responsible for which findings.

@security-team

8 findings

Avg Severity: CRITICAL

Paths: /api/auth/*, /crypto/*

@frontend-team

3 findings

Avg Severity: MEDIUM

Paths: /components/*, /ui/*

@data-team

12 findings

Avg Severity: HIGH

Paths: /models/*, /db/*

7. API Reference

Spyda 2.0 exposes a comprehensive RESTful API for programmatic access to scanning, scoring, and policy management.

Core Endpoints

POST/v1/score

Calculate TrustScore from scanner findings

{
  "project": "my-app",
  "findings": [...],
  "policy_name": "production"
}

Response:
{
  "trustscore": 85,
  "grade": "PASS",
  "breakdown": { "vulnerabilities": 80, ... }
}

GET/v1/scan/:scanId

Retrieve historical scan results by ID

POST/v1/findings/waiver

Request waiver for false positive findings

PUT/v1/policy/:policyId

Update policy configuration

Full API Documentation: Visit /docs/api for complete OpenAPI specification with interactive testing.

8. Deployment & Operations

Kubernetes Deployment

Spyda 2.0 is designed for cloud-native deployment with Kubernetes, featuring horizontal autoscaling, health checks, and zero-downtime rolling updates.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: spyda-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: spyda-api
  template:
    metadata:
      labels:
        app: spyda-api
    spec:
      containers:
      - name: api
        image: spyda/api:2.0
        ports:
        - containerPort: 8000
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: spyda-secrets
              key: db-url
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 30
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "2000m"

Infrastructure Requirements

PostgreSQL

• PostgreSQL 15+
• JSONB support required
• GIN indexes enabled
• Recommended: 4 vCPU, 8GB RAM

Redis

• Redis 7+
• Persistence enabled
• Used for job queue & cache
• Recommended: 2 vCPU, 4GB RAM

API/Workers

• API: 3+ replicas
• Workers: 2+ replicas
• Per pod: 2 vCPU, 4GB RAM
• HPA: Scale to 10 pods

Monitoring & Observability

• Metrics: Prometheus metrics exported on /metrics endpoint (API latency, queue depth, DB connections)
• Logs: Structured JSON logging to stdout, aggregated via FluentD or Datadog
• Tracing: OpenTelemetry integration for distributed tracing across API → Workers → DB
• Alerts: PagerDuty integration for critical errors (DB connection loss, queue overflow)

9. Future Roadmap

Spyda 2.0 is continuously evolving. Upcoming features focus on deeper AI integration, advanced threat modeling, and expanded ecosystem support.

Q1 2025: ML-Enhanced Confidence

Replace heuristic confidence scoring with gradient-boosted decision trees trained on historical false positive rates, improving accuracy by 15-20%.

Q2 2025: Threat Modeling as Code

Generate STRIDE threat models from codebase structure, identifying architectural security gaps beyond individual vulnerabilities.

Q3 2025: GitLab & Bitbucket Support

Extend GitHub-native integration to GitLab and Bitbucket, maintaining feature parity with SARIF uploads and merge request gating.

Q4 2025: Compliance Automation

Auto-generate compliance evidence for SOC 2, ISO 27001, and FedRAMP based on Spyda scan history and remediation tracking.

Version 2.0 • Last Updated: December 2024

Spyda 2.0: Intelligent Security Analysis Platform

A comprehensive technical specification for the next-generation TrustScore platform with confidence-weighted scoring, GitHub-native integration, and advanced algorithmic intelligence.

Authors: Spyda Security Team

Classification: Enterprise & Open Source

1.
Executive Summary
2.
System Architecture
1. 2.1 FastAPI & Celery Backend
2. 2.2 PostgreSQL JSONB Storage
3. 2.3 Redis Caching Layer
4. 2.4 Kubernetes Deployment
3.
Confidence-Weighted Scoring Engine
1. 3.1 5-Factor Probabilistic Classifier
2. 3.2 Dynamic Weight Adjustment Layer
3. 3.3 Scanner Reliability Calibration Loop
4. 3.4 Exploitability Boost Layer
5. 3.5 Contradiction Penalty 2.0
4.
Advanced Algorithms
1. 4.1 Multi-Scanner Correlation Graph
2. 4.2 Temporal Drift Algorithm
3. 4.3 AI-Signature Detection Engine
4. 4.4 PQC Readiness Classifier 2.0
5.
GitHub Native Integration
1. 5.1 GitHub App Model
2. 5.2 Actions Workflows
3. 5.3 SARIF Upload & Security Tab
4. 5.4 PR Checks & Inline Annotations
5. 5.5 Dependency Graph Integration
6. 5.6 Issue Automation
7. 5.7 GHAS Interoperability
6.
UI/UX: Spyda Console
1. 6.1 PR TrustScore Delta View
2. 6.2 Findings Panel with GitHub Context
3. 6.3 Fixer Bot Interactions
4. 6.4 Policy Editor with Repo Binding
5. 6.5 Code Owner Mapping Visualization
7.
API Reference
8.
Deployment & Operations
9.
Future Roadmap

1. Executive Summary

Key Innovations

Confidence-Weighted Scoring Engine: 5-factor model (Corroboration, Clarity, Credibility, Exploitability, Contradiction) with dynamic weight adjustment based on project context
Advanced Algorithmic Intelligence: Multi-scanner correlation graphs, temporal drift detection, AI-signature analysis, and PQC readiness classification
GitHub-Native Integration: First-class support for GitHub Actions, SARIF uploads, PR checks, dependency graphs, and GitHub Advanced Security interoperability
Production-Ready Architecture: FastAPI + Celery backend, PostgreSQL with JSONB storage, Redis caching, and Kubernetes-ready deployment

🎯 Core Value Proposition

2. System Architecture

Architectural Overview

Spyda 2.0 follows a modern microservices architecture designed for scalability, reliability, and real-time analysis of security findings.

┌─────────────────────────────────────────────────────────────┐
│                      GitHub Ecosystem                       │
│  Actions • SARIF • Dep Graph • Secret Scan • CodeQL       │
└─────────────────────┬───────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────┐
│                   FastAPI REST API Layer                    │
│  /v1/scan • /v1/score • /v1/findings • /v1/policy        │
└─────────────────────┬───────────────────────────────────────┘
                      │
        ┌─────────────┴─────────────┐
        ▼                           ▼
┌───────────────────┐       ┌───────────────────┐
│  Celery Workers   │       │   Redis Cache     │
│  • Correlation    │◄──────┤   • Job Queue     │
│  • Drift Analysis │       │   • Session Store │
│  • AI Detection   │       └───────────────────┘
└─────────┬─────────┘
          │
          ▼
┌───────────────────────────────────────────────────────────┐
│            PostgreSQL 15 with JSONB Storage               │
│  • scan_results (JSONB reports with GIN indexing)       │
│  • policies • waivers • scanner_credibility             │
└───────────────────────────────────────────────────────────┘

2.1 FastAPI Backend

• RESTful API with automatic OpenAPI documentation
• Async/await for high-concurrency request handling
• Pydantic models for request/response validation
• JWT-based authentication with GitHub OAuth

2.2 Celery Task Queue

• Background processing for heavy algorithms
• Distributed task execution across workers
• Priority queues for critical findings
• Automatic retries with exponential backoff

2.3 PostgreSQL + JSONB

• JSONB columns for flexible schema-less storage
• GIN indexes for deep JSON querying
• Partitioning by project/repo for scaling
• Time-series optimization for dashboard queries

2.4 Kubernetes Deployment

• Horizontal pod autoscaling (HPA) for API/workers
• Helm charts for reproducible deployments
• Health checks & liveness probes
• Rolling updates with zero downtime

🔧 Technical Stack Summary: FastAPI 0.109+ • PostgreSQL 15+ • Redis 7+ • Celery 5+ • Kubernetes 1.27+ • Python 3.10+

3. Confidence-Weighted Scoring Engine

3.1 Five-Factor Probabilistic Classifier

Confidence = (w₁ × Corroboration) + (w₂ × Clarity) + (w₃ × Credibility) + (w₄ × Exploitability) - (w₅ × Contradiction)

Default weights: w₁=30%, w₂=20%, w₃=20%, w₄=20%, w₅=10% (dynamically adjusted per policy)

Factor 1: Corroboration (30%)

How many independent sources confirm the finding? Higher corroboration = higher confidence.

Sources	Score	Interpretation
1 source	0.50	Single source, possible false positive
2 sources	0.75	Moderate confidence, likely valid
3+ sources	1.00	High confidence, corroborated by multiple tools

Factor 2: Clarity (20%)

Specificity of evidence: file paths, line numbers, function names, exact vulnerability details.

High Clarity (1.0): "SQL injection in login.py:42, function authenticate_user()"

Medium Clarity (0.6): "SQL injection detected in authentication module"

Low Clarity (0.3): "Potential SQL injection in codebase"

Factor 3: Source Credibility (20%)

Average reliability of scanning tools, calibrated over time based on false positive rates.

Calibration Formula: Credibility_final = 0.70 × Learned_credibility + 0.30 × Base_credibility

Initial credibility values: Snyk (90%), Semgrep (85%), Trivy (80%), Unknown tools (50%)

Factor 4: Exploitability (20%)

Real-world threat intelligence from CISA KEV, EPSS scores, and vulnerability databases.

CISA KEV Listed1.00 (100%)

EPSS ≥ 0.30+0.20 boost

GitHub Advisory TrendingDynamic boost

Factor 5: Contradiction (-10%)

Penalty when scanners disagree on severity, preventing overconfidence.

Severe Conflict (CRITICAL vs INFO)-40%

Minor Conflict (HIGH vs MEDIUM)-20%

Agreement0%

3.2 Dynamic Weight Adjustment Layer

Spyda adapts confidence weights based on project characteristics detected from GitHub metadata.

Supply Chain Projects

Detected via high dependency count

Corroboration: 25%

Clarity: 15%

Credibility: 20%

Exploitability: 35% ⬆

Contradiction: 5%

AI/ML Projects

Detected via tensorflow, pytorch deps

Corroboration: 25%

Clarity: 35% ⬆

Credibility: 20%

Exploitability: 15%

Contradiction: 5%

Compliance-Focused

Detected via SOC2, HIPAA tags

Corroboration: 20%

Clarity: 20%

Credibility: 40% ⬆

Exploitability: 15%

Contradiction: 5%

3.3 Scanner Reliability Calibration Loop

Spyda learns scanner reliability over time, dynamically adjusting credibility based on false positive rates per team and per repository.

Calibration Algorithm

1Initial State: All scanners start with default credibility (Snyk=90%, Semgrep=85%, etc.)
2False Positive Tracking: When findings are marked as false positives in console, scanner credibility decreases by 5%
3Blended Formula: Credibility_final = 0.70 × Learned + 0.30 × Base (minimum 30%)
4Context-Specific: Calibration is per-repo and per-team, so a scanner may be reliable for Python but not for JavaScript

Example: Scanner X produces 10 FPs → Credibility drops from 85% → 50%
Formula: 0.70 × 0.35 + 0.30 × 0.85 = 0.50

3.4 Exploitability Boost Layer

Real-time threat intelligence integration automatically boosts confidence when vulnerabilities are actively exploited in the wild.

KEV

CISA Known Exploited Vulnerabilities

CVE listed in CISA KEV catalog → Exploitability = 100%

API: https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json

EPSS

EPSS Score ≥ 0.30

Exploit Prediction Scoring System ≥ 30% → Add +20% boost

API: https://api.first.org/data/v1/epss?cve=CVE-XXXX-YYYY

GitHub Security Advisories

Trending vulnerabilities in GitHub advisories → Dynamic boost based on severity and popularity

API: https://api.github.com/advisories

3.5 Contradiction Penalty 2.0

Dynamic penalty applied when scanners provide conflicting severity assessments, preventing overconfidence.

Severe Conflict

SAST: CRITICAL vs DAST: INFO/LOW

-40%

Minor Conflict

1-2 severity levels apart (HIGH vs MEDIUM)

-20%

Agreement

All scanners report same severity

4. Advanced Algorithms

Spyda 2.0 introduces four cutting-edge algorithms that transform raw scanner output into contextualized, high-confidence security intelligence.

4.1 Multi-Scanner Correlation Graph (MSCG)

Graph-based correlation linking scanner findings with GitHub dependency and code-owner maps for improved corroboration accuracy.

Algorithm Overview

1Node Creation: Each finding from each scanner becomes a graph node with metadata (file, component, dependencies, code owners)
2Edge Calculation: Edges connect findings based on correlation strength
3Corroboration Boost: Graph structure enhances confidence for correlated findings

Edge Weight Calculation

Relationship	Weight	GitHub Integration
Same File	0.90	Direct file path match
Same Component	0.85	GitHub language stats + package detection
Dependency Chain	0.60	GitHub Dependency Graph API traversal
Code Owner	0.30	CODEOWNERS file parsing

Mathematical Formula

λ (corroboration_multiplier) = (Σ edge_weights) / max_possible_edges

Enhanced Confidence = Base_Confidence × (1 + 0.2 × λ)

Example:
  - Snyk finds SQL injection in login.py (Base: 75%)
  - Semgrep confirms same file (edge weight: 0.9)
  - λ = 0.9 / 1.0 = 0.9
  - Enhanced: 75% × (1 + 0.2 × 0.9) = 88.5%

4.2 Temporal Drift Algorithm

Detects "zombie vulnerabilities" - stale risk in unchanged files with old vulnerabilities, applying exponential decay to credibility.

Exponential Decay Model

D(t) = 1 - e^(-α × t)

where:
  D(t) = Drift score (0-1, higher = more stale)
  α = Decay constant (policy-defined, default 0.005)
  t = Days since last file modification

Credibility_adjusted = Credibility_base × (1 - D × 0.7)

Floor: Never drop below 30% credibility

Drift Tiers

High Drift (Zombie Vulnerability)

-30%

File unchanged for 180+ days with 90+ day old vulnerability

Moderate Drift

-15%

File unchanged for 90+ days with 30+ day old vulnerability

No Drift

Recent code or newly detected vulnerability

4.3 AI-Signature Detection Engine

Uses token patterns, syntax embedding clusters, and training-data fingerprints to detect AI-generated code that may contain security flaws.

Detection Signals (Max 6)

1. Suspicious Comments

"// TODO: This function", "// Example usage:", "// Helper function to"

2. Generic Naming

Functions named handle*, process*, execute*, perform*, do*

3. Excessive Comments

>30% of lines are inline comments (unusual)

4. Generic Commits

"update", "fix", "add feature", "improve code"

5. No File History

Large file (500+ lines) with zero prior commits

6. Repetitive Patterns

Short, generic arrow functions with no domain specificity

Confidence = min(1.0, signal_count / 6)
isAIGenerated = Confidence > 0.5

Model Fingerprinting

GPT-4/CopilotContains "// Example:" or "// Usage:" patterns

Claude/GeminiVerbose explanatory comments with formal language

UnknownAI signals detected but no clear model fingerprint

4.4 PQC Readiness Classifier 2.0

Classifies repositories into Post-Quantum Cryptography risk tiers, integrating GitHub secret scanning and dependency graph for comprehensive quantum-threat assessment.

Risk Tier Classification

Tier 0: Critically Vulnerable

CRITICAL

Using broken/deprecated algorithms (MD5, SHA-1, DES, hardcoded keys)

Action: Emergency migration to SHA-256/SHA-3 before PQC considerations

Tier 1: Critical PQC Risk

HIGH

5+ vulnerable crypto operations (RSA, ECDSA, DH) with no PQC-ready libraries

Action: Immediate migration to Kyber, Dilithium required

Tier 2: High PQC Risk

MEDIUM

Vulnerable crypto usage, no PQC primitives

Action: Begin PQC readiness assessment

Tier 3: Moderate PQC Risk

LOW

Mixed crypto with some PQC-ready libraries

Action: Expand PQC coverage to all operations

Tier 4: Low PQC Risk

SAFE

PQC-ready or minimal crypto exposure

Status: Repository is quantum-safe

Vulnerable Algorithms Detected

RSADSAECDSAECDHDHElGamalMD5SHA-1

PQC-Ready Libraries Detected

liboqskyberdilithiumfalconsphincsntru

🔗 GitHub Integration: Automatically scans GitHub Dependency Graph for crypto libraries and GitHub Secret Scanning results for hardcoded keys that need quantum-safe rotation.

5. GitHub Native Integration

First-class GitHub ecosystem integration for seamless security workflows

Spyda 2.0 is built from the ground up for GitHub. All features integrate natively with GitHub's ecosystem, from Actions workflows to Security tabs to dependency graphs.

5.1 GitHub App Model

Spyda installs as a GitHub App with granular permissions, ensuring secure read-only access to repositories.

Required Permissions

Contents: Read-only

Access repository code and file structure

Metadata: Read

Repository topics, language stats, owner info

Dependency Graph: Read

Analyze dependencies for supply chain risks

Checks: Write

Create PR status checks and annotations

Security Events: Write

Upload SARIF results to Security tab

Issues: Write

Auto-open issues for critical findings

5.2 GitHub Actions Workflow

Spyda runs as a GitHub Actions workflow, triggering on push and pull_request events.

.github/workflows/spyda-scan.yml

GitHub Actions

name: Spyda Security Scan
on: [push, pull_request]

jobs:
  spyda-analysis:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/cache@v4
        with:
          path: .spyda/cache
          key: spyda-${{ runner.os }}-${{ hashFiles('**/lock.json') }}
      
      - uses: spyda-sec/action@v2
        with:
          api-key: ${{ secrets.SPYDA_API_KEY }}
          fail-on-threshold: true
          threshold: 80  # Block PR if TrustScore < 80
          
      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: spyda-results.sarif

5.3 SARIF Upload & Security Tab

All findings are uploaded as SARIF (Static Analysis Results Interchange Format) to GitHub's native Security tab, providing inline code annotations and historical tracking.

SARIF Benefits

✓Native GitHub UI: Findings appear in Security > Code Scanning Alerts with file/line annotations
✓Historical Tracking: GitHub tracks when findings are introduced/resolved across commits
✓PR Integration: Findings automatically comment on PRs that introduce new issues
✓Team Collaboration: Security teams can dismiss, assign, or comment on findings directly in GitHub

5.4 PR Checks & Inline Annotations

Spyda creates GitHub Checks on pull requests, blocking merges when TrustScore drops below policy thresholds.

PR Check Behavior

TrustScore ≥ 80 → ✓ Check Passes

PR can be merged, no blocking issues detected

TrustScore < 80 → ✗ Check Fails

PR merge blocked until findings are resolved or waived

5.5 Dependency Graph Integration

Spyda reads GitHub's dependency graph to correlate findings with specific packages and maintainer health stats.

Dependency Metadata Enrichment

→Transitive Dependencies: Deep traversal to find vulnerabilities in nested packages
→Maintainer Activity: GitHub repo activity, last commit date, contributor count
→Supply Chain Risk: Detect abandoned packages, single-maintainer risks, unmaintained forks
→License Compliance: Identify incompatible licenses across dependency tree

5.6 Issue Automation

Spyda automatically opens GitHub Issues when TrustScore drops significantly or when critical vulnerabilities are detected.

Auto-Generated Issue Example

criticalsecurity

[Spyda Alert] TrustScore dropped from 85 → 62 (CRITICAL)

5.7 GitHub Advanced Security (GHAS) Interoperability

Spyda enhances GitHub Advanced Security (CodeQL, Secret Scanning) by correlating results with multi-scanner findings, applying confidence weights, and reducing false positives.

GHAS Integration Flow

CodeQL Findings

Secret Scanning

Spyda Confidence Engine

High-Confidence TrustScore

6. UI/UX: Spyda Console

The Spyda Console provides security teams with an intuitive, GitHub-integrated interface for viewing TrustScores, triaging findings, and managing security policies.

6.1 PR TrustScore Delta View

Real-time TrustScore delta display on pull requests, showing the impact of code changes on security posture.

Base (main branch)

→

Head (PR branch)

DELTA

-7

Impact Analysis:

+2 CRITICAL vulnerabilities detected
SQL injection in new auth endpoint
Hardcoded API key in config file

6.2 Findings Panel with GitHub Context

Findings are displayed with rich GitHub context: file owners, commit history, dependency chain, and PR links.

CRITICAL95% Confidence

2 sources

SQL Injection in Authentication Module

api/auth.py:42

Code Owner: @security-team

Introduced: 2 days ago in PR #342

EPSS Score: 0.87 (Active Exploitation Likely)

6.3 Fixer Bot Interactions

Spyda's AI-powered Fixer Bot can automatically suggest or apply fixes for common vulnerabilities, creating draft PRs for review.

Spyda Bot

I detected a SQL injection vulnerability in your authentication function. I can fix this by using parameterized queries. Would you like me to create a PR?

6.4 Policy Editor with Repo Binding

Customize confidence weights, severity thresholds, and waiver rules per repository or organization.

Policy: Production Apps

Applied to: @org/frontend, @org/backend

Corroboration Weight

30%

Exploitability Weight

35% (Supply Chain Focused)

Pass Threshold

Fail Threshold

6.5 Code Owner Mapping Visualization

Visual map of code ownership from CODEOWNERS file, showing which teams are responsible for which findings.

@security-team

8 findings

Avg Severity: CRITICAL

Paths: /api/auth/*, /crypto/*

@frontend-team

3 findings

Avg Severity: MEDIUM

Paths: /components/*, /ui/*

@data-team

12 findings

Avg Severity: HIGH

Paths: /models/*, /db/*

7. API Reference

Spyda 2.0 exposes a comprehensive RESTful API for programmatic access to scanning, scoring, and policy management.

Core Endpoints

POST/v1/score

Calculate TrustScore from scanner findings

{
  "project": "my-app",
  "findings": [...],
  "policy_name": "production"
}

Response:
{
  "trustscore": 85,
  "grade": "PASS",
  "breakdown": { "vulnerabilities": 80, ... }
}

GET/v1/scan/:scanId

Retrieve historical scan results by ID

POST/v1/findings/waiver

Request waiver for false positive findings

PUT/v1/policy/:policyId

Update policy configuration

Full API Documentation: Visit /docs/api for complete OpenAPI specification with interactive testing.

8. Deployment & Operations

Kubernetes Deployment

Spyda 2.0 is designed for cloud-native deployment with Kubernetes, featuring horizontal autoscaling, health checks, and zero-downtime rolling updates.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: spyda-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: spyda-api
  template:
    metadata:
      labels:
        app: spyda-api
    spec:
      containers:
      - name: api
        image: spyda/api:2.0
        ports:
        - containerPort: 8000
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: spyda-secrets
              key: db-url
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 30
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "2000m"

Infrastructure Requirements

PostgreSQL

• PostgreSQL 15+
• JSONB support required
• GIN indexes enabled
• Recommended: 4 vCPU, 8GB RAM

Redis

• Redis 7+
• Persistence enabled
• Used for job queue & cache
• Recommended: 2 vCPU, 4GB RAM

API/Workers

• API: 3+ replicas
• Workers: 2+ replicas
• Per pod: 2 vCPU, 4GB RAM
• HPA: Scale to 10 pods

Monitoring & Observability

• Metrics: Prometheus metrics exported on /metrics endpoint (API latency, queue depth, DB connections)
• Logs: Structured JSON logging to stdout, aggregated via FluentD or Datadog
• Tracing: OpenTelemetry integration for distributed tracing across API → Workers → DB
• Alerts: PagerDuty integration for critical errors (DB connection loss, queue overflow)

9. Future Roadmap

Spyda 2.0 is continuously evolving. Upcoming features focus on deeper AI integration, advanced threat modeling, and expanded ecosystem support.

Q1 2025: ML-Enhanced Confidence

Replace heuristic confidence scoring with gradient-boosted decision trees trained on historical false positive rates, improving accuracy by 15-20%.

Q2 2025: Threat Modeling as Code

Generate STRIDE threat models from codebase structure, identifying architectural security gaps beyond individual vulnerabilities.

Q3 2025: GitLab & Bitbucket Support

Extend GitHub-native integration to GitLab and Bitbucket, maintaining feature parity with SARIF uploads and merge request gating.

Q4 2025: Compliance Automation

Auto-generate compliance evidence for SOC 2, ISO 27001, and FedRAMP based on Spyda scan history and remediation tracking.