← Back to dashboard

Scoring Parity Review

Generated 2026-05-19T22:22:49Z · Overall: FAIL

Inputs

FilePresentas_of
data/rankings.jsonTrue2026-05-19 05:22 PM CDT
data/watchlist_rankings.jsonTrue2026-05-19 05:22 PM CDT

Coverage by group

GroupRowsai_scorefundamentaltechnicalsentimentlow_riskswing_score
main_rankings100 100/100 (100%) 100/100 (100%) 100/100 (100%) 100/100 (100%) 100/100 (100%) 100/100 (100%)
watchlist_main_pipeline67 67/67 (100%) 67/67 (100%) 67/67 (100%) 67/67 (100%) 67/67 (100%) 67/67 (100%)
watchlist_supp74 74/74 (100%) 65/74 (88%) 74/74 (100%) 0/74 (0%) 50/74 (68%) 0/74 (0%)

Per-group verdicts

main_rankings OK

ScoreStatusDetail
ai_scoreOK100/100 present (100%)
fundamentalOK100/100 present (100%)
technicalOK100/100 present (100%)
sentimentOK100/100 present (100%)
low_riskOK100/100 present (100%)
swing_scoreOK100/100 present (100%)

watchlist_main_pipeline OK

ScoreStatusDetail
ai_scoreOK67/67 present (100%)
fundamentalOK67/67 present (100%)
technicalOK67/67 present (100%)
sentimentOK67/67 present (100%)
low_riskOK67/67 present (100%)
swing_scoreOK67/67 present (100%)

watchlist_supp WARN

ScoreStatusDetail
ai_scoreOK74/74 present (100%)
fundamentalOK65/74 present (88%) — partial-by-design; coverage unexpectedly high — sanity-check
technicalOK74/74 present (100%)
sentimentWARN0/74 present (0%) — partial-by-design until EODHD enrichment lands
low_riskWARN50/74 present (68%) — partial-by-design until EODHD enrichment lands
swing_scoreWARN0/74 present (0%) — partial-by-design until EODHD enrichment lands

Cross-group parity (main_rankings vs watchlist_main_pipeline) FAIL

Known selection bias on low_risk: low_risk drift explained by selection bias (watchlist universe is more speculative / higher-volatility). Same-ticker max |Δ| = 0.0 across 10 shared tickers (no formula divergence). low_risk row demoted from FAIL to WARN; raw drift preserved below.

ScoreStatusDetail
ai_scoreWARNmean delta watchlist_main - main = -1.38 (main=7.672, wlm=6.291)
fundamentalWARNmean delta watchlist_main - main = -1.07 (main=7.784, wlm=6.7164)
technicalWARNmean delta watchlist_main - main = -1.49 (main=7.55, wlm=6.0627)
sentimentFAILmean delta watchlist_main - main = -2.14 (main=8.17, wlm=6.0269)
low_riskWARN KNOWN_BIASmean delta watchlist_main - main = -1.61 (main=7.224, wlm=5.6149) — explained by selection bias (drift verdict: selection_bias, confidence: medium); same-ticker max |Δ| = 0.0 on 10 shared tickers
swing_scoreWARNmean delta watchlist_main - main = -1.12 (main=6.681, wlm=5.5612)

Distribution summary

GroupFieldnmeanmedianminmaxnull
main_rankingsai_score1007.6727.67.38.70
main_rankingsfundamental1007.7847.86.09.10
main_rankingstechnical1007.557.855.39.00
main_rankingssentiment1008.178.73.810.00
main_rankingslow_risk1007.2247.23.19.90
main_rankingsswing_score1006.6816.85.07.90
watchlist_main_pipelineai_score676.2916.42.18.10
watchlist_main_pipelinefundamental676.71646.71.29.10
watchlist_main_pipelinetechnical676.06276.72.78.40
watchlist_main_pipelinesentiment676.02697.10.010.00
watchlist_main_pipelinelow_risk675.61496.20.09.40
watchlist_main_pipelineswing_score675.56125.53.57.60
watchlist_suppai_score744.64594.62.27.50
watchlist_suppfundamental655.21575.51.938.919
watchlist_supptechnical743.63513.750.010.00
watchlist_suppsentiment074
watchlist_supplow_risk505.21765.6150.09.7724
watchlist_suppswing_score074

Provenance / basis counts

main_rankings

{
  "data_source": {
    "values": {},
    "field_absent_rows": 100
  },
  "source": {
    "values": {},
    "field_absent_rows": 100
  },
  "fundamental_source": {
    "values": {},
    "field_absent_rows": 100
  },
  "ai_score_basis": {
    "values": {},
    "field_absent_rows": 100
  },
  "eodhd_fundamentals": {
    "values": {},
    "field_absent_rows": 100
  },
  "eodhd_deferred": {
    "values": {},
    "field_absent_rows": 100
  },
  "enrichment_source": {
    "values": {},
    "field_absent_rows": 100
  },
  "instrument_kind": {
    "values": {},
    "field_absent_rows": 100
  }
}

watchlist_main_pipeline

{
  "data_source": {
    "values": {
      "main_pipeline": 67
    },
    "field_absent_rows": 0
  },
  "source": {
    "values": {
      "tradingview": 38,
      "csv": 14,
      "both": 15
    },
    "field_absent_rows": 0
  },
  "fundamental_source": {
    "values": {},
    "field_absent_rows": 67
  },
  "ai_score_basis": {
    "values": {},
    "field_absent_rows": 67
  },
  "eodhd_fundamentals": {
    "values": {},
    "field_absent_rows": 67
  },
  "eodhd_deferred": {
    "values": {},
    "field_absent_rows": 67
  },
  "enrichment_source": {
    "values": {},
    "field_absent_rows": 67
  },
  "instrument_kind": {
    "values": {},
    "field_absent_rows": 67
  }
}

watchlist_supp

{
  "data_source": {
    "values": {
      "supplemental_yfinance": 74
    },
    "field_absent_rows": 0
  },
  "source": {
    "values": {
      "tradingview": 28,
      "both": 14,
      "csv": 32
    },
    "field_absent_rows": 0
  },
  "fundamental_source": {
    "values": {
      "yfinance": 65,
      "null": 9
    },
    "field_absent_rows": 0
  },
  "ai_score_basis": {
    "values": {
      "supp_composite": 65,
      "supp_technical_only": 9
    },
    "field_absent_rows": 0
  },
  "eodhd_fundamentals": {
    "values": {
      "bool:False": 74
    },
    "field_absent_rows": 0
  },
  "eodhd_deferred": {
    "values": {
      "bool:True": 65,
      "bool:False": 9
    },
    "field_absent_rows": 0
  },
  "enrichment_source": {
    "values": {
      "yfinance_fundamentals": 65,
      "yfinance_price_only": 9
    },
    "field_absent_rows": 0
  },
  "instrument_kind": {
    "values": {
      "equity": 65,
      "etf": 7,
      "crypto": 2
    },
    "field_absent_rows": 0
  }
}

Top SUPP rows: technical-only or missing fundamentals (showing 9)

TickerCompanyAIBasisFundTechSentRiskSourceEODHDDeferredKind
GUSHDirexion Daily S&P Oil & Gas Ex7.3supp_technical_only7.3supplemental_yfinanceFalseFalseetf
USOUnited States Oil Fund6.9supp_technical_only6.9supplemental_yfinanceFalseFalseetf
QTUMDefiance Quantum ETF4.4supp_technical_only4.4supplemental_yfinanceFalseFalseetf
BTC-USDBitcoin USD4.2supp_technical_only4.2supplemental_yfinanceFalseFalsecrypto
GLDSPDR Gold Shares4.1supp_technical_only4.1supplemental_yfinanceFalseFalseetf
IBITiShares Bitcoin Trust ETF4.0supp_technical_only4.0supplemental_yfinanceFalseFalseetf
SOL-USDSolana USD3.3supp_technical_only3.3supplemental_yfinanceFalseFalsecrypto
SLViShares Silver Trust2.8supp_technical_only2.8supplemental_yfinanceFalseFalseetf
MSOSAdvisorShares Pure US Cannabis2.6supp_technical_only2.6supplemental_yfinanceFalseFalseetf

Recommendations

  1. EODHD fundamentals on SUPP rows is 0 — confirm `EODHD_API_KEY` secret is populated in CI before treating SUPP fundamentals as live.
  2. Investigate score drift between main_rankings and watchlist_main_pipeline on `sentiment` (mean delta -2.14). Same scoring engine should produce near-identical means; gap suggests universe mix or weight drift.
  3. low_risk drift between main_rankings and watchlist_main_pipeline is a known selection-bias finding (watchlist universe is more speculative / higher-volatility). Surfaced as WARN, not FAIL — no action required on the scoring formula.