SignalRankRanking Signal Analysis
ReproduceGitHub

Ordering improved; absolute fit remained weak

On the same nine held-out clients, Ridge produced the strongest rank ordering: Spearman ρ was 0.165, compared with −0.096 for the baseline and 0.117 for the random forest (Figure 2). Ridge RMSE was 9.883 position units and R² was −0.005, so it did not beat a constant-mean prediction in squared-error terms (capstone notebook, cell 7 output).

Held-out Spearman correlations for baseline, Ridge, and random forest models
Figure 2. Held-out-client rank correlation. Each model is evaluated on the same client partition. Figure data

The earlier May refit yielded Ridge R² of 0.068 and Spearman ρ of 0.217. The rank correlation between absolute Ridge coefficient lists across the two cutoffs was 0.624, indicating moderate month-to-month agreement (capstone notebook, cell 7 output).

Largest standardized coefficients from the Ridge model
Figure 3. Largest standardized Ridge associations. Positive coefficients align with numerical position deterioration. Correlated measures can exchange coefficient magnitude. Figure data

Current position had the largest positive random-forest permutation contribution at 0.063, followed by position volatility at 0.020 (capstone notebook, cell 7 output). The fitted response shapes are steepest for current position, while the remaining curves are smaller or less regular (Figure 4). These are model summaries rather than treatment effects.

Partial dependence plots for five numeric ranking signals
Figure 4. Random-forest partial dependence for five leading numeric signals, averaged over the evaluation sample. The curves describe fitted model behavior and are not causal response functions. Model record

In the descriptive query analysis, entropy correlated 0.145 with numerical position deterioration, while click and impression momentum correlations were −0.072 and −0.087 (capstone notebook, cell 9 output). Because the query window overlaps the outcome, these values are hypothesis-generating and were excluded from all fitted comparisons.