CompositeTargetDiscoveryConfig field reference

Auto-generated by scripts/gen_composite_config_reference.py from mlframe.training.configs.CompositeTargetDiscoveryConfig. Do not edit by hand -- regenerate after changing the config (a doc-drift test fails CI otherwise).

The config exposes 174 fields. Each row's description is the field's own source comment.

Passing the config

train_mlframe_models_suite(..., composite_target_discovery_config=...) accepts EITHER a CompositeTargetDiscoveryConfig instance OR a plain dict of field overrides. The dict is converted via _ensure_config, which is STRICT: an unknown key (e.g. a typo enabledd=True) raises ValueError instead of being silently dropped, so misspelled knobs fail loud. None uses all defaults.

# instance form
from mlframe.training.configs import CompositeTargetDiscoveryConfig
cfg = CompositeTargetDiscoveryConfig(enabled=True, auto_base_top_k=5)
train_mlframe_models_suite(..., composite_target_discovery_config=cfg)

# equivalent dict form (validated + type-checked the same way)
train_mlframe_models_suite(..., composite_target_discovery_config={"enabled": True, "auto_base_top_k": 5})

MLFRAME_DISABLE_COMPOSITE=1 forces composite discovery off regardless of the config.

Fields

Field	Type	Default	Description
enabled	bool	False
time_column	Optional[str]	None	Optional chronological-order column (timestamp / monotone index). When set, discovery SORTS the MI-screening sample by it so the tiny-model CV is a forward-walk (TimeSeriesSplit) not a shuffled K-fold -- the canonical non-monotone lag(y) base defeated the legacy base-monotonicity heuristic, so the screen leaked future->past. None keeps the legacy auto-detection.
time_series_transforms_enabled	bool	False	Opt-in: add the 3 chronological-order transforms (ewma_residual / rolling_quantile_ratio / frac_diff) to the candidate set. They need the screen in time order, so set time_column too. Default OFF -- on a shuffled frame they model a meaningless row sequence.
group_column	Optional[str]	None	Per-entity group key. On a group-sequential target (well log per well_id, panel per entity) the causal-base engineer builds strictly-causal per-group lag / rolling bases of y; group_column names that key. None => no grouped causal engineering (unless engineer_causal_group_column overrides). Also consumed by linear_residual_grouped.
engineer_causal_bases	bool	True	Grouped causal base engineering (default ON): materialise strictly-causal per-group lag_k / trailing-mean / expanding-mean bases of the target, ordered WITHIN each group by time_column (a monotone within-group order such as MD; None => caller-guaranteed per-group frame row order). These bases are causal by construction (shift>=1 within group), so their additive inverse y = T_hat + y_prev stays in-range on unseen groups -- the single most valuable base class for autoregressive sequential targets. No-op unless a group key is available. Set False for legacy replay.
engineer_causal_group_column	Optional[str]	None
engineer_causal_lags	Tuple[int, ...]	(1,)
engineer_causal_trailing_windows	Tuple[int, ...]	(3,)
engineer_causal_ops	Tuple[str, ...]	('lag', 'trailing_mean', 'expanding_mean')
engineer_causal_first_fill	str	'group_first'
causal_base_gate_exempt	bool	True	Exempt strictly-causal bases (grouped-causal engineered __gcausal_* or a named {y}_prev lag) from the near-copy-of-y and structural-fragility gates. Those gates drop bases whose additive inverse extrapolates on unseen groups, but a causal base re-injects a REAL per-row previous value so it stays in-range -- the gates' failure mode does not apply, and on a strong-AR target the causal lag is the single best base. Provenance-only exemption (never a marginal-correlation match), so a contemporaneous near-copy of y is still dropped. Default ON.
composite_achievable_ceiling_precheck	bool	True	Measured achievable-ceiling precheck: before running discovery, MEASURE raw-y / lag_predict / optimistic-composite RMSE on a bounded subsample and SKIP (deploy the failsafe) when the best achievable composite cannot beat min(raw, lag) by ..._margin. This is the AUTHORITATIVE skip signal -- orthogonal to the legacy extreme_ar_group_aware_skip autocorr fast-path, which must never disable the measured ceiling or the lag_predict failsafe (prod footgun: setting that flag False disabled the entire skip). Records composite_precheck_verdict_. Default ON.
composite_achievable_ceiling_margin	float	0.02
composite_achievable_ceiling_sample_n	int	30000
composite_achievable_ceiling_holdout_frac	float	0.3
composite_achievable_ceiling_strong_floor_frac	float	0.5
near_copy_increment_learnability_precheck	bool	True	Near-copy increment-learnability precheck: before dropping a near-copy-of-y base (|corr|>base_max_abs_corr_with_y), measure whether the residual y-base retains LEARNABLE signal from the other features (permutation-null-corrected bin-MI). If it does, the composite genuinely helps -> keep the base; else drop as before. Distinguishes a strong legitimate base from one that adds nothing over feeding base as a plain feature. Default ON. Causal-provenance bases stay exempt regardless (see causal_base_gate_exempt).
near_copy_increment_learnability_mi_threshold	float	0.05
near_copy_precheck_max_sample	int	5000
moe_gate_enabled	bool	True	MoE selection gate at the composite deploy boundary (run_composite_post_processing): route the shipped prediction among {composite, raw, lag} per group with a never-worse-than-lag guarantee. Default ON; no-op without lag / groups.
moe_gate_shrink_rtol	float	0.0
moe_gate_min_group_rows	int	1
base_ranking_criterion	str	'mi'	Base-candidate ranking criterion. "mi" (default) ranks by pairwise MI(base, y); "mrmr" reranks by min-redundancy-max-relevance so a top-K of near-duplicate strong bases is diversified (score = MI(base,y) - beta * mean redundancy vs already-picked). Default "mi" keeps discovery byte-identical; "mrmr" is opt-in pending a broad benchmark before flipping the default.
base_ranking_mrmr_beta	float	1.0
ar1_failsafe_val_crosscheck	bool	True	AR(1) lag-failsafe val cross-check. The ensemble deploys zero-param lag_predict when its group-K-fold OOF RMSE ties the best trained component. That OOF underestimates the full-data model, so a tie can ship lag over a model that generalises far better (prod: lag test 12.29 vs trained 9.31 at an OOF tie of 13.64). When a trained component beats lag on the group-disjoint VAL split (same honest regime as test) by more than lag_predict_failsafe_tolerance, veto the failsafe and deploy the trained component. Default ON (corrective); conservative -- only ever prevents a lag deployment in favour of a val-confirmed-better trained model.
ood_lag_routing_enabled	bool	True	Per-row OOD-lag routing on the deployed model. After the val cross-check keeps the trained model (it wins overall), some unseen groups still sit OUTSIDE the train target range, where the model can only extrapolate/clamp while lag_predict is exact. Route those rows (lag outside the train target range) to lag -- but only when it strictly improves the honest group-disjoint val RMSE. The rule uses the fixed train range (not group ids), so it transfers to unseen test groups. Default ON; ood_lag_router_margin_frac optionally widens the train range before routing.
ood_lag_router_margin_frac	float	0.0
volatility_lag_routing_enabled	bool	True	Per-row VOLATILITY-lag routing on the deployed model. On a strong-AR target the groups where lag beats the trained model are IN-range but locally SMOOTH (consecutive target values barely move within the well), so lag_predict is near-perfect there. Route rows whose MD-local target volatility is low to lag, only when it improves the honest group-disjoint val RMSE. Ordering is EXPLICIT by time_column (MD) -- requires group_column + time_column on the frame; no order column -> no routing (never a frame-row-order guess). Default ON; safe no-op when it does not help.
emit_composite_value_report	bool	True	Emit the composite-target VALUE report (per-group did-it-help / hurt / worse-than-lag breakdown + net weighted lift).
stability_group_aware	bool	True	Spec stability selection resamples whole GROUPS (leave-wells-out) rather than rows whenever a group key resolves, so a spec that is stable only because it memorised per-group levels is demoted (row resampling puts a group's rows in both the replicate and its complement, hiding the overfit). Default ON; no group key => bit-identical row resampling.
region_adaptive_enabled	bool	False	Opt-in discovery steps (wired via discovery._opt_in_steps). ALL default OFF -> a flag-gated no-op that leaves the discovered specs byte-identical to the legacy flow. Each enables one standalone helper over the already-kept single-base specs at the END of fit: - region_adaptive: per-region best-transform selection routed by frozen quantile edges of each kept base (_region_adaptive.fit_region_adaptive); results surface on CompositeTargetDiscovery.region_adaptive_specs_. - interaction_base_discovery: surface a OP b synthetic interaction bases whose MI beats both marginals (_interaction_bases.discover_interaction_bases); results surface on interaction_bases_ / interaction_base_records_. - auto_chain_discovery: compose every residual x tail-unary chain and keep those that beat both single stages on held-out y-scale RMSE (_auto_chain.discover_chains); winning chains are APPENDED to specs_ (their composed Transform is registered so iter_transform resolves it) and also surface on auto_chains_. Default ON: these opt-in discovery steps each have test-confirmed business value (region-adaptive +47% OOS on region-dependent data, interaction-base +90% on pure-interaction targets, auto-chaining beats both single stages 8/8 seeds) and are no-harm by construction -- region-adaptive + interaction surface as informational artefacts (region_adaptive_specs_ / interaction_bases_) WITHOUT altering the selected specs_, and auto-chaining only APPENDS chains that already beat both single stages on held-out RMSE (empty when none win). All three are compute-bounded by their caps below. Set False to skip the extra discovery passes for the fastest possible fit. Default OFF: region-adaptive is a committed-but-REJECTED research prototype (see discovery/_region_adaptive.py module docstring). On a large prod run it burned ~7.5 min fitting specs that collapsed at deploy. Keep the opt-in flag for benches; do not run it by default.
region_adaptive_k	int	4
interaction_base_discovery_enabled	bool	True
interaction_base_top_k	int	4
interaction_base_max_pairs	int	3
auto_chain_discovery_enabled	bool	True
auto_chain_top_k	int	2
base_candidates	Union[List[str], str]	'auto'	Base candidate selection. - "auto": rank all numeric features by structural MI gain (MI(y - LinearFit(x), X \ {x}) on train) and take the top auto_base_top_k after applying forbidden-pattern + corr + ptp filters. - list[str]: explicit list of column names. Still passes through the forbidden / corr / ptp guards; columns failing the guard are skipped with a warning.
auto_base_top_k	int	3
base_max_abs_corr_with_y	float	0.9995	Reject a base candidate whose |corr(base, y)| on the screening sample exceeds this -- such a base is a near-COPY of y (e.g. a kalman/particle-filter posterior or a lag that reproduces y up to noise). The residual T = y - alphabase is then ~noise and the inverse y = T_hat + alphabase is carried ENTIRELY by base, so any base distribution shift on unseen groups blows the inversion up (the prod TVT collapse: every composite built on pf_tvt_post_* / TVT_prev, |corr(base,base)|=1.000 with each other and ~1.0 with y). Default 0.9995 keeps legitimate strong-AR lags (|corr|~0.99) but excludes literal copies. 1.0 / None disables the filter (legacy behaviour).
auto_base_structural_boost_corr_gate	float	0.98	Do NOT apply the structural "near-affine predictor of y -> prime linear_residual base" boost to a column that is itself a near-copy of y (|corr(col, y)| above this) -- boosting promotes exactly the fragile leaked columns to the top of the base ranking. Default 0.98. 1.0 disables the gate.
dominant_features_hint	Optional[List[str]]	None	Priority-base hint: features treated as base candidates regardless of pairwise MI(y, x) ranking. _auto_base puts these first (given order) and fills the rest up to auto_base_top_k with the top MI-ranked features. Exists because pairwise MI is fooled by features with global trend but no structural residual (e.g. spatial coords on geo-trended targets); BaselineDiagnostics ablation (drop-feature RMSE delta) is far more reliable for "what drives prediction" and train_mlframe_models_suite populates the hint from it automatically. Hint features still pass the standard filters (forbidden_pattern / non_numeric / constant / corr_threshold); failures are logged + dropped.
transforms	List[str]	['diff', 'additive_residual', 'median_residual', 'ratio', 'logratio', 'linear_residual', 'linear_residual_robust', 'quantile_residual', 'monotonic_residual', 'y_quantile_clip', 'cbrt_y', 'log_y', 'yeo_johnson_y', 'quantile_normal_y', 'chain_linres_cbrt', 'chain_linres_yj', 'chain_monres_cbrt', 'chain_monres_yj', 'asinh_residual', 'centered_ratio', 'polynomial_residual_deg2', 'rank_residual', 'smoothing_spline_residual', 'reciprocal_residual']	Transform names from the registry (mlframe.training.composite). The default extends beyond the original 4 to include the SINGLE-BASE, DROP-IN transforms: - quantile_residual -- conditional-on-bin centering + scaling. - monotonic_residual -- monotone PCHIP spline residual. These accept the standard (y, base) signature and need no special orchestration -- discovery evaluates them like linear_residual. NOT in default list (need orchestration): linear_residual_multi (multi-column base selection via forward stepwise; single-base mode == linear_residual), linear_residual_grouped (group_column extraction + groups kwarg), and the four chronological-order / recurrent transforms ewma_residual / rolling_quantile_ratio / frac_diff / volatility_normalized_residual (now reachable via time_series_transforms_enabled + time_column). asinh_residual_multi and linear_residual_multi_robust are also excluded -- both are multi-base transforms needing the same forward-stepwise orchestration as linear_residual_multi. All accessible via explicit CompositeTargetEstimator(...) and ship their own tests. REJECTED default-list addition (measured, not just untried): box_cox_y / seasonal_residual / nadaraya_watson_residual / gaussian_copula_residual are single-base drop-in candidates with the standard signature (no orchestration blocker) -- trying them in ISOLATION against a deliberately narrow 5-transform list finds real lift (gaussian_copula_residual beats monotonic_residual by 1.35% honest y-RMSE on a lognormal/distorted-marginal DGP), but against the REAL (already 29-transform) default pool the measured full-pool A/B shows 0.00% RMSE improvement (the existing pool already covers this DGP class via quantile_normal_y) at +1433% / +605.7% wall-clock overhead on the positive and negative-control DGPs respectively (discovery/_benchmarks/ bench_widened_default_transforms.py). All four remain fully available via explicit transforms=[...] (own tests, own registry entries) -- only default-pool membership is rejected, on measured cost/benefit, not on principle.
multi_base_enabled	bool	True	Multi-base forward-stepwise auto-promotion: after single-base discovery + raw-y gate + rerank, each kept linear_residual spec greedily ADDS bases from the auto-base pool, upgrading to linear_residual_multi when the marginal CV-RMSE gain clears multi_base_min_marginal_rmse_gain. Default ON (benchmark: +83% geo-mean on additive DGPs, no-harm on single-dominant / collinear pools; benchmarks/composite_multi_base_benchmark.py); set multi_base_enabled=False for highly-correlated pools / very small n_train.
multi_base_max_k	int	3
multi_base_min_marginal_rmse_gain	float	0.005	Marginal relative CV-RMSE gain a candidate base must clear to be ADDED. Default 0.005 (= 0.5%). The old 2% gate stopped early and left genuinely-helpful weak orthogonal bases out: on a disjoint honest holdout (discovery never sees it), 0.005 beats 0.02 in 20/20 non-tied seeds across additive multi-base DGPs (16-19% holdout-RMSE win) with ZERO regression on the single-dominant + noise-decoy DGP (the paired-fold majority gate in forward_stepwise_multi_base independently rejects noise bases regardless of this relative threshold). Bench: discovery/_benchmarks/bench_multibase_min_marginal_gain.py. Set to 0.02 for the legacy conservative gate.
multi_base_skip_when_pool_corr_above	float	0.98	Auto-skip multi-base promotion when the base pool's mean pairwise |corr| exceeds this: stacking near-identical bases adds no orthogonal signal but doubles the inverse's base-shift amplification on unseen groups. Default 0.98 (1.0 disables). Detects the "highly-correlated pool" case the multi_base_enabled docstring says to set False for, instead of relying on the user to do it.
cv_selector_mode	Literal['mean', 'mean_minus_std', 'median_minus_mad', 't_lcb', 'quantile']	'mean'	Robust CV-selector: argmin(mean(fold_rmses)) silently rewards lucky candidates whose mean wins by less than the per-fold std. cv_selector_mode != "mean" augments the per-fold scores with a dispersion penalty before the argmin (stable mediocre beats unstable lucky); see _cv_aggregation.aggregate_fold_scores. Default "mean" is bit-identical.
cv_selector_alpha	float	1.0
cv_selector_confidence	float	0.9
cv_selector_quantile_level	float	0.9
cv_persist_fold_scores	bool	False
use_stacked_discovery	bool	False	Stacked 2-pass composite discovery. When True the suite calls CompositeTargetDiscovery.fit_stacked instead of plain fit. Pass 1 is the normal discovery; for the top stacked_max_pass1_specs specs we compute OOF predictions on the train rows, append them as new feature columns, and re-run discovery on the augmented feature set. Pass-2 specs may absorb residual-of-residual structure that the first pass missed (e.g. y = f(x_a) + g(x_b): pass 1 takes f(x_a), pass 2 finds g(x_b) on the leftover). Default False so the path is opt-in until measured on real data; switch to True after biz_val on your target. Default-flip eval measured on profiling/bench_stacked_discovery_default_flip.py on a TRUE residual-of-residual synthetic y = 1.5*x_a + 3*sin(x_b) + noise (pass-1 linear_residual captures the x_a slope; pass-2 should find the non-linear sin-on-x_b structure that pass-1 cannot represent). Even on this structurally favourable problem the feature-stack pass discovered the SAME specs as single-pass and yielded NO holdout-RMSE improvement on the measured n=4000. Verdict: keep default False. Re-run the benchmark on a problem closer to your production target before opting in.
stacked_n_oof_folds	int	3
stacked_max_pass1_specs	int	3
use_stacked_discovery_residual	bool	False	Residual-target stacked discovery (alternative to use_stacked_discovery). When True the suite calls CompositeTargetDiscovery.fit_stacked_on_residual instead of fit_stacked. Pass 1 specs collectively predict pass1_pred; y - pass1_pred becomes the new target for pass 2 discovery. Mathematically the more direct path for residual-of-residual structure when the feature-stack route blocks at the discovery gate (pass 1 OOF prediction is too correlated with target so pass 2 mi_gain looks marginal). Mutual exclusion: if BOTH use_stacked_discovery and use_stacked_discovery_residual are True, residual wins (per the docstring's "more direct" recommendation) and a warning is logged. Default False; opt-in after biz_val on your target.
stacked_residual_aggregation	str	'mean'	"mean" averages OOF predictions across pass-1 specs (robust to a single overfit spec); "first" uses the best pass-1 spec only. Forwarded to fit_stacked_on_residual(residual_aggregation=...).
stacked_residual_max_pass1_specs_to_aggregate	int	3	Cap on how many top-ranked pass-1 specs (best tiny CV-RMSE first) contribute their OOF predictions to the residual-target aggregate in fit_stacked_on_residual. Previously that path aggregated EVERY pass-1 spec (a no-op ranked[:max(1,len(ranked))] slice), so weak tail specs polluted the mean aggregate and the leftover residual. Default 3 matches the feature-stack sibling (stacked_max_pass1_specs); <=0 restores the historical aggregate-all behaviour. Immaterial when stacked_residual_aggregation="first" (single best spec only).
discovery_n_jobs	int	0	Parallel evaluation of (base, transform) candidates in CompositeTargetDiscovery.fit via joblib(threading). 0 = auto (min(len(work_items), cpu_count)); 1 = serial; >1 = explicit. Phase A is ~20% of fit time (Phase B / _tiny_model_rerank dominates), but the per-candidate MI work is numpy/numba (GIL-released) so the threading parallelism is near-free and bit-equivalent to serial (tests/training/test_composite_discovery_parallel.py). Default 0 (auto) alongside the auto tiny_rerank_n_jobs for full Phase A + B parallelism out of the box.
composite_skip_when_raw_dominates_ratio	float	0.0	Skip the entire composite-target training block when the raw model already dominates the dummy-baseline ceiling. The discovery's raw-y baseline RMSE / y_std ratio is a cheap proxy for "raw is already near-perfect": when ratio < composite_skip_when_raw_dominates_ratio, composite training is unlikely to add measurable lift on the BASELINE model. Set 0.0 to never skip. Default is 0.0 (always run discovery). Prior defaults 0.02 / 0.03 were tuned against the Ridge / CB / XGB / LGB zoo where "raw R^2 already > 0.99" reliably meant "composite has no headroom". That assumption silently breaks for model-mix suites containing nonlinear / mis-configured downstream models: e.g. an Identity- activation MLP can extrapolate to ~-17 sigma on the random-group test split while Ridge nails R^2=1.00 on the same data. A composite like y - top_ar_feature would have given the MLP a residual target near zero and saved it. Because the gate fires off the raw-Ridge baseline only, it skipped composite discovery and the MLP collapsed. The 15+ min compute cost is the price for not gambling on this assumption.
composite_skip_when_ablation_delta_pct	float	0.0	Complementary skip signal using BaselineDiagnostics' ablation delta%. When the top-ranked feature's drop causes ablation RMSE to balloon by more than this fraction, the raw model is essentially auto-regressive on that one feature. Set 0.0 to never skip; positive value re-enables the heuristic. Default is 0.0 for the same model-mix reason as the ratio gate above: "one feature explains everything for Ridge" doesn't mean "all downstream models will be fine". An MLP that extrapolates badly on unseen wells benefits enormously from the composite y - top_ar_feature even when Ridge doesn't need it.
skip_wrap_pass_predict	bool	True	Skip the wrap-pass y-scale predict() calls per composite entry per split (train+val+test). Wide model zoos x multi-million-row frames see 5-15 min wall time on this block (MLP predict is the worst at 15-30s each). When True the wrapper still installs CompositeTargetEstimator (so downstream consumers get y-scale predict output) but the y-scale metadata block stays empty -- recover it on demand via _phase_composite_post.recover_composite_y_scale_metrics. Safe because T-scale metrics from the per-target phase already cover the watchdog invariants (T-MAE == y-MAE for additive-invertible transforms).
enable_wrap_pass_watchdog	bool	True	Disable the runtime watchdog when its extra wrapper.predict + inner.predict per (entry, split) costs more than the rare-bug catch is worth. Measured overhead on n=200k, 2 composites x 5 models = 30 (entry, split) pairs: +72.6% wall time (median 0.353s ON vs 0.204s OFF, see profiling/bench_pack_g_watchdog_overhead.py). On 4M-row frames with MLP inners (predict cost 1-5s) the overhead scales linearly, adding 1-5 min per training pass. Production environments that have verified the wrapper math in staging can disable here. Default True (catch silent bugs).
mi_sample_n	Optional[int]	100000	MI screening. Sample to keep the diagnostic under one minute on 4M-row datasets; mi_sample_n=None uses full train. Default lowered 200_000 -> 100_000: prod log analysis showed 5.3 min discovery dominated by 200k MI compute. Halving to 100k gives ~2x speedup with adequate sample size for typical regression / balanced-binary scenarios. HONEST CAVEAT: 100k may be insufficient for two regimes: (a) Imbalanced classification with minority-class rate < 5% -- 100k * 5% = 5000 positives per 20-bin MI is borderline; consider mi_sample_n=200_000 if you see spec drift between runs. (b) Heavy-tail regression where the tail carries the signal -- 100k may under-sample the extremes; mi_sample_n=None (use full train) eliminates the risk at the cost of 20x compute. Final raw_baseline_rmse gate AND tiny_model_rerank use FULL train_idx so final spec precision is unaffected by mi_sample_n.
top_k_after_mi	int	32	Top-K trim after the MI gate. Generous default so pure-lag composites (y - alpha*lag_y) -- which have structurally NEGATIVE mi_gain because their residual is noise -- aren't sorted to the bottom of the (mi_gain, name) ordering and truncated out. With 4 unary + ~5 bivariate transforms across 3 bases the max candidate count is ~19; 32 is "keep them all".
eps_mi_gain	float	-10.0	Pre-filter threshold for mi_gain = MI(T, X_no_base) - MI(y, X_no_base). Defaults walked +0.01 -> -0.5 -> -10.0 across a real regression incident: pure-lag composite T = y - y_prev = noise has MI(T, X_no_base) ~ 0 while MI(y, X_no_base) can be large (0.5-1.5 for AR-1 datasets where lag explains nearly everything), so mi_gain is structurally very negative for the correct composite. -0.5 left the MLP-saving composite still below the gate (mi_y > 0.5). -10.0 effectively disables the MI pre-filter; broken composites (e.g. logratio on negative y) are still caught by the transform's own domain_check and is_degenerate flag earlier in the pipeline. The downstream raw-y baseline gate (Phase B) is the real "is this composite useful" decision -- but that gate is now off by default for the same model-mix safety reason (see require_beats_raw_baseline).
mi_n_neighbors	int	3
mi_estimator	str	'bin'	MI estimator. "knn" uses the Kraskov estimator (sklearn default, accurate but slow on n>10k); "bin" uses a quantile-binning estimator (5-10x faster, biased low on heavy-tail). Default flipped from "knn" -> "bin" after a statistical review noted that: 1. kNN is biased high on heavy-tail / mixed-density distributions and the bias scales DIFFERENTLY for raw y (potentially fat- tailed) vs T = transform(y, base) (sub-Gaussian after linear_residual). That asymmetric bias inflates apparent mi_gain even when the true gain is zero -- which matches the production failure mode where MI passes but RMSE doesn't. 2. bin (quantile) estimator is approximately bias-free under monotone transforms because the bin edges follow the transformed distribution -- exactly what the registry's transforms (diff/ratio/logratio/linear_residual) do. 3. bin is 10x faster on the 200K-row screening sample we typically run. Set to "knn" explicitly for non-monotone transforms or when n < 5*nbins (bin floor needs ~80 rows at default nbins=16).
mi_nbins	int	16
mi_aggregation	str	'mean'	Aggregation across feature columns when comparing MI(T, X_no_base) against MI(y, X_no_base). Legacy "sum" is biased (overcounts shared information when X is correlated, and the over-count differs between numerator and denominator). Mean is invariant to feature count and is the cleaner default; users on existing benchmarks can pin "sum" for reproducibility.
mi_sample_strategy	str	'stratified_quantile'	MI sampling strategy. "stratified_quantile" (default) bins y into mi_n_strata quantile bins and samples equally from each, guaranteeing per-bin coverage so the rare-tail rows that carry most of the signal on heavy-tail targets (financial returns, fraud scores, queue lengths) are never dropped by an unlucky uniform draw. "random" is the cheaper legacy uniform sample -- set it explicitly only when you have verified the target is light-tailed and the per-stratum draw buys nothing. Default flipped "random" -> "stratified_quantile": under "random" the heavy-tail mi_n_strata auto-boost in discovery (skew>2/kurt>5 -> mi_n_strata_heavy_tail) was a dead no-op because the uniform draw never consulted the strata. Stratified sampling is what makes that boost (and the mi_n_strata knob at all) actually steer which rows the MI screen sees; the per-stratum quotas raise tail coverage materially on skewed y without changing the screen on already-uniform y.
mi_n_strata	int	10
transform_waic_validation_enabled	bool	True	Phase B: tiny-model rerank. After MI screening narrows to top-K, train a tiny model (LightGBM or per-family) per surviving candidate and re-rank by CV-RMSE measured on the y-scale (after inverse). This is the "true objective" -- MI is a proxy. Skip by setting screening = "mi". Default raised from "mi" -> "hybrid" after a production case where MI-only screening kept composites whose bases (spatial coordinates) had trivial pairwise MI(y, x) but zero structural signal for residual learning. The MI-gain test passed barely (mi_gain ~ 0.01) but the resulting models had WORSE OOF RMSE than raw-y because subtracting the base added noise to the target. Phase B's CV-RMSE-on-y-scale catches this directly. Cost: ~0.5-2 min per target on a 4M-row dataset. Information-criterion validation of transform choice. When True, the discovery loop MAY consult a WAIC/LOO-style score (the expected pointwise out-of-fold predictive density of the tiny-CV residuals, penalised for effective across-fold complexity -- see discovery._eval_waic) as an ADDITIONAL ranking signal alongside MI-gain, never replacing it. It separates two transforms that MI-gain ties when one genuinely generalises and the other overfits the screen. Default ON: the tiny-rerank folds the per-transform WAIC into its ordering as a tie-break, re-ranking only specs whose tiny-CV RMSE falls within a relative noise band (so it never overrides a real RMSE difference) and only when every tied spec has a valid score -- a strict refinement that picks the generalising transform over an overfit one RMSE alone cannot tell apart. Costs one extra cheap K-fold OOF pass per surviving candidate on the small screen sample. Set False to restore the pre-tie-break (RMSE+name) ordering.
transform_waic_n_folds	int	4
screening	str	'hybrid'
tiny_model_n_estimators	int	60
tiny_model_num_leaves	int	15
tiny_model_learning_rate	float	0.1
tiny_model_cv_folds	int	3
tiny_model_sample_n	int	20000
top_m_after_tiny	int	10
tiny_model_n_jobs	int	0
tiny_rerank_n_jobs	int	0	Parallelise the per-spec rerank loop in _tiny_model_rerank. Each spec runs _tiny_cv_rmse_y_scale_multiseed per family — typically the dominant wall-time slice of Phase B on subsample=200k+ configs. Threads share base/x_matrix arrays via backend="threading"; LightGBM and the inner CV release the GIL. Set to 0 = auto (min(len(kept_specs)*len(families), cpu_count)). Default 0 (auto): Phase B dominates discovery wall-time and the rerank threads share base/x_matrix arrays (no copy) while LightGBM + inner CV release the GIL, so threading parallelism is near-free and bit-equivalent to serial (test_composite_discovery_parallel.py). Set 1 to force serial.
deterministic_screening_models	bool	False	Force deterministic mode on the tiny models built INSIDE Phase B (_build_tiny_model). When True, injects the well-known determinism flags per family: - LightGBM: deterministic=True, force_row_wise=True - XGBoost: explicit tree_method="hist", predictor="auto" - CatBoost: boosting_type="Plain" (Plain is deterministic; Ordered is the non-deterministic default) Bit-exact run-to-run reproducibility on the rerank stage at a 5-10% per-fit cost. Default OFF. Scope: this controls the tiny models we BUILD ourselves for rerank. The actual composite-target inner training (the K LightGBM/XGB models that train the per-spec composite targets) is configured via hyperparams_config, not this flag.
tiny_screening_models	str	'per_family'	Per-family screening: instead of one tiny LightGBM, train a tiny model of each family in the user's mlframe_models list (cb / lgb / xgb / linear). Different families pick different top features on the same data, so a candidate that wins for one family may lose for another. Aggregation via tiny_consensus: - "single_lgbm" (legacy): one LightGBM, fastest but model-agnostic proxy gates lie for downstream linear / neural models. - "per_family" (default): train one tiny model per entry in tiny_screening_families and aggregate by tiny_consensus ("union": top-M from each family; "borda": Borda-count rank). Default families ("lightgbm", "linear") cover the two distinct downstream regimes: tree boosters (LGBM proxy) and linear / neural models (Ridge proxy). A composite useful only for one of the two still survives via the union aggregation. The 2x screening compute is the price for model-mix safety (observed in prod: single-LGBM proxy rejected the only composite that would have saved the downstream Identity-MLP).
tiny_screening_families	Tuple[str, ...]	('lightgbm', 'linear')
tiny_consensus	str	'union'
require_beats_raw_baseline	bool	False	Raw-y baseline gate. During tiny-model rerank, also train a tiny model on the RAW target (no composite transform) on the same screening sample / folds and use its CV-RMSE as a hard floor: any composite whose CV-RMSE >= raw_baseline * tolerance is rejected as a regression. Catches the "wrong base" case where MI-gain passes but the resulting target is actually harder to predict (e.g. subtracting a spatial coordinate that has global trend with y but no structural residual signal). Tolerance > 1.0 allows composites that are slightly worse on the screening sample but might still help in the cross-target ensemble. 1.0 = strict (composite MUST beat raw); 1.02 = within 2% of raw on tiny LGBM. Default flipped True -> False because the gate uses a tiny LGBM as a model-agnostic proxy, and that proxy LIES for downstream linear models. The pure-lag incident: composite y - alpha*lag_y has residual T = noise, so any downstream model trained on T predicts T_hat ~ 0 and the composite estimator returns y_hat = lag_y -- essentially predict y by its lag. Tiny LGBM on this composite gets CV-RMSE = std(noise) which can be 5-10x worse than tiny LGBM on raw y (which directly fits y from features including lag_y). The gate therefore rejects the composite, even though for an Identity-MLP downstream model the composite is the ONLY thing preventing OOD extrapolation collapse on unseen-groups test splits. Set True to re-enable when running tree-only zoos and you want to cut the per-target training compute on composites the boosters won't benefit from.
raw_baseline_tolerance	float	1.02
raw_baseline_per_bin_tolerance	float	1.1	Regime-aware gate. In addition to the global mean RMSE comparison, also check per-quintile-of-base RMSE: a spec is rejected if its tiny CV-RMSE in any quintile exceeds raw_baseline-in-that-quintile by raw_baseline_per_bin_tolerance. This catches "two-regime" failure modes where logratio is correct on multiplicative-regime rows but actively wrong on additive-regime rows; mean RMSE hides this and the spec ships even though it's miscalibrated half the time. Tolerance defaults looser than the global gate (1.10 vs 1.02) because per-bin estimates have higher variance on small screening samples. per_bin_n_bins=0 disables the check. Default flipped 5 -> 0 (off) alongside require_beats_raw_baseline since the per-bin gate is a refinement of the same tiny-LGBM- proxy logic and inherits the same model-mix safety problem. Set > 0 to re-enable when running tree-only zoos.
per_bin_n_bins	int	0
force_inject_diff_on_top_ablation_pct	float	0.0	Force-inject the (top_ablation_feature, diff) and (top_ablation_feature, additive_residual) specs into the discovery output when the top-feature ablation delta% exceeds this threshold, regardless of gate / MI / top-K filtering. This is the "AR-target safety net" -- ensures the simplest possible residualisation composite (y - top_AR_feature) is always tried for AR-dominated targets, since that's the composite that bounds linear-stack MLP extrapolation damage on group-aware splits. Default 0.0 (DISABLED). The current gate flips (eps_mi_gain=-10.0, require_beats_raw_baseline=False, top_k_after_mi=32) + the additive_residual transform already produce the AR-diff spec organically on AR-style data; this flag is an explicit insurance for paranoid configurations where a user re-enables the gates. Enable by setting > 0.0 (typical threshold 50.0 to match hint_strength_threshold_pct). Full implementation pending plumbing of per-feature ablation pct into discovery internals.
tiny_model_n_seed_repeats	int	3	Median-of-seeds gate. Tiny CV-RMSE with 3 folds is variance-prone (one unlucky split can drag the mean). Optionally repeat the K-fold split with multiple seeds and take the MEDIAN across (folds × seeds) for both raw-y and per-spec CV-RMSE. The gate then compares median composite vs median raw, which is more stable than the mean. Compute scales linearly. Default 3 (raised from 1): a prod run showed the previously-winning linres-lag1 spec getting displaced by monres-Y chain variants because the single-seed rerank had high variance; with n_seed_repeats=3 the rerank picks the spec that wins on the MEDIAN of 3 splits instead of one unlucky split. 3x compute on screening sample is cheap (sub-minute) vs losing the actual winning spec.
enable_multiseed_early_stop	bool	True	Sequential early-stop across the multiseed rerank loop. The final per-candidate score is median(seed_scores) over the finite (non-degenerate) seeds. Because RMSE-like scores are non-negative, inserting hypothetical zeros for every remaining un-run seed can only ever DECREASE (or leave unchanged) the eventual median -- so median(observed_finite + [0]*remaining) is a rigorous LOWER BOUND on the true multiseed median, no distributional assumption needed. Once that lower bound already clears the raw-baseline rejection threshold, no possible outcome of the remaining seeds can pull the true median back under the threshold, so the candidate is doomed and further seeds are skipped. This never touches the ACCEPT path (an accepted spec's exact score is still needed for cross-candidate ranking), so early-stopped candidates are always rejects -- the kept-spec set and the ranking among survivors are unaffected. Only sound for cv_selector_mode == "mean" (see _screening_tiny._seed_median_lower_bound); every other selector mode is a strict no-op (all seeds always run, bit-identical to the flag being off). Default True: proven no accuracy cost (identical kept-spec sets/rankings vs all-seeds) in tests/training/composite/discovery/test_multiseed_early_stop.py; cuts wasted seed-fits on candidates the raw-baseline gate rejects anyway.
use_wilcoxon_gate	bool	False	Paired one-sided Wilcoxon signed-rank test on per-fold-pair RMSE differences (composite minus raw). Replaces the static raw_baseline * tolerance threshold with a non-parametric significance test: spec is rejected unless the median of per-fold differences is significantly negative (composite < raw) at level gate_alpha. Scipy must be available; falls back to threshold-only gate if not. Cost: requires per-fold RMSE pairs from BOTH composite and raw runs, which we already collect when tiny_model_n_seed_repeats > 1. With n_seed_repeats=1 the test has 3 fold pairs total -- the test will be too low-power to reject anything except egregious cases. Recommended: n_seed_repeats=5 for the test to have meaningful power.
gate_alpha	float	0.05
detect_linear_residual_alpha_drift	bool	True	Detect alpha-drift in linear_residual. Fit alpha on first half of train and on second half; compare via Chow-style |Δα| / pooled SE. If the absolute z-score exceeds alpha_drift_z_threshold (default 3.0), the linear_residual spec for that base is flagged in metadata with reason alpha_drift_detected and (optionally) rejected. Catches concept-drift / non-stationary y/base relationships that LR's point-estimate alpha silently degrades on at test.
alpha_drift_z_threshold	float	3.0
alpha_drift_min_effect_size	float	0.01	Effect-size floor that the slope drift must ALSO clear before a spec is rejected. The z-test alone is sample-size sensitive: SE(alpha) shrinks ~1/sqrt(n), so at multi-million-row train a practically negligible slope shift gives z >> 3 and cascade-drops every spec on the base (prod TVT 4.1M rows: 23 of 62 candidates). effect_size is the half-to-half slope change scaled into y-units as a fraction of y_std: \|a1-a2\| * std(base) / std(y). A spec is dropped only when z > threshold AND effect_size >= this floor, so a drift must be both statistically detectable and practically meaningful. Default 0.01 (the inverse must move predictions by >=1% of y_std). Set 0.0 for the legacy z-only behaviour.
reject_on_alpha_drift	bool	True	When True, drop linear_residual specs that fail the drift check; when False, keep them but log a warning + record in metadata. Default True ("enable corrective mechanisms by default"): a drifting alpha means the residual T = y - alphabase is fit on a non-stationary y/base relationship, so the inverse y = T_hat + alphabase is unstable out-of-distribution -- exactly the catastrophic group-aware test collapse observed in prod (R^2=-146 on unseen wells). Set False to keep the legacy informational-only behaviour.
reject_base_on_alpha_drift	bool	True	When alpha-drift rejects the linear_residual spec for a base, ALSO drop every other spec on that SAME base (poly2 / yj / cbrt / monres ...). A base whose simplest bounded-inverse residual is non-stationary (fails the Chow slope test) cannot be safe under a MORE violent nonlinear inverse: keeping the base and switching to poly2/yj just trades a bounded blow-up for an O(base^2) one (prod TVT: the drift gate stripped the 3 safe linres specs, leaving the fragile families that then collapsed to R^2=-146). Only active when reject_on_alpha_drift is also True.
yscale_holdout_gate_enabled	bool	True	y-scale group-aware holdout gate. The MI-gain / i.i.d. honest-holdout screens the FORWARD transform but never the predict-T -> invert-to-y pipeline production actually runs. On a group-aware split (unseen groups/wells) a residual spec whose inverse amplifies a standardized base by ~std(y) blows up: y_hat slams the prediction envelope -> constant -> R^2 << 0. This gate replicates the prod path with a tiny model on a GROUP-DISJOINT holdout carved from the training groups and DROPS any spec whose inverted y-scale RMSE collapses (predictions degenerate to ~constant) or loses to the raw-y tiny baseline by more than yscale_holdout_gate_tolerance. Default ON; the gate no-ops (keeps all specs) when group ids are absent or too few groups exist to carve a disjoint holdout. Set enabled False to restore the pre-gate behaviour.
yscale_holdout_gate_tolerance	float	1.1
yscale_holdout_gate_sample_n	int	30000
yscale_holdout_gate_min_groups	int	4
yscale_holdout_gate_holdout_group_frac	float	0.3
structural_fragility_gate_enabled	bool	True	Structural fragility gate (runs BEFORE the val-split gate, from train alone). Drops base-ADDITIVE specs (diff / additive_residual / linear_residual / poly2 -- inverse y = T_hat + sbase) whose base variance is dominated by BETWEEN-group (well) level differences, because such a base takes out-of-range values on unseen groups and the additive inverse re-injects them. Catches the val-passes-but-test-collapses case the single val sample can miss (addres/diff on a per-well aggregate). Reject when between_var/total_var > frac AND sbetween_group_std > ratiostd(y). Default ON; group-aware only (no-op without group ids). 1.0 thresholds effectively disable.
structural_fragility_between_group_var_frac	float	0.6	Reject a base-additive spec when this fraction of the base's variance is BETWEEN-group (a per-group LEVEL that extrapolates on unseen groups). Scale-invariant -- the absolute-amplitude variant was scale-buggy (pipeline-standardized bases made between_std incomparable to std(y), so the gate silently never fired).
structural_fragility_min_base_sensitivity	float	0.5
structural_fragility_max_amplification_ratio	float	0.5	Deprecated/unused: the gate is now the scale-invariant between/total ratio above, not an absolute amplitude vs std(y). Kept for back-compat with configs that set it; has no effect.
oof_max_train_rows	int	200000	Cross-target ensemble honest-OOF stacking: cap the number of TRAIN rows the K-fold OOF refit uses to estimate the (~dozen-component) NNLS / dummy-floor blend weights. The weights are a tiny convex-ish solve that saturates far below millions of rows -- bench (_benchmarks/bench_oof_subsample_speedup.py) shows a group-aware subsample to ~30k yields an ensemble RMSE within ~1e-4 of the full-data weights while the refit wall drops ~Nx (6.5x@200k on cheap Ridge; ~100x at 2.96M on boosters -- the prod 4.5h -> minutes). The subsample keeps WHOLE groups so the group-aware OOF structure is preserved. 0 / None disables the cap (use every train row, the legacy behaviour). The full per-target models are unaffected -- this only bounds the WEIGHT-ESTIMATION refits, never the deployed components.
mi_gain_bootstrap_n	int	0	Bootstrap CI on mi_gain. The point-estimate mi_gain has noise floor that scales with the screening sample size and the heaviness of the y-tail; the eps_mi_gain absolute threshold misses this. Optional bootstrap (resample the screening sample, recompute MI, take 2.5/97.5 percentiles) produces an honest CI; the gate then compares eps_mi_gain against the lower CI bound, not the point estimate. Cost: mi_gain_bootstrap_n extra MI evaluations per spec (default 0 = disabled; recommended 50 for confidence band).
mi_gain_bootstrap_random_state	int	12345
mi_gain_fdr_control	bool	True	Family-wise multiplicity control across the many (base, transform) gain tests evaluated in one sweep. Each spec's per-comparison bootstrap CI / MI prefilter controls only its OWN error rate; testing dozens of specs inflates the chance that at least one noise spec spuriously "beats baseline". When enabled AND a per-spec bootstrap p-value exists (mi_gain_bootstrap_n > 0), discovery applies a Benjamini-Hochberg FDR correction over the whole family of per-spec p-values AFTER all candidates are scored, and drops specs whose BH-adjusted p-value exceeds mi_gain_fdr_alpha (one-sided H0: mi_gain <= 0). Default ON: it is a no-op under the shipped defaults (bootstrap disabled -> no p-values -> no specs filtered), so it never regresses recovery on default configs, and it tightens the false-discovery rate the moment a user re-enables the bootstrap gate. mi_gain_fdr_alpha is the target family-wise FDR level.
mi_gain_fdr_alpha	float	0.1
mi_n_strata_heavy_tail	int	30	Boost n_strata on heavy-tail targets when stratified MI sampling is enabled. Default 10 strata is too few for tail-driven signal -- tail rows get one bin each and MI estimates become unstable. Auto-detection: when y skew > 2.0 OR kurtosis > 5.0, boost mi_n_strata to mi_n_strata_heavy_tail. Manual override via setting mi_n_strata explicitly.
honest_holdout_frac	Optional[float]	0.2	Post-selection-inference holdout (winner's curse de-bias). The winner spec is selected on the SAME mi_gain statistic that is then reported, so its reported in-screen gain is the MAX over many candidates evaluated on the screening sample -- optimistically biased upward (the curse grows with candidate count). Before screening runs, carve honest_holdout_frac of the train rows into a holdout the discovery NEVER touches (screening, FDR gate, tiny-rerank, multi-base promotion, opt-in steps all consume only the screening pool); after the winner(s) are picked, RE-SCORE the final spec(s) on this fresh holdout for an honest generalisation gain. The honest gain is reported ALONGSIDE the in-screen gain (both labelled), so downstream generalisation claims use the de-biased number, not the selection score. Default ON per "enable corrective mechanisms by default"; set 0.0 / None to disable (callers who need every row for screening, e.g. tiny train_idx).
honest_rmse_gate_enabled	bool	True	Honest-holdout OOS predictive-error gate. The honest-holdout MI re-score above de-biases the REPORTED gain but MI is monotone-invariant and bias-inflated: a transform can raise MI while WORSENING y-scale OOS RMSE (canonical case: a ratio dividing by a small noisy base amplifies noise, yet MI(T, X) still climbs). This gate replicates the actual prediction objective on the same never-touched holdout: fit the tiny screening model on the screening rows for each spec's T, invert to y-scale on the holdout, and compare the holdout RMSE against the same tiny model on raw y. Specs whose y-scale holdout RMSE loses to raw by more than honest_rmse_gate_tolerance are DROPPED, and honest_holdout_rmse / honest_holdout_rmse_gain are stamped on the survivors. Crucially this protects the screening="mi" path too, which otherwise has NO OOS predictive gate end-to-end. Default ON ("enable corrective mechanisms by default"); no-ops when the honest holdout is absent (honest_holdout_frac disabled / too small).
honest_rmse_gate_tolerance	float	1.05	A spec survives only when its y-scale holdout RMSE <= raw-y tiny RMSE * this tolerance (1.05 = within 5% of raw).
honest_rmse_gate_sample_n	int	20000	Row cap per side (screen fit rows / holdout eval rows) for the gate's tiny-model fits; bounds cost on huge frames.
max_base_candidates	Optional[int]	None	Hard cap on the base-candidate grid entering the per-(base, transform) MI screen. The "auto" path is already capped by auto_base_top_k; an EXPLICIT base_candidates list had no cap, so a long list multiplied the whole transform grid. When set and the surviving explicit list is longer, the candidates are re-ranked by a cheap direct MI(y, x) pass and trimmed to the cap; the cap also trims the auto path when tighter than auto_base_top_k. None (default) preserves current behavior.
knn_mi_auto_downgrade	bool	True	knn-MI cost guard. The Kraskov estimator is O(n log n) PER COLUMN PER (base, transform) pair; on a large screen sample with many features/pairs a knn fit can silently take hours where bin-MI takes seconds. When enabled and mi_estimator="knn", discovery times a small per-column probe on the screen sample, extrapolates the full sweep cost (n_pairs * per-pair per-column cost), and DOWNGRADES the estimator to "bin" for this fit (with a warning) when the estimate exceeds knn_mi_budget_seconds. Set False (or budget <= 0) to never downgrade.
knn_mi_budget_seconds	float	600.0
honest_oof_selection	bool	True	Rank composite specs by HONEST group-OOF reconstruction RMSE (predict-T -> invert-to-y on the never-touched group-disjoint honest holdout) instead of the optimistic group-INTERNAL CV-RMSE. MI-gain stays the cheap pre-filter (top_k_after_mi); only the tiny-rerank's final ORDERING key changes. The honest holdout contains the out-of-range base tail where a base-additive inverse extrapolates and collapses -- a group-internal CV fold (bounded by the train base range) never samples it, so a fragile spec wins the internal CV yet collapses on the disjoint holdout. Promoting this estimate from a post-hoc GATE to the RANKING key means fragile specs are never promoted in the first place. No-op when group ids (_group_ids_for_rerank) or a non-empty honest holdout are absent -> falls back to the prior group-internal CV-RMSE ordering, so non-group / synthetic runs stay bit-identical. Set False for legacy replay.
honest_oof_selection_tolerance	float	1.05	In the rerank raw-baseline gate, when honest-OOF selection is active a spec must beat the raw-y honest-OOF reconstruction by this factor. Mirrors yscale_holdout_gate_tolerance.
honest_oof_floor_reject_enabled	bool	True	Enforce the honest-OOF floor as a REJECTION, not just a ranking key: when honest-OOF selection is active and a finite floor exists, drop every spec whose honest reconstruction RMSE cannot beat min(raw-y, AR-lag) within honest_oof_selection_tolerance. Without this the raw-baseline gate (require_beats_raw_baseline, off by default) is the ONLY thing that rejects, so honest-OOF merely reorders and a spec that loses to the lag_predict failsafe we deploy anyway is still carried into the ensemble (prod incident: 13.30 ensemble vs 11.58 lag floor). The floor is measured on the group-disjoint holdout; a spec whose holdout measurement degenerated is never floor-dropped (falls back to the group-internal CV rank). Set False for legacy rank-only behaviour.
per_group_discovery_enabled	bool	False	Per-group/per-cluster composite discovery (OPT-IN; formerly a REJECTED design decision -- see discovery/__init__.py module docstring near CompositeTargetDiscovery.fit). The original proposal was rejected for "10-15 values per cluster too few for stable per-cluster discovery", with an explicit reopen condition: "revisit at 500+ rows per cluster". This flag implements that reopened path: when True, fit ALSO runs discovery independently per distinct value of per_group_column (delegating to a fresh CompositeTargetDiscovery instance per group -- never a duplicated pipeline), for every group with >= per_group_min_rows rows. Groups below the floor fall back to the single GLOBAL spec set (specs_, unchanged). Results land on specs_by_group_. Default False: the global-only path (specs_) stays byte-identical when this flag is off, which is the default.
per_group_column	Optional[str]	None
per_group_min_rows	int	500
forbidden_base_patterns	List[str]	['^target_enc_', '^mean_target_', '_te$', '^lagged_target_', '^y_smooth_']	Forbidden base filters. Block columns whose names match any of these regex patterns (target leakage via target encoding / rolling target stats / etc.).
detect_base_leakage	bool	True	Structural pre-discovery leakage guard (detect_base_target_leakage). Default ON, but it only ACTS when a time_ordering is supplied to fit() -- the lag-probe then distinguishes a genuine time-shifted lag(y) base (NOT leaky, the canonical composite case) from a same-time near-identity re-encoding of y (leaky), so it never drops a legitimate lag. Catches target-encoding / rolling-target bases the forbidden_base regex misses. On non-temporal data (no time_ordering) it is a no-op so it can never mistake autocorrelation for leakage.
forbidden_base_corr_threshold	float	0.99999	Block columns whose Pearson |corr(base, y)| exceeds this threshold. Intent: catch literal copies / trivial linear transforms of y (e.g. y_renamed = y, y_scaled = y / 1000). NOT intended to catch autoregressive lag features such as y_prev -- those legitimately reach corr ~ 0.999 on slow-moving series due to autocorrelation, and they are exactly the kind of dominant feature composite-target discovery exists to handle. The primary defence against target-encoding leakage is the regex patterns above (forbidden_base_patterns); the corr threshold is just a backstop. Default raised from 0.999 to 0.99999 after observing it filtered out a legitimate y_prev (lag-1) on a real production run.
constant_base_eps	float	1e-12	Block constant or near-constant base columns (zero variance -> OLS in linear_residual is degenerate; ratio / logratio are uninformative).
min_valid_domain_frac	float	0.7	Domain validity. Drop a (base, transform) candidate entirely if fewer than this fraction of train rows pass the transform's domain_check (e.g. logratio requires y, base > 0).
fail_on_no_gain	str	'fallback_raw'	Behaviour when no candidate clears eps_mi_gain. - "fallback_raw": warn and emit no composite targets (caller trains on raw target only). - "raise": raise RuntimeError -- useful in CI / scripted modes to flag degenerate inputs. - "warn": warn but emit the best-of-bad candidates anyway.
random_state	int	42
cross_target_ensemble_strategy	str	'nnls_stack'	Cross-target ensemble strategy. Run after each composite-target model is wrapped to y-scale, builds one combined predictor over all (raw + K composite) wrappers. - "off": no ensemble; models[type][f"_CT_ENSEMBLE__{target}"] not created. - "mean": equal-weight average over all components. - "oof_weighted": gain-over-baseline weighting using per-component RMSE (train-RMSE proxy by default; honest holdout RMSE when oof_holdout_frac > 0); auto-falls-back to best-single component if no component clears the baseline. - "linear_stack": Ridge regression on per-component predictions. - "nnls_stack": non-negative least squares on per-component preds. nnls_stack chosen as default after composite_ensemble_shootout.py (6 scenarios x 3 seeds, 11 strategies): NNLS was the only strategy with positive mean improvement vs best-single-by-train (+1.24%, 13/18 wins). Single-spec case falls back to best-single inside the ensemble class. Set to "off" to skip ensemble construction entirely.
auto_skip_on_baseline_optimal	bool	False	When True AND the per-target baseline_diagnostics reports composite_recommendation == "unlikely_to_help", discovery short-circuits with a warning and produces no specs. Saves the MI / tiny-model / re-fit cost on targets where composite mode is unlikely to add value (init_score baseline already captures the dominance, or no feature dominates strongly). Default False so explicit opt-ins don't get silently overridden.
use_baseline_diagnostics_hint	bool	True	Use BaselineDiagnostics ablation top-K as priority base candidates (dominant_features_hint) instead of relying on pairwise MI(y, x) ranking alone. Pairwise MI gets fooled by features with global trend but no structural residual signal (spatial coords on geographically-trended y); ablation directly measures predictive contribution and is much more reliable. When True, train_mlframe_models_suite runs BaselineDiagnostics inline (cached) before discovery and injects the top-K ablation-ranked features as the hint. When the inline diagnostic fails or returns no dominant features, falls back silently to MI-only ranking. Default True since it strictly improves auto-base on the production failure mode and the inline BD cost is amortised (the same diagnostic runs in the per-target loop later; caching reuses it).
baseline_diagnostics_hint_top_k	int	3
hint_strength_threshold_pct	float	50.0	Hint-strength threshold for the adaptive hint cap. When the top hint feature has BaselineDiagnostics ablation delta_pct >= hint_strength_threshold_pct, _auto_base uses the FULL hint list (no cap) instead of capping at max(1, top_k // 2). Set to a high value (e.g. 1000) to effectively disable the strong-hint shortcut.
auto_base_dedup_corr_threshold	float	0.95	Cross-base correlation dedup. After auto-base ranking, drop a candidate base if its absolute Pearson correlation against any already-kept candidate exceeds this threshold on the screening sample. Stops near-duplicate lag variants (y_prev, y_prev_lag2, y_smooth_3) from all surviving into Phase B and inflating ensemble correlation. Set to 1.0 to disable.
dedup_x_remaining_for_mi_baseline	bool	True	De-duplicate near-collinear feature columns from x_remaining BEFORE the per-base mi_y_compare MI baseline. x_remaining excludes only the base column itself; a near-duplicate sibling of the base (a second lag, a smoothed copy) left in the remaining set carries almost the same info as the removed base, so it inflates MI(y, x_remaining) while contributing little to MI(T, x_remaining) -- biasing mi_gain DOWN. The bias is conservative (it never over-keeps a spec, only wrongly sinks one), but it most hurts exactly the lag-family bases discovery is built to find. When enabled, columns whose absolute Pearson correlation with an already-kept-column exceeds dedup_x_remaining_corr_threshold are dropped from x_remaining (the FIRST of each collinear group is kept) so both halves of mi_gain score the de-duplicated feature set. Default ON per the "enable corrective mechanisms by default" convention; the dedup is a strict no-op when no two surviving columns are that correlated, and the threshold defaults high (0.99) so only genuine near-duplicates are removed. Set False to reproduce the pre-fix full-x_remaining baseline.
dedup_x_remaining_corr_threshold	float	0.99
auto_base_mi_per_pair_mask	bool	True	Rank auto-base candidates by MI computed with PER-PAIR (per-column) NaN masking instead of the global all-column finite intersection. For mid-range-NaN columns the global intersection keeps only the rows where EVERY feature is observed -- a non-random (MNAR) subset -- so MI(y, x_j) on it is biased by the joint-observability pattern and silently shifts which base wins. Per-pair masking estimates each column's MI on its own observed rows (matching _mi_to_target and the prebinned -1-sentinel path) and is bit-identical when the screening sample has no NaN. Default ON per the "enable corrective mechanisms by default" convention; set to False only to reproduce the pre-fix global-mask ranking.
auto_base_mnar_per_pair_threshold	float	0.5	Fraction of the per-pair-available row mass below which the global intersection is judged to be MNAR-shrinking the ranking sample; used purely to LOG that the per-pair ranking diverged from what the global mask would have produced (auditability). Does not change behaviour when auto_base_mi_per_pair_mask is True (per-pair is always used).
auto_base_null_perms	int	20	Permutation-MI null distribution test in _auto_base. For each candidate feature compute MI(y, x) AND MI(y, shuffle(x)) on auto_base_null_perms shuffles, then require the candidate's MI to exceed mean_null + n_sigma * std_null. Catches features whose MI(y, x) is non-trivial only because of a shared monotonic component (time/spatial trend), not structural information about y. Cost: auto_base_null_perms extra MI evaluations per candidate (default 20 × ~1ms each on bin-MI estimator = ~20ms per feature on the screening sample). Set auto_base_null_perms=0 to disable.
auto_base_null_z_threshold	float	3.0
auto_base_null_block_length	Union[str, int]	'auto'	Block-shuffle length for temporal datasets so the null preserves marginal autocorrelation. "auto" uses int(sqrt(n)); explicit int for fixed length; 1 for row-level shuffle (i.i.d. assumption).
auto_base_demote_time_index	bool	True	Structural detectors for time-index and spatial-coordinate features. Demote (push to bottom of MI ranking) features that look like: - Time index: |Spearman(rank(x), arange(n))| > 0.95. Catches a row-counter or timestamp masquerading as a base candidate; on temporal data the row index correlates with y purely from drift, no structural information. - Spatial coordinate block: pairwise correlations among 3+ numeric features form a block where each pair has |corr| > 0.5. Catches X/Y/Z lat-lon-altitude triplets where pairwise MI(y, coord) is high purely from spatial drift, not structural information. Demoted features are ALSO available as bases when their MI genuinely exceeds non-demoted candidates (defensive, not blocking). Set to False to disable.
auto_base_demote_spatial_coords	bool	True
auto_base_structural_boost	bool	True	Structural-affinity boost for _auto_base. Surfaces OBVIOUS base columns from data shape / correlation that the MI ranking alone can miss when a noisier competitor's pairwise MI(y, x) lands a hair higher: - Near-affine predictor (\|corr(y, x)\| very high AND the OLS residual variance collapses): prime linear_residual base. - Low-cardinality integer column (a small set of distinct integer levels, not one-per-row): prime grouped base. - Monotone / timestamp column (forward diffs share one sign): prime time base. The boost is a BOUNDED additive nudge scaled to the candidate MI spread (auto_base_structural_boost_fraction of the MI range), so it can lift a near-tie but never override a clearly larger MI gap -- it AUGMENTS the MI ranking, it does not replace it. Bit-identical to "no boost" on data with no detectable structure. Default ON per the "enable corrective mechanisms by default" convention; set False to reproduce the pre-boost MI-only ranking.
auto_base_structural_boost_fraction	float	0.25
collapse_linear_residual_alpha_eps	float	0.05	Collapse linear_residual -> diff when the fitted alpha is approximately 1.0. linear_residual is a strict generalisation of diff (diff = linear_residual with alpha=1, beta=0). When OLS lands at alpha~1 on stationary lag features, the two transforms produce numerically identical T columns -- but linear_residual carries TWO learned parameters with train-time variance. diff is the lower- variance answer. The threshold compares the scale-invariant ratio \|alpha - 1\| * std(base) / std(y); below this value, the linear_residual spec is considered redundant with diff and dropped if a diff spec for the same base also kept. Set to 0.0 to disable (always keep both).
multilabel_strategy	str	'per_target'	Handling multilabel (multi-output) regression targets, i.e. target_by_type[regression][name] is a 2-D array of shape (n_rows, n_outputs). - "per_target" (default): expand into n_outputs separate 1-D regression targets named {name}_out{j}; discovery runs independently per output, naming composites {name}_out{j}__{transform}__{base}. Per-target training loop downstream sees them as ordinary 1-D targets. - "skip": legacy behaviour -- mark with metadata note, produce no composites for that target. Useful when the caller knows they don't want the per-output expansion (e.g. the training loop downstream expects the 2-D shape intact).
max_inference_components	Optional[int]	None	Cap the number of components combined at predict time. Useful for online single-row latency-sensitive serving where running K=8 wrappers per row blows the SLA. When > 0, the ensemble keeps only the top-N components by weight (after the standard weight computation), drops the rest, and re-normalises. None / 0 means keep all components (default).
calibrate_cross_target_output	bool	False	Post-hoc recalibration of the cross-target ensemble's blended output. When True, after the OOF gate the suite fits a monotone OutputCalibrator on the SAME OOF holdout surface the NNLS / gain weights were derived from -- it blends the OOF component matrix with the frozen ensemble weights, then fits an isotonic (or sigmoid / linear) map of that OOF blend onto the OOF truth and applies it as a final monotone post-map at predict time. This removes the systematic (often S-shaped) miscalibration a least-squares blend of biased components leaves behind, without changing the ensemble's ranking. Leakage-free: the fit consumes only out-of-fold predictions, never a re-prediction of train. Default False: with no calibrator attached predict returns the raw blend bit-for-bit, so the feature is bit-identical when off.
cross_target_calibration_method	str	'isotonic'	Calibration map family: "isotonic" (free-form monotone, default), "sigmoid" (Platt-style 2-param S), or "linear" (affine scale+offset).
oof_holdout_frac	float	0.2	Honest OOF for the ensemble gate / stacking. When > 0, the suite carves an extra holdout slice (this fraction of filtered_train_idx) and at ensemble-build time re-fits a clone of every component on the (1-frac) stack_train slice, then predicts on the held-out slice. The honest holdout predictions feed the stacking solvers (linear_stack / nnls_stack) and the gain-over-baseline weights, replacing the train-RMSE proxy that overstates accuracy. Cost: re-fits every component once on (1-frac) of train rows. Default flipped from 0.0 -> 0.2 because the default cross_target_ensemble_strategy is nnls_stack. Fitting NNLS on in-sample component predictions is a stacking leak: every component has effectively memorised its training rows, so NNLS picks weights that overweight whichever component fits noise best. A 20% honest holdout is the standard "stacking on OOF" cure documented in Sill et al. 2009 (Feature-Weighted Linear Stacking) and removes the leak at the cost of one extra fit per component on 80% of train rows. Set to 0.0 explicitly to opt out (e.g. when using a non-stacking strategy like mean where the train-RMSE proxy is harmless).
oof_random_state	int	42
oof_holdout_source	str	'kfold'	OOF holdout source for the cross-target ensemble stacker / weights / gate. Three modes: - "kfold" (default): true train-K-fold OOF. Each component is re-fit on K-1 folds and predicts the held-out fold; the concatenated (n_train, K) OOF matrix drives stack weights, gain-over-naive weighting, and the honest-OOF gate. This is the only source that never reuses the early-stopping (val) surface for weighting: the booster components were early-stopped against val, so weighting on val double-dips a biased surface and systematically over-weights whichever component fit the val noise best. K-fold OOF is the standard "stacking on OOF" cure (Sill et al. 2009). Cost: K-1 extra fits per component on (K-1)/K of train. - "external_val": fit each component clone on the FULL train slice, predict on the suite's val frame. Cheaper (one fit per component) but the val frame was the early-stopping surface for the booster components, so the resulting weights/gate are optimistically biased toward components that overfit val. A one-time WARN is emitted. Keep this for a representativeness cross-check against the kfold source, or when K-fold is too expensive; do not use it as the production weighting surface for early-stopped models. - "train_tail": legacy single-slice carve from the trailing oof_holdout_frac of train (time-aware when time_ordering is monotone, random shuffle otherwise). Use when val is unavailable / single fit is required and the train-tail distribution matches test.
oof_kfold	int	5	Number of folds for oof_holdout_source="kfold". 5 is the standard stacking default; higher K gives a larger per-component training fraction (less pessimistic OOF) at linear extra fit cost.
stacking_aware_gate_enabled	bool	False	Stacking-aware gate (measure-first NNLS gate). When True AND cross_target_ensemble_strategy is linear_stack or nnls_stack, the ensemble-build path first runs :func:stacking_aware_gate over the component predictions to drop components whose NNLS weight falls below stacking_aware_gate_min_weight. The surviving subset feeds the actual stacker. Skipped when fewer than 2 components survive (the stacker handles single-component falls back on its own).
stacking_aware_gate_min_weight	float	0.05
gate_kind	str	'nnls'	gt_05: which weighting the gate above uses to decide survivors. "nnls" (default) is the legacy regression-coefficient gate -- collinear/near-duplicate components can get an arbitrary all-or- nothing weight split, making pruning unstable under redundancy. "shapley" uses :func:mlframe.training.composite.ensemble.stacking.shapley_aware_gate instead: the symmetry axiom makes near-duplicates share credit equally, so pruning is stable under redundancy. Default stays "nnls" until a majority-win bench across real-ish pools (gt_05 acceptance criteria) -- this is purely a same-cost drop-in alternative behind the SAME stacking_aware_gate_enabled flag.
ct_ensemble_dedup_enabled	bool	False	Residual-correlation dedup. Before weighting / stacking, compute the pairwise Pearson correlation of the honest-OOF residuals (residual_correlation_matrix) and, for any pair whose |corr| exceeds ct_ensemble_dedup_corr_threshold, drop the WEAKER member (higher OOF RMSE). Near-duplicate components otherwise split the NNLS weight between themselves and let a redundant pair dominate the stack. Always keeps at least 2 members. Default OFF pending the committed bench (training/_benchmarks/bench_ct_ensemble_residual_dedup.py); flip ON only if it wins on the majority of seeds.
ct_ensemble_dedup_corr_threshold	float	0.95
lag_predict_failsafe_tolerance	float	0.1	AR(1) failsafe: when lag_predict is injected into the CompositeCrossTargetEnsemble component pool and its OOF holdout RMSE is within (1 + lag_predict_failsafe_tolerance) of the best single trained component, prefer the zero-parameter lag_predict over the multi-component stack. Defends against the train-tail-vs-test distribution mismatch on strong-AR targets. Default 0.10 (10%). The earlier 0.50 was calibrated for group-blind train-tail carves where the trained-model OOF was artificially inflated by ~25% due to within-group leakage in the inner early-stopping eval; now that _carve_inner_eval_split is group-aware, the trained-model honest OOF is no longer biased high vs lag, so the tolerance no longer needs to absorb the gap. Set to 0 to disable.
ct_ensemble_dummy_floor_enabled	bool	True	Dummy-floor gate: when honest OOF predictions are available, drop any component from the CompositeCrossTargetEnsemble pool whose OOF RMSE > raw target's strongest-dummy RMSE x (1 + tolerance). A trained model that loses to a parameter-free dummy on the honest holdout cannot improve the ensemble; keeping it dilutes NNLS weight and harms test performance. Prod observed this as the load-bearing failure mode: composite-target models on residual T overfit on group-aware splits (pred_std up to 5x target_std, R2 down to -22 on test), but NNLS still gave them positive weight. With the gate the pool reduces to {lag_predict} + components that genuinely beat the dummy, and the ensemble cannot ship worse than the dummy. Set ct_ensemble_dummy_floor_enabled = False to disable; bump ct_ensemble_dummy_floor_tolerance to keep components within a small slack above the dummy (default 0 = strict).
ct_ensemble_dummy_floor_tolerance	float	0.0
extreme_ar_group_aware_skip	bool	True	Extreme-AR + group-aware skip. When the target is dominated by an AR lag (lag1_autocorr_per_group >= extreme_ar_threshold) AND the split is group-aware (test wells/groups unseen at training), composite-target discovery is short-circuited because the residual T = y - alpha * lag has near-zero signal on unseen groups; any trained model on T overfits per-group patterns and produces predictions worse than the trivial median(T) dummy on the test split (observed in prod: 3 composite specs shipped, all 9 trained models on residuals failed dummy gate with R2 <0, ~10 min of wall-time wasted per target). Set extreme_ar_group_aware_skip = False to force discovery to run anyway.
extreme_ar_threshold	float	0.99
always_build_ct_ensemble_for_raw	bool	True	Build CT_ENSEMBLE for raw targets even when ZERO composite specs were discovered. Without this knob, the entry guard in _phase_composite_post requires composite_specs_by_target_type to be non-empty, so the dummy-floor gate + lag_predict injection + AR(1) failsafe never run on raw-only targets. The suite then ships a simple-arithmetic ensemble of raw models that is worse than the best single component when 3-of-4 boosters are above the lag floor (observed in prod: EnsARITHM TEST=12.45 vs Ridge alone 11.63, lag_predict 11.58). With this knob the OOF gate sees Ridge alone and prefers it, or falls back to lag_predict if the AR-failsafe tolerance is met. Disable to revert to the legacy raw-models-only ensemble path (mean/median/arith flavours only).
composite_feature_stacking_enabled	bool	False	produces an opt-in stub call to composite_oof_predictions / composite_predictions_as_feature on the discovered specs so downstream code can attach the predictions as engineered features. Default False; full wiring requires the downstream FE pipeline to consume the new column, which is caller-specific.