Quant And Machine Learning Portfolio Construction: Random Weights, K-Means, Fixed Portfolio, KNN Projections

We combine ideas from:

Quant And Machine Learning Portfolio Construction: KNN Projections Using Futures Calibration Data
Quant And Machine Learning Portfolio Construction: Random Weights, K-Means, Fixed Portfolio

to explore weights for the following portfolio.

Vanguard Stock Sectors and Treasury Bonds

	Name
VOX	Vanguard Communication Services ETF
VCR	Vanguard Consumer Discretionary ETF
VDC	Vanguard Consumer Staples ETF
VDE	Vanguard Energy ETF
VFH	Vanguard Financials ETF
VHT	Vanguard Health Care ETF
VIS	Vanguard Industrials ETF
VGT	Vanguard Information Technology ETF
VAW	Vanguard Materials ETF
VNQ	Vanguard Real Estate ETF
VPU	Vanguard Utilities ETF
BIL	SPDR Bloomberg 1-3 Month T-Bill ETF
IEF	iShares 7-10 Year Treasury Bond ETF
IEI	iShares 3-7 Year Treasury Bond ETF
SHY	iShares 1-3 Year Treasury Bond ETF
TIP	iShares TIPS Bond ETF
TLT	iShares 20+ Year Treasury Bond ETF

Weights are generated via the following steps:

Calibration data is identical to that in the first reference above.
Choose a trade direction (long only here), generate 25000 sets of random weights (normalized).
For each set of weights, simulate trades for 210 days, rebalancing each quarter, to generate a time series.
Transform each time series into arrays of dual moving averages and implement the same robust scaler used in 1 (this was done in the first reference above for each asset in the portfolio).

Here we generate two sets of weights.

First, we match the transformed time series with calibration data using 50 nearest neighbors. Then we choose the best and worst 63 day projected: expectancy, 50th percentile percent return, long win percent.

Best and Worst 63 Day Projections

VOX	VCR	VDC	VDE	VFH	VHT	VIS	VGT	VAW	VNQ	VPU	BIL	IEF	IEI	SHY	TIP	TLT	10th Pctl	20th Pctl	30th Pctl	40th Pctl	50th Pctl	60th Pctl	70th Pctl	80th Pctl	90th Pctl	Expectancy	Up %	Down %	Median Up	Median Down
0.1286	0.1232	0.0669	0.0035	0.0123	0.0587	0.1298	0.0802	0.0079	0.0402	0.0189	0.1024	0.0161	0.0482	0.1443	0.0063	0.0127	-1.0876	-0.3688	0.3764	1.8199	3.363	6.4031	8.2897	8.9765	13.1252	5.6174	74.0	26.0	7.841	-0.7113
0.1301	0.0119	0.0079	0.046	0.0169	0.0682	0.0708	0.1178	0.1021	0.0732	0.0485	0.0173	0.0531	0.0662	0.0043	0.117	0.0485	-8.2174	-4.3383	-3.5372	-2.6412	-1.5049	-0.6941	-0.0462	1.2303	4.232	-1.9376	30.0	70.0	1.617	-3.461
0.1427	0.1061	0.1218	0.003	0.0019	0.1257	0.1448	0.1351	0.0064	0.0228	0.0156	0.0081	0.0501	0.0325	0.006	0.0347	0.0427	-3.2925	-0.5539	1.2787	3.0053	5.203	6.181	7.3816	9.3684	12.8239	4.5052	76.0	24.0	6.6167	-2.1814
0.0351	0.0055	0.0169	0.1535	0.0999	0.0012	0.1102	0.0409	0.1135	0.1499	0.0181	0.0294	0.1185	0.0019	0.063	0.019	0.0234	-6.4614	-4.3498	-3.5813	-2.9341	-2.1418	-1.5564	0.8652	2.2212	4.8432	-1.0785	34.0	66.0	3.2674	-3.3172
0.1292	0.1256	0.0404	0.0348	0.032	0.0855	0.0185	0.1153	0.0642	0.0138	0.0812	0.0473	0.1058	0.0868	0.0067	0.0122	0.0007	-0.7402	0.1798	1.6588	2.8016	4.2583	5.3393	7.3294	8.8134	15.3944	4.1946	82.0	18.0	5.335	-1.0005
0.0033	0.0242	0.0426	0.0045	0.0017	0.0635	0.0465	0.0427	0.0768	0.0771	0.0281	0.0498	0.1123	0.1143	0.0651	0.1301	0.1173	-3.9979	-2.4309	-1.9444	-1.2286	-0.7678	-0.5412	-0.1853	0.1396	2.2201	-0.5455	26.0	74.0	2.081	-1.4683

Second, we want to examine a broad range of weights. We do this by applying K-Means with 50 clusters to the 25000 sets of weights. The weight closest to its centroid is extracted and we present projections in the table below.

K-Means Weights and Projections

VOX	VCR	VDC	VDE	VFH	VHT	VIS	VGT	VAW	VNQ	VPU	BIL	IEF	IEI	SHY	TIP	TLT	10th Pctl	20th Pctl	30th Pctl	40th Pctl	50th Pctl	60th Pctl	70th Pctl	80th Pctl	90th Pctl	Expectancy	Up %	Down %	Median Up	Median Down	Cluster Number
0.0349	0.0808	0.0448	0.0771	0.0393	0.0483	0.0755	0.0436	0.0635	0.0733	0.0318	0.0704	0.0984	0.0215	0.0634	0.0791	0.0542	-4.6346	-2.9863	-2.3276	-1.1728	-0.6621	0.0194	0.2913	0.8971	1.6682	-1.0335	42.0	58.0	0.8121	-2.3699	0
0.0342	0.045	0.0727	0.0423	0.0404	0.0717	0.09	0.0639	0.072	0.0476	0.0991	0.0349	0.0661	0.0404	0.0396	0.0604	0.0796	-2.8931	-1.546	-1.0443	-0.5992	0.4092	0.8761	1.6278	2.4499	5.1876	0.4294	56.0	44.0	1.9007	-1.443	1
0.0941	0.0662	0.107	0.0175	0.0546	0.0344	0.045	0.0567	0.0797	0.0867	0.0924	0.0542	0.0255	0.0542	0.0387	0.0266	0.0664	-4.4649	-2.558	-1.693	-0.3847	0.1175	1.5119	3.8995	4.4759	6.2546	1.0835	54.0	46.0	4.0843	-2.4393	2
0.0707	0.0656	0.0224	0.0783	0.0492	0.0856	0.0522	0.0675	0.0266	0.0844	0.0172	0.069	0.0806	0.0356	0.0786	0.0376	0.0788	-4.4649	-2.2082	-1.1713	-0.6602	-0.2451	0.082	0.5734	1.5481	5.3843	-0.1557	46.0	54.0	1.4418	-1.5165	3
0.0705	0.071	0.0264	0.0435	0.076	0.0573	0.0628	0.0531	0.0685	0.0624	0.0638	0.0804	0.0781	0.0738	0.0529	0.0133	0.0461	-4.6854	-3.1498	-2.2208	-1.1888	-0.4952	0.1149	1.4661	3.2544	6.2546	0.146	46.0	54.0	3.1356	-2.4007	4
0.0785	0.0656	0.0889	0.0635	0.028	0.024	0.0489	0.0572	0.0974	0.089	0.0424	0.038	0.0534	0.0862	0.0266	0.0777	0.0347	-4.2993	-2.8131	-2.0351	-0.2032	0.5125	1.5691	3.3267	5.4588	8.9274	0.8667	58.0	42.0	3.4893	-2.755	5
0.081	0.068	0.069	0.0546	0.0509	0.05	0.0694	0.0648	0.0567	0.0676	0.0902	0.0667	0.0619	0.0328	0.0222	0.0164	0.0778	-5.0649	-3.5374	-2.2208	-0.2533	0.2001	1.5119	3.3267	4.4498	7.0401	0.6808	56.0	44.0	3.6849	-3.1425	6
0.0507	0.0551	0.0932	0.0936	0.0721	0.0807	0.0727	0.0787	0.043	0.0658	0.0103	0.0009	0.0322	0.0935	0.0663	0.0583	0.033	-5.1466	-3.691	-2.9554	-1.6304	-0.5381	0.6779	2.2278	3.7848	7.3372	-0.2004	48.0	52.0	3.3732	-3.4992	7
0.0713	0.0581	0.0725	0.0198	0.0529	0.0795	0.0832	0.0732	0.0798	0.0376	0.017	0.0406	0.0764	0.0597	0.07	0.0299	0.0785	-5.0649	-2.8131	-2.1858	-0.7269	0.0674	1.1129	3.8995	4.3764	5.705	1.0359	52.0	48.0	4.1957	-2.3871	8
0.0577	0.067	0.0423	0.0473	0.078	0.0228	0.0817	0.0617	0.0403	0.0089	0.0631	0.0817	0.0813	0.0655	0.0705	0.0669	0.0632	-3.5	-2.3215	-1.4471	-0.7623	-0.3881	0.0242	0.2913	1.0824	2.2555	-0.4555	44.0	56.0	0.8858	-1.5094	9
0.0854	0.0678	0.08	0.052	0.0875	0.0278	0.0828	0.045	0.0696	0.0251	0.063	0.0426	0.062	0.0466	0.053	0.0604	0.0493	-5.2531	-4.1949	-2.9736	-1.783	0.0178	0.8953	1.7866	3.7888	8.3424	-0.1365	50.0	50.0	3.2571	-3.5301	10
0.015	0.0286	0.0943	0.047	0.063	0.056	0.0728	0.0996	0.0615	0.0519	0.0536	0.083	0.089	0.0362	0.0877	0.0288	0.032	-2.8931	-1.5096	-1.0898	-0.6326	-0.0518	0.22	0.8032	1.4885	2.0853	-0.0444	50.0	50.0	1.2375	-1.3264	11
0.0683	0.0323	0.0493	0.0596	0.0854	0.0669	0.0664	0.0258	0.1003	0.0218	0.0601	0.0314	0.0553	0.0645	0.0615	0.0923	0.059	-3.8539	-3.0278	-1.3466	-0.5992	0.2354	0.6279	1.4928	2.1155	5.9416	-0.4007	56.0	44.0	1.5381	-2.8682	12
0.0779	0.0807	0.0435	0.0736	0.0576	0.0857	0.0028	0.0603	0.0466	0.0753	0.0787	0.085	0.0176	0.0441	0.061	0.0632	0.0464	-4.6854	-2.3277	-1.3099	-0.6123	0.0435	0.3124	1.3339	2.7327	6.2546	-0.1741	54.0	46.0	1.5371	-2.1828	13
0.0265	0.011	0.0677	0.0294	0.0541	0.041	0.1125	0.0389	0.0815	0.0894	0.0536	0.0553	0.0372	0.0963	0.0738	0.0543	0.0776	-4.0519	-2.3301	-1.9168	-1.0926	0.2319	1.0875	2.8048	3.7104	5.3254	0.7079	54.0	46.0	3.2528	-2.2796	14
0.0675	0.0759	0.0788	0.0299	0.0815	0.0662	0.0381	0.0398	0.0529	0.0621	0.0693	0.0362	0.075	0.0434	0.028	0.0774	0.0778	-2.8931	-2.0494	-1.2248	-0.6326	0.0435	0.3211	0.8032	1.6066	2.673	-0.2015	54.0	46.0	0.9723	-1.5794	15
0.0519	0.0883	0.0416	0.0691	0.0666	0.0679	0.0281	0.0606	0.0791	0.0198	0.0749	0.0429	0.0647	0.0497	0.052	0.0611	0.0816	-3.0888	-2.3277	-1.1713	-0.6602	-0.2451	0.1149	0.5627	1.2984	1.8624	-0.365	48.0	52.0	0.8922	-1.5254	16
0.026	0.0233	0.0796	0.0702	0.0944	0.0426	0.0945	0.0793	0.0417	0.0502	0.0408	0.0227	0.0839	0.0631	0.0613	0.092	0.0344	-4.6854	-3.5495	-2.51	-1.2308	-0.6621	-0.1774	0.2187	0.9003	1.6682	-1.1367	40.0	60.0	1.0325	-2.5828	17
0.0319	0.112	0.0802	0.0917	0.0625	0.0624	0.0546	0.0934	0.036	0.0351	0.0465	0.0238	0.0885	0.0519	0.0147	0.0442	0.0707	-5.1466	-4.2077	-2.8698	-1.783	-0.9244	-0.0694	0.535	2.1876	4.0298	-0.7546	38.0	62.0	2.5091	-2.755	18
0.088	0.0169	0.0513	0.0596	0.0727	0.0716	0.0702	0.0657	0.0567	0.0711	0.0273	0.0305	0.072	0.0794	0.0544	0.0483	0.0643	-4.6854	-2.8063	-1.2846	-0.6602	-0.0956	0.1149	0.5734	1.4885	4.3443	-0.4959	48.0	52.0	1.0733	-1.9445	19
0.0299	0.0387	0.0878	0.0221	0.05	0.0788	0.071	0.0184	0.0572	0.0581	0.0912	0.0774	0.0359	0.0736	0.059	0.0728	0.0781	-4.6409	-3.3493	-1.8406	-1.0791	-0.4435	1.3049	2.1352	3.0225	4.6864	-0.2411	46.0	54.0	2.7155	-2.7597	20
0.0866	0.0526	0.0616	0.0643	0.0237	0.0273	0.0358	0.0948	0.033	0.0519	0.0299	0.0939	0.0215	0.0635	0.0843	0.0844	0.0909	-3.7963	-3.0334	-1.6334	-1.2621	-0.6436	-0.0613	0.4972	1.6222	2.3912	-0.3211	40.0	60.0	1.709	-1.6745	21
0.0398	0.0744	0.0722	0.0632	0.0774	0.0611	0.0765	0.0746	0.0509	0.0504	0.0715	0.0734	0.0218	0.0302	0.018	0.0765	0.068	-5.1169	-4.1949	-3.0593	-2.2335	-1.1055	0.4852	1.5707	3.601	7.2739	-0.3996	46.0	54.0	3.2571	-3.5145	22
0.1007	0.0989	0.0396	0.0323	0.0541	0.0708	0.0627	0.0787	0.0209	0.0094	0.0403	0.0568	0.0441	0.0917	0.0446	0.0796	0.0748	-7.085	-3.8227	-2.2666	-1.343	-0.2517	-0.0377	1.5303	4.1289	5.1719	0.2887	40.0	60.0	4.1957	-2.3159	23
0.0739	0.0528	0.0651	0.0663	0.0787	0.0298	0.059	0.0382	0.0558	0.0729	0.0143	0.0536	0.0805	0.0775	0.0773	0.0312	0.073	-4.6346	-3.1295	-2.3368	-1.1728	-0.6621	0.0194	0.2913	0.8971	1.6682	-1.0513	42.0	58.0	0.8121	-2.4007	24
0.0627	0.0614	0.0802	0.0289	0.0688	0.029	0.0514	0.0514	0.0929	0.0455	0.0832	0.0392	0.0301	0.0871	0.0013	0.098	0.0888	-3.0888	-2.168	-1.4234	-0.6602	0.0435	0.3211	0.8601	1.7887	2.673	-0.2666	54.0	46.0	1.2375	-2.0323	25
0.0727	0.0372	0.0367	0.0331	0.0232	0.0825	0.0324	0.0856	0.0547	0.0304	0.0935	0.0643	0.0586	0.0738	0.0817	0.0878	0.0518	-2.9108	-1.8221	-1.2618	-0.1272	0.4764	0.8884	1.4501	2.8069	4.992	0.1444	56.0	44.0	1.6702	-1.7976	26
0.0604	0.0379	0.0678	0.0703	0.0463	0.0447	0.0768	0.0736	0.0776	0.0412	0.0816	0.0872	0.044	0.0415	0.0588	0.0346	0.0556	-4.6854	-2.3277	-1.3099	-0.6123	0.0195	0.22	1.0014	2.1109	6.2546	-0.1074	52.0	48.0	1.5299	-1.8811	27
0.0836	0.0804	0.078	0.0886	0.0051	0.0761	0.0656	0.0785	0.0321	0.0622	0.0767	0.0618	0.0393	0.0485	0.071	0.0485	0.0039	-8.951	-4.4103	-3.4931	-2.2596	-0.9244	-0.2784	0.535	2.1876	4.0298	-1.0809	36.0	64.0	2.8831	-3.3107	28
0.0643	0.0529	0.0556	0.0552	0.0381	0.0372	0.0373	0.0279	0.0283	0.1039	0.0769	0.0916	0.0908	0.0369	0.0693	0.0967	0.037	-3.795	-2.2845	-1.7282	-1.1464	-0.0104	0.4682	1.7018	3.0568	4.9941	0.0702	50.0	50.0	2.3091	-2.1687	29
0.0465	0.0507	0.0435	0.063	0.0557	0.0983	0.0664	0.1027	0.0405	0.0801	0.0095	0.0646	0.0123	0.0374	0.0502	0.0756	0.1028	-5.0649	-3.1425	-2.2208	-0.7478	0.0622	0.7009	3.2838	4.3136	6.2546	0.771	52.0	48.0	3.9122	-2.6319	30
0.0762	0.0924	0.0264	0.0494	0.0525	0.0351	0.088	0.0291	0.0638	0.0896	0.0392	0.0425	0.0399	0.0751	0.074	0.0897	0.037	-5.0531	-3.5374	-2.3484	-1.1926	0.0178	0.7009	1.6267	3.5675	7.2739	-0.1229	50.0	50.0	2.5091	-2.755	31
0.0586	0.0195	0.0746	0.0361	0.0611	0.0858	0.0275	0.0742	0.083	0.0568	0.0664	0.0601	0.0711	0.0199	0.0661	0.0824	0.0568	-3.551	-2.0494	-1.4234	-0.6561	0.2825	0.8045	1.5008	2.3141	5.1507	0.1409	54.0	46.0	1.6673	-1.6509	32
0.0315	0.0539	0.043	0.0422	0.0796	0.0646	0.0175	0.0859	0.0907	0.0751	0.0749	0.0234	0.0285	0.0753	0.0762	0.0832	0.0547	-3.0875	-2.0878	-1.2248	-0.6602	0.0195	0.3124	0.8032	1.4885	2.2878	-0.1626	52.0	48.0	1.1049	-1.5358	33
0.0512	0.0655	0.0211	0.0556	0.0779	0.0379	0.008	0.0444	0.0857	0.0744	0.0289	0.1033	0.0244	0.103	0.0907	0.0718	0.0561	-3.5009	-2.1789	-1.0879	-0.5992	0.2179	0.5819	0.8916	1.689	3.163	-0.2579	54.0	46.0	1.2375	-2.0133	34
0.0416	0.0282	0.0891	0.0692	0.0516	0.0884	0.0763	0.0308	0.0573	0.0782	0.0235	0.0888	0.0116	0.0568	0.073	0.0525	0.0833	-4.2315	-2.6969	-1.9648	-1.023	0.2497	0.6384	1.6999	3.0501	5.3621	-0.1358	56.0	44.0	1.7791	-2.573	35
0.0138	0.0698	0.0643	0.0884	0.1022	0.0076	0.0327	0.0876	0.0788	0.0939	0.0569	0.0835	0.0247	0.0455	0.059	0.0278	0.0637	-4.4893	-3.3598	-2.3693	-1.1926	-0.2368	0.7009	1.5766	2.5134	5.4929	-0.4948	48.0	52.0	2.015	-2.8115	36
0.065	0.0849	0.0516	0.051	0.0461	0.0104	0.0739	0.045	0.0828	0.0486	0.0375	0.0676	0.0927	0.0625	0.0728	0.0529	0.0547	-4.4649	-2.2082	-1.0401	-0.3847	0.0435	0.4923	1.1782	3.9212	5.7189	0.0716	52.0	48.0	1.586	-1.5691	37
0.0546	0.063	0.0149	0.0887	0.0468	0.0429	0.0666	0.0671	0.0383	0.0778	0.0581	0.1001	0.0772	0.0413	0.0391	0.0384	0.0849	-4.462	-2.4735	-1.2945	-0.7623	-0.2368	0.1149	0.5627	1.0284	1.8624	-0.6018	48.0	52.0	0.8528	-1.9445	38
0.0618	0.0402	0.0809	0.0398	0.0961	0.0413	0.0537	0.0552	0.0863	0.0226	0.0927	0.0603	0.082	0.0581	0.0371	0.0399	0.0519	-3.8539	-2.9863	-1.5649	-1.0582	-0.6025	0.082	0.5627	1.2984	2.1236	-0.6779	46.0	54.0	1.2375	-2.3095	39
0.0774	0.0846	0.0958	0.0331	0.0521	0.0424	0.0912	0.0567	0.0411	0.0652	0.088	0.0408	0.0763	0.0526	0.0412	0.0275	0.0341	-5.2401	-3.6617	-2.4601	-1.3638	-0.0693	0.629	2.2278	4.2424	7.2739	0.1466	48.0	52.0	3.6849	-3.1194	40
0.0334	0.0789	0.0398	0.0679	0.0493	0.0371	0.057	0.0333	0.0696	0.0871	0.0391	0.0524	0.0708	0.0804	0.0897	0.0564	0.0578	-3.5009	-2.2445	-1.3466	-0.6326	0.1168	0.4862	0.9397	1.689	2.6701	-0.2492	52.0	48.0	1.3879	-2.0228	41
0.0601	0.0631	0.0705	0.0538	0.0526	0.0696	0.0384	0.0917	0.0793	0.0364	0.0409	0.0819	0.0708	0.0234	0.0516	0.0922	0.0237	-5.2401	-4.2071	-2.4601	-1.3638	0.0178	1.1129	3.3267	5.0867	8.3424	0.2147	50.0	50.0	3.944	-3.5145	42
0.0325	0.0776	0.0839	0.0789	0.0705	0.0767	0.0783	0.0145	0.0227	0.0321	0.0948	0.0732	0.0747	0.0485	0.0477	0.0625	0.0307	-3.8741	-2.9051	-1.9648	-0.7803	0.1168	0.5819	1.3339	2.0433	5.3621	-0.3525	52.0	48.0	1.6104	-2.4788	43
0.0368	0.0298	0.0222	0.0841	0.0215	0.091	0.0554	0.0647	0.0953	0.0836	0.0447	0.0906	0.034	0.0755	0.035	0.0937	0.0421	-3.7222	-2.7873	-1.3466	-0.6602	0.043	0.4862	1.2421	1.7887	4.5896	-0.2145	52.0	48.0	1.5014	-2.0734	44
0.0357	0.0844	0.0559	0.0903	0.0461	0.0539	0.0648	0.0967	0.0519	0.0117	0.0948	0.0587	0.0114	0.089	0.0423	0.0442	0.0681	-5.0649	-3.5374	-2.4601	-1.3638	0.2001	1.5119	3.2141	5.0867	8.3424	0.2676	54.0	46.0	3.4893	-3.5145	45
0.0367	0.0586	0.0375	0.0661	0.0727	0.0735	0.0248	0.0987	0.0504	0.0498	0.0763	0.0566	0.0266	0.0679	0.0839	0.042	0.078	-3.0888	-2.376	-1.2552	-0.6602	-0.2451	0.1484	0.6732	1.4885	2.0853	-0.2645	48.0	52.0	1.1126	-1.5358	46
0.0343	0.0468	0.1046	0.0815	0.0254	0.0758	0.0081	0.0647	0.0867	0.0355	0.082	0.0485	0.0537	0.0702	0.0121	0.0997	0.0706	-2.8329	-2.2614	-1.4241	-0.6285	0.1185	0.6313	1.6999	2.775	4.9941	0.1036	52.0	48.0	1.9735	-1.9222	47
0.0057	0.0436	0.0768	0.1016	0.0614	0.0204	0.0429	0.0658	0.0302	0.0967	0.0951	0.0248	0.0366	0.057	0.0933	0.0951	0.053	-4.0519	-2.2921	-1.4466	-0.1399	0.2896	1.4861	2.0093	3.9046	5.9416	0.3914	58.0	42.0	2.3091	-2.2568	48
0.0954	0.0406	0.0223	0.0686	0.0801	0.0679	0.0708	0.0626	0.0897	0.0279	0.0846	0.0775	0.0357	0.0243	0.0725	0.0285	0.051	-5.1466	-4.1949	-3.2414	-2.2854	-0.9178	0.2945	1.3477	3.3035	6.1429	-0.691	44.0	56.0	2.8831	-3.4992	49

Additionally, we employ multidimensional scaling to convert K-Means weights to a two dimensional representation and plot them.
Multidimensional Scaling K-Means Weights – Open in a new tab, place cursor on circles to display data, use the tools on the right side to enable zooming in and out, etc.

Our goal in this analysis is to explore a large possible number of weights for a given portfolio. We note that views can be expressed by specifying calibration data, data representation, simulation days and rebalancing, projection days, etc. Thus the emphasis is on exploration rather than optimization.