Really do appreciate your long responses, so thank you and keep doing that.
More parameters than data points shows up a lot these days and not because of non-linearity. Now its common to measure everything you can about an object and then worry later about what is useful. So the situation is few objects but gobs and gobs of measurement for each of them.
Of course now you have a rank deficient situation with an entire affine space for a solution. But thats no good, application wants to find the sparsest point in that affine space in the given basis. ML has tricks up its sleeves to do that in poly time. Its quite unbelievable that this is even possible.
BTW little surprised that you dont talk about Lehman much.
And you are searching for quality in the wrong place. Go for the proceedings of the COLT conference, ICML and read some Vapnik to start.
More parameters than data points shows up a lot these days and not because of non-linearity. Now its common to measure everything you can about an object and then worry later about what is useful. So the situation is few objects but gobs and gobs of measurement for each of them.
Of course now you have a rank deficient situation with an entire affine space for a solution. But thats no good, application wants to find the sparsest point in that affine space in the given basis. ML has tricks up its sleeves to do that in poly time. Its quite unbelievable that this is even possible.
BTW little surprised that you dont talk about Lehman much.
And you are searching for quality in the wrong place. Go for the proceedings of the COLT conference, ICML and read some Vapnik to start.