This work considers methods for uncertainty quantification and data-driven modeling using low-dimensional representations. Specifically, sparse polynomial approximations (or expansions) and active subspaces. Comprised of three main sections, we investigate sparse polynomial chaos expansions via compressed sensing and D-optimal designs, data-driven discovery of dynamical systems via robust project matrix optimization and D-optimal designs, and uncertainty quantification for electrical capacity expansion planning. D-optimal designs, because of their coherence properties, can be used to enhance estimation accuracy with compressed sensing algorithms. We prove that the quality of a D-optimal design can be improved if samples are drawn according to a coherence-optimal distribution. The greatest benefit from this approach is for high-order, low-dimensional models, particularly if the uncertain variables follow normal distributions. We show that accuracy can be improved by precoherencing with robust projection matrix optimization for under-determined linear systems with noise and L-curve optimization. We also demonstrated that D-optimal designs can be used to improve accuracy when dealing with an abundance of data, i.e., over-determined linear systems with noise, and that both robust projection matrix optimization and D-optimal designs can be used in conjunction for further benefit. We apply some of our methods for uncertainty quantification and data-driven modeling, by quantifying the uncertainty in output decisions of a large-scale electrical capacity expansion planning model in collaboration with the National Renewable Energy Laboratory.