Alternative to model selection, model combination gives a combined result from the individual candidate models to share their strengths. Yang (2001, 2004) proposed square-loss-based combining methods for regression analysis and forecast combinations. In this work, we propose robust combinations of statistical procedures. The theoretical properties of the robust combination methods are obtained, which show that the combined procedure automatically performs as well as the best one among the candidate models in estimation or prediction. Systematic simulations and data examples show that the robust methods outperform the square-loss-based combining methods when outliers are likely to occur and perform similarly to them when there are no outliers.