Recommendation systems have become an integral part of our everyday lives. Although there have been many works focusing on recommendation quality, many real-world aspects of the recommendation process are typically overlooked: How can we ensure that the very top recommendations users see are engaging? How to recommend venues matching user interests, while preventing many users from being directed to the same venue? Can we design recommenders which first converse with users and then give a recommendation? What is the best way to model recommendation systems as interactive systems, while learning on-the-fly the user-item structure? To what extent can a malicious party perform machine learned adversarial attacks against a recommender? The goal of this thesis is to pave the way towards the next generation of recommendation systems tackling such real-world challenges to improve the user experience, while giving good recommendations. This thesis, bridging techniques from machine learning, optimization, and real-world insights, introduces novel tools to address the above questions. We focus on three directions: (1) encoding real-world constraints into the objective functions, (2) learning to interact with users, and (3) modeling machine learned fake users with malicious goals. For the first direction, by adjusting the optimization objective to capture real-world constraints---(1a) the screen space is small, creating the need for the top recommendations to be relevant, (1b) the item capacities are limited---we suitably guide the learning of model parameters. For the second direction, to balance the need to explore users' preferences with the desire to exploit what has been learned, at a large user and item scale, we combine interactive learning techniques with the principle that similar users tend to behave similarly. This combination results in novel recommendation systems that learn to (2a) converse with new users, and (2b) collaboratively interact with users. For the third direction, taking the perspective of an adversary of the recommender, we use machine learning to learn fake user profiles, which are indistinguishable from real ones, while having a malicious goal. Illustrating the vulnerability of modern recommenders to machine learned attacks will arguably create new directions for designing robust recommendation systems against such attacks.