The three pillars of optimization theory - theoretical and algorithmic foundation, familiarity with applications, and the ability to apply theory and algorithms to actual problems - are combined in this book, which rigorously and gradually builds the connection between theory, algorithms, applications, and implementation. Readers will find more than 170 exercises that deepen and enhance the reader's understanding of the topics. The author includes several subjects not typically found in optimization books, including optimality conditions in sparsity-constrained optimization, hidden convexity, and total least squares. It also offers a large number of applications, such as circle fitting, the Fermat-Weber problem, denoising, clustering, and orthogonal regression. Theoretical and algorithmic topics are demonstrated by the MATLAB toolbox CVX and a package of m-files posted on the book's website. This book is intended for students of mathematics, computer science, and engineering at the advanced undergraduate level plus.