This text presents a collection of mathematical exercises with the aim of guiding readers to study topics in statistical physics, equilibrium thermodynamics, information theory, and their various connections.  It explores essential tools from linear algebra, elementary functional analysis, and probability theory in detail and demonstrates their applications in topics such as entropy, machine learning, error-correcting codes, and quantum channels.  The theory of communication and signal theory are also in the background, and many exercises have been chosen from the theory of wavelets and machine learning.  Exercises are selected from a number of different domains, both theoretical and more applied.  Notes and other remarks provide motivation for the exercises, and hints and full solutions are given for many.  For senior undergraduate and beginning graduate students majoring in mathematics, physics, or engineering, this text will serve as a valuable guide as theymove on to more advanced work.