Distributed cooperative control of multi-agent systems is broadly applied in artificial intelligence in which time delay is of great concern because of its ubiquitous. This paper considers the controllability of leader-follower multi-agent systems with input and communication delays. For the first-order systems with input delay, neighbor-based protocol is adopted to realize the interactions among agents, yielding a system with delay existed in state and control input. New notions of interval controllability and interval structural controllability for the system are defined. Algebraic criterion is established for interval controllability, and graph-theoretic interpretation is put forward for the interval structural controllability. Results imply that input delay of the multi-agent systems has significant influence on the interval controllability and interval structural controllability. Corresponding conclusions are generalized to the first-order systems and the high-order ones with communication delays, respectively. Example is attached to illustrate the work.