This article studies the distributed optimization problem for multi-agent systems with communication delays and external disturbances in a directed network. Firstly, a distributed optimization algorithm is proposed based on the internal model principle in which the internal model term can effectively compensate for external environmental disturbances. Secondly, the relationship between the optimal solution and the equilibrium point of the system is discussed through the properties of the Laplacian matrix and graph theory. Some sufficient conditions are derived by using the Lyapunov–Razumikhin theory, which ensures all agents asymptotically reach the optimal value of the distributed optimization problem. Moreover, an aperiodic sampled-data control protocol is proposed, which can be well transformed into the proposed time-varying delay protocol and analyzed by using the Lyapunov–Razumikhin theory. Finally, an example is given to verify the effectiveness of the results.