In this paper, we first estimate the boundedness of a new proposed 4-dimensional (4D) hyper-chaotic system with complex dynamical behaviors. For this system, the ultimate bound set Ω1 and globally exponentially attractive set Ω2 are derived based on the optimization method, Lyapunov stability theory and comparison principle. Numerical simulations are presented to show the effectiveness of the method and the boundary regions. Then, to prove the existence of hyper-chaos, the hyper-chaotic dynamics of the 4D nonlinear system is investigated by means of topological horseshoe theory and numerical computation. Based on the algorithm for finding horseshoes in three-dimensional hyper-chaotic maps, we finally find a horseshoe with two-directional expansions in the 4D hyper-chaotic system, which can rigorously prove the existence of the hyper-chaos in theory.