There exist $5$ consecutive positive integers such that their sum is a square, and the sum of the middle three is a cube. What is the smallest one of these five numbers?