There are several ways to solve this.

The first method is to construct such a number: $\underbrace{333\cdots 33}_{2023}4$, i.e. $2023$ threes followed by $1$ four, fits the bill. (see # 4787.)

The second method is use the squeeze method. The claim is equivalent to showing that there exists a square number $N^2$ satisifying the following equation: $$\underbrace{111\cdots 1}_{2024} \times 10^k \le N^2 < \underbrace{111\cdots 1}_{2023}2\times 10^k $$

where $k$ is a non-negative integer. The above equation can be re-written as: $$\sqrt{\underbrace{111\cdots 1}_{2024} \times 10^k} \le N < \sqrt{ \underbrace{111\cdots 1}_{2023}2\times 10^k}$$

It is obvious that as long as the difference between $\sqrt{ \underbrace{111\cdots 1}_{2023}2\times 10^k}$ and $\sqrt{ \underbrace{111\cdots 1}_{2024}\times 10^k} $ is greater than $1$, there must exist an integer between these two values which leads to the desired result.

It is initutive to believe that as long as $k$ is sufficiently large, the difference between these two number can be greater than $1$. To show this, let $a=\underbrace{111\cdots 1}_{2024}$, our goal is to find an integer $m$ such that $$\sqrt{a+1}\times 10^{m} - \sqrt{a} \times 10^{m} > 1\Leftrightarrow 10^m > \frac{1}{\sqrt{a+1}-\sqrt{a}} \Leftrightarrow m > log_{10}\frac{1}{\sqrt{a+1}-\sqrt{a}}$$

There is an infinite number of $m$ satisfying this condition. For each such $m$, let $k=2m$ guarentees the existance of an integer $N$ meeting the requirement.