Rewrite $2021^n$ as $(2020+1)^n$. It is safe to discard any term in  expanded form of $(2020+1)^n$ where the power of $2020$ is greater or equal to $4$ because such term will be a multiple of $10^4$. Therefore, $$\begin{align*} &2021^{2020}\ = (1+ 2020)^{2020} \\ \\ & \equiv 1 + \binom{2020}{1}\cdot 2020 + \binom{2020}{2}\cdot 2020^2 + \binom{2020}{3}\cdot 2020^3 \pmod{10^4} \end{align*}$$

Meanwhile, $$\begin{align*}\binom{2020}{1}\cdot 2020 & \equiv 2020^2 \equiv 20^2 \equiv 400 &\pmod{10^4} \\ \\ \binom{2020}{2}\cdot 2020^2 & \equiv \displaystyle\frac{2020\times 2019}{2}\times 2020^2 \equiv 6000 &\pmod{10^4}\\ \\ \binom{2020}{3}\cdot 2020^3 &\equiv (\cdots)\times 2020^4 \equiv 0 &\pmod{10^4}\end{align*}$$

Setting these to the previous relation gives $$1+ 400 + 6000 + 0 \equiv \boxed{6401}\pmod{10^4}$$