Because $3^4\equiv 1\pmod{5}$, therefore $3^{x+4}\equiv 3^x\pmod{5}$. Meanwhile, it is obvious that $x+5\equiv x\pmod{5}$. It follows that $$3^{(x+20)}+ (x+20)^2\equiv 3^x + x^2\pmod{5}$$

This means that we only need to examine positive integers not exceeding $20$ to get the result. There are only four possible values $2$, $4$, $16$, and $18$ satisfying the requirement. Therefore the answer to the given question is $4\times 5=\boxed{20}$.