Let's first prime factorize $24= 2^3\times 3$. We note that $3$ can only stand by itself or multiply $2$ to become $6$. Therefore

• If one digit is $3$, then the other two digits must be either $(1, 8)$ or $(2, 4)$. Therefore $3! +3!=12$ possibilities.
• If one digit is $6$, then the other two digits must be either $(1,4)$ or $(2,2)$, Therefore $3!+3=9$ possibilities.

Hence, the answer is $12+9=\boxed{21}$.