Let $w$, $r$, and $y$ be the number of white, red, and yellow balls which are retrieved. Then, the desired answer is the non-negative integer solutions to the following equation $$w+r+y=3$$

subject to the conditions that $0\le w\le 2$, $0\le r\le 3$, and $0\le y\le 1$.

One way to solve such a problem is to use the generating function. In this case, the result is the coefficient of $x^3$ in the expanded form of the following polynomial: $$(1+x+x^2)(1+x+x^2+x^3)(1+x)$$

The answer is $\boxed{6}$.

As the involved numbers are relative small, this problem can also be solved by casework.

• If no yellow ball is retrieved, there will be $3$ possibilities depending on the number of white balls retrieved.
• If $1$ yellow ball is retrieved, there will be again $3$ possibilities depending on the number of white balls retrieved.

Hence, the final answer is $3+3=\boxed{6}$.