The curve and the line intersects at the point $(x, y)=(4, 2)$. Hence the desired area should equal $$\int_0^4\sqrt{x}dx -\frac{1}{2}\times (4-2)\times 2$$

The first part represents the area below the curve $y=\sqrt{x}$ and the second part represents the triangle under the line $y=x-2$. The above equation can be calculated as $\boxed{\frac{10}{3}}$.