Because $f(x)$ is a polynomial, both $f(x+1)$ and $f(x-1)$ will have the same degree of $f(x)$. Therefore $f(x)$ must be quadratic. Let's assume $f(x) = ax^2 + bx+c$. $$ \begin{array}{lclcl} f(x+1)&= a(x+1)^2 +b(x+1) +c &= ax^2 + (2a+b)x+(a+b+c)\\ f(x-1)&= a(x-1)^2 +b(x-1) +c &= ax^2 + (b-2a)x+(a-b+c) \end{array} $$ $$\implies f(x+1)+f(x-1)=2ax^2 + 2bx + 2(a+c)=2x^2-4x$$ This implies $$ \left\{ \begin{array}{cl} 2a &=2 \\ 2b &=-4\\ a+c&=0 \end{array} \right. \implies \left\{ \begin{array}{cl} a &=1 \\ b &=-2\\ c &=-1 \end{array} \right. \implies f(x) = \boxed{x^2 - 2x - 1} $$