Wsrtjtyk

I have a polynomial equation in $x$ and $y$,

$$
(a-b)(xy+1) + (ab+1)(y-x) = 0.
$$
What transformation can act on $x$ and $y$ so that any point that satisfies this equation is mapped to another point that satisfies it? If there is a general way to find these transformations given a polynomial equation, that would also be nice to know.

You have $a,b,x,yinmathbb{R}$ ($mathbb{R}$ can be replaced by $mathbb{C}$ in every occurrence in this answer) such that
$$frac{a-b}{ab+1}=frac{x-y}{xy+1}.tag{*}$$
Let $c:=arctan(a)$, $d:=arctan(b)$, $u:=arctan(x)$, and $v:=arctan(y)$. Consider $c$, $d$, $u$, and $v$ as elements of $mathbb{R}/pimathbb{Z}$. Then, we have
$$tan(c-d)=tan(u-v),.$$
Consequently, $c-d=u-v$.

Now, any transformation $T:mathbb{R}/pimathbb{Z}tomathbb{R}/pimathbb{Z}$ of the form $$T(t)=t+qtext{ for all }tinmathbb{R}/pimathbb{Z},,$$ for some fixed constant $qinmathbb{R}/pimathbb{Z}$ satisfies
$$T(t_1)-T(t_2)=t_1-t_2,$$
for all $t_1,t_2inmathbb{R}/pimathbb{Z}$. Consider the function $f:mathbb{R}tomathbb{R}$ sending $$smapsto tanbig(arctan(s)+qbig)=frac{s+p}{1-sp}text{ for all }sinmathbb{R},,$$
where $qinmathbb{R}$ is a fixed constant and $p:=tan(q)$. Then, if $a,b,x,yinmathbb{R}$ satisfy (*), then
$$frac{a-b}{ab+1}=frac{f(x)-f(y)}{f(x),f(y)+1}$$ too.

In fact, only functions $f$ of the given form satisfy the requirement. Thus, all solutions $(x,y)inmathbb{R}timesmathbb{R}$ to (*) is of the form
$$(x,y)=left(frac{a+p}{1-ap},frac{b+p}{1-bp}right)$$
for some $pinmathbb{R}$ such that $pneqdfrac{1}{a}$ and $pneq dfrac{1}{b}$.

the substitutions
$$ x = u + frac{ab+1}{b-a} ; , ; $$
$$ y = v - frac{ab+1}{b-a} ; , ; $$
take us to the hyperbola
$$ uv = C $$
where $C = C(a,b)$ is a constant.

One motion, for real $t neq 0,$ is
$$ (u,v) mapsto left(tu, frac{v}{t} right) $$