By law if he can see it from his property, he can record it. you have no expectation of privacy, in places that are openly visible.
Why are you so worried? let him record, it is no different than if he watches them play. Now if he ever publishes them, minors with faces not blurred, than he would be breaking the law.
Tell him, you do not care, have a lawyer send him preserve the evidence letter, so he saves all the footage, and hands it over anytime you demand it, he can not delete it, it would be destruction of evidence, not all of it is true, but he most likely not gonna want to have these problems, and hire a lawyer to find out, and he'll move the camera himself.