As you mention file manager works, its not your site.
Maybe check the ftp setting in your program?
I used to see this now and then, and after rebooting my computer, it would work.
So I switched to a different ftp program (CoreFTP lite)
and have not seen this again.
I suspect my settings became corrupt after an unscheduled shutdown (power failure).