Jump to content

How To Synchronize Files To A Shared Host


Recommended Posts

The Problem


The ONLY protocol supported by TCH shared hosts is FTP. (Not counting the cPanel file manager and such.)


(Just so you know... FTP is insecure. Nothing is encrypted. Usernames, passwords, commands, and most data are sent in plain text so any packet sniffer can figure them out.)


Ok, so we know that FTP is insecure and inefficient, but we want to use it any way (what choice do we have?).


We've installed a web server on our local computer and designed and tested our site, at least to the point where we can put it on the internet so some other people can check it out. The normal way to get our files on the server is FileZilla (free for Windows, Mac, and Linux). Just copy the local files to the server by drag-n-drop.


But let's say some people have checked out the site and tweaked files on the server. If that's all that's changed, we could just copy all the files from the server (it might take a while). What if we've continued working on the site locally, so we have changes both locally and on the remote site. Do we want to go through each folder and find all the files that are different and figure out which one is really the newest? Not really. Even worse, what if some files contain changes both locally and on the server? FileZilla is not the tool for this.


We need a way to synchronize files both ways and resolve conflicts when both sides have been changed.


Windows Tools


On Windows, it looks like WinSCP *could* work for this (I haven't actually tried it this way). You can also use a method in Windows similar to the one I'll be discussing for Linux below... Unison works on Windows. And you can mount an FTP server as a windows drive using FTPUse (ferrobackup.com/ftpuse/)


For Development Within A Linux VirtualBox (On Windows)


I use Windows as my main computer, because Windows still has a lot more programs and such available than Linux. I use Linux (I'm using a Debian-based distro) in a VirtualBox for web development, because a lot of web server stuff is just simpler to install in Linux (or not even available for Windows... just to contradict myself).


I have not figured out a very good way of reading VirtualBox disk images in Windows, so I do the synchronization from within Linux. I had lots of trouble getting FTP to work reliably until I changed from the NAT mode (the default) to Bridged mode. So the first thing to do is change Adapter 1. For some details....

  1. Run VirtualBox
  2. Select the Linux virtual machine that you use (I'm assuming you have one installed... if not, I'm leaving it as an exercise for the student to install one)
  3. Click the toolbar icon to open the virtual machine's settings
  4. Select Network
  5. Enable the adapter if it's not enabled. Change the "Attached to" setting from NAT to Bridged Adapter. For "Name", select the network adapter your computer uses. I set my promiscuous mode to "Allow all", but it might not be necessary.
  6. Start the virtual machine.

Linux Tools


On Linux, it looks like csync could be used to sync to an FTP site, but I couldn't get it to work, and it's not really being developed any more. A lot of the Linux methods require the remote side to be running the same software as the local side. But shared hosts don't have that option, so we're stuck.


Or are we? Enter curlftpfs. This program mounts a remote folder onto the local file system, so we can use it just like local files. This opens a lot of doors. To get curlftpfs working (I'll show the commandline stuff because it's more universal)...

  1. Install using Synaptic or some other package manager, or enter
    >> sudo apt-get install curlftpfs

  2. Create the folder where you want to mount the remote folder. The normal place for mounting things in Linux is /mnt/ (or perhaps /media/). Your folder name doesn't have to be the same as the site, but it makes I think it makes it more clear what it is.
    >> sudo mkdir /mnt/mysite.com

  3. Now we need to do the actual mounting. TCH ftp usernames are something like 'myftpusername@mysite.com'. The '@' makes this tricky and we need to encode it using %40 as below.
    >> sudo curlftpfs -o allow_other myftpusername%40mysite.com:mypassword@ftp.mysite.com /mnt/mysite.com

  4. So if you change to that folder and list the files, you should see them
    >> cd /mnt/mysite.com
                       > ls -l

  5. If you want to mount this more easily (step 3 is kind of long), add a line to /etc/fstab, like this
    >> echo "curlftpfs#myftpuser%40mysite.com:maypassword@ftp.mysite.com /mnt/mysite.com fuse allow_other,rw,user,noauto 0 0" | sudo tee -a /etc/fstab

    and then to mount, you just need to

    >> sudo mount /mnt/mysite.com

So now, we have the ftp site psuedo-local, it's just a matter of syncing from one folder to another. There are a whole bunch of ways to do this. One might think 'rsync', but actually, it's really only for one-way syncing. I am currently using Unison for the two-way sync.

  1. Install Unison. Meld here is optional; it's for comparing the local and the remote version of the same file against each other. You can use some other diff/merge program if you want.
    > > sudo apt-get-install unity meld

  2. Add a profile to the settings (we'll name the profile mysite.com)
    >> sudo nano ~/.unity/mysite.com.prf

    Enter the following. Change as needed.

    ># roots indicate which folders are being synced. The first one appears on the left# pane of the unison GUI and the second one appears on the right pane.
                       root = /path/to/my/local/folder
                       root = /mnt/mysite.com
                       # what program to use (and how to start it) for comparisons when the same file
                       # has been changed in both the local and remote folder
                       diff = /usr/bin/meld CURRENT1 CURRENT2
                       merge = Name * -> meld CURRENT1 CURRENT2
                       confirmmerge = true

There is an excellent write-up about Unison usage here (granneman.com/techinfo/security/backup/unisonbackup/), so that's all I'm going to say about it.

Link to comment
Share on other sites

Well, that's interesting. Here's what a TCH tech told me in a recent support ticket: "there is no other protocol to transfer files from local to the server than FTP - for shared server accounts."


Semantics aside (whether SFTP aka FTP over SSH is the same protocol as FTP), either one is still pretty inefficient.

Edited by drewkeller
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...