update: In early 2016, Resilio was spun out of BitTorrent to bring distributed technology to the enterprise. This is awesome news and I’ll be posting some updates about what Resilio is up to moving forward. Below is my initial post from 2013 that was syndicated on the Bittorrent Sync blog.
What is BitTorrent Sync?
The concept is simple, using a local client on your desktop or laptop Sync will synchronize the contents of the selected folder to other remote Sync clients sharing the same key. Synchronization is done securely via an encrypted (AES) bittorrent session. This ends up being effective for moving a lot of data across multiple devices and while I think it was initially designed for secure private dropbox style replication, I’ve been testing this as an alternative method of geo-replication between glusterfs clusters on Fedora.
Right off the bat there were a few things that got my gears turning:
- a known and proven P2P protocol (monthly BitTorrent users are estimated at something insane like a quarter of a billion users)
- encrypted transfers
- multi platform
- KISS oriented configuration
What is GlusterFS?
GlusterFS is an open source project leveraging commodity hardware and the network to create scale-out, fault tolerant, distributed and replicated NAS solutions that are flexible and highly available. It supports native clients, NFS, CIFS, HTTP, FTP, WebDAV and other protocols. (more info here)
GlusterFS has native Geo Replication. Why not use it?
Leveraging the native GlusterFS georeplication for a single volume is a one way street today. I’m not sure if this is something that will change moving forward but today, a volume replicated is configured in a traditional Master/Slave configuration.
In addition to simple failover configurations, it can also be configured for cascading configurations that allow for more interesting archival type configurations.
While I’m sure this works for replication and certain disaster recovery scenarios, I’m looking at multi master configurations, or with multiple datacenter configurations that are all “hot”, possibly removing the need for a centralized repository. I’d also like a scenario that allows for all sites to serve as DR locations for any other participant while leveraging the closest cluster as a data end point for writing. Something that looks a bit more like this…
This type of configuration also allows for a more easily grown environment and a quick way to bring another site online.
Leveraging bittorrent, one of the more interesting features is the optional use of a tracker service. This helps with peer discovery, letting the tracker announce SHA2(secret):IP:port to help peers connect directly. They tracker service also serves acts as a STUN server, helping with NAT traversal for peers that can’t directly see each other behind firewalls. This is interesting for environments where you don’t want to have to deal with reconfiguration of firewalls. It’s important to note thought that even leveraging the tracker service all transmission of data is encrypted in flight.
For quick testing, find a couple of boxes that you want to get replication moving between. These could be minimal install linux boxes, SAMBA servers for your SMB, webservers (backup replication?), or in my case, a single node of a gluster cluster. If you’re interested in getting started with gluster, here’s a good place to get started.
a quick note if you’re using gluster: On one of the nodes, make sure the glusterfs client is installed. Create a directory and using the glusterfs client mount the volume you want to have replicated. There are more complicated ways to do this, but for testing, this will work fine.
Download the client
First we’ll need to untar the download and get some config files ready to go. Additionally, we’ll want to build an init.d script to ensure the client is running on startup. You don’t have to do all of this, but I wanted to have it available to manage as a service.
We’ll want to move the binary to a better location
Next, create a directory for the configuration and generated storage files
We should identify or create the directory we want to use as a replication target as well. As an example I’ll create a new directory….
With our directories created and in place it’s time to generate the initial config file and edit it appropriately.
Using your favorite text editor, edit the following lines…
As we’re going to identify the replicated folders via the conf file it’s important to note that the webui that is normally available for the linux client will be disabled. First thing you’ll need to do is generate a “secret” that you’re going to use for your share. from the command line:
will give you a secret you can use,but I find it easier to just go ahead and dump the secret at the bottom of the conf file I’m going to use and just move it around from there.
In the shared folder section look for the following line:
and replace MY_SECRET_1 with the secret you generated. As an example:
you’ll want to change the directory line as well…
In the shared folders section either edit or comment out the known host section. The easiest thing is to comment out the examples provided. Change…
IMPORTANT: You’ll need to remove the leading /* and trailing */ of the shared folders section.
With the config file set start bittorrent sync using the config set.
sync init script
I’m by no means claiming this is a work of art. It gets the job done though. you’ll want to create a file /etc/init.d/btsync with the following content:
Testing the sync service out
With that done you’ll want to change the mode of that file to 755. This will allow it to be run as a service.
and ensure it’s run at startup:
Other nodes and additional thoughts
With the above in place you’ll want to configure additional btsync clients on gluster nodes (or whatever test system you’re using) at your remote locations using the same secret you used above. The mount point / local folder can be different, but the secret must be the same. This will allow for replication to start amongst the identified folders. Thanks for reading and check out other cool usescases for bittorrent sync on the bittorent sync forums.