Category: Home Network


I’ve noticed that each of the three iPhones I’ve owned over the last three years have a problem passing off to the 3G network when the WiFi signal is too weak to use. In other words, when the signal strength on the WiFi network is very low, the iPhone will still prefer WiFi to 3G, even if the WiFi connection is unusable. This happens every time I get to the far side of the house, but it also gets me when I’m on the bus and signal is coming and going, or when I’m walking around outside at work, and while I can still get a little of work’s network, I can’t use my phone without disabling WiFi.

I can’t fix the problem at work – I’m not in control of that – but I can fix things at home. Or so I had thought, with Apple’s mesh networking implementation in their Airport line of products. In previous iterations of my home networking setup, I’ve tried using an Airport Express to extend the network of an Airport Extreme. While it works, it’s not the most reliable thing, and as I have older Airport hardware, I can’t run simultaneous 802.11g and 802.11n networks while keeping things running at d decent speed. Finally, the cost to upgrade to the newest line of Apple networking products is substantial – the current generation Airport Express is $99, and the current generation Extreme is $179. Poking about online, I think I have the answer I need: Open Mesh

Specfically, the MR500 Mesh Router from Open Mesh. It’s $99 per unit, and cleverly runs 2 CPU’s – one for user traffic, and one for mesh traffic. It also uses both the 2.4 ghz band and the 5 ghz band – but the 5 ghz band is reserved for mesh traffic. This is all done in the name of enterprise grade reliability – the packets must keep flowing. The system also supports multiple gateway devices from the wired network to the wireless network, even ones on different ISPs, so failover between networks should be seamless. There is also the provision for per-user rate limiting, public and private networks, and self-restarting hardware if the software stack crashes. All for the $99 per-unit cost. It also has a built in 5 port 10/100 switch, so you can use it to bridge physical network segments.

The only downside to the system, as best I can tell, is that all the management and configuration of the devices is done on the Open Mesh website. They run a web app that you create an account on, and when the device powers it, it looks for it’s MAC address on the service, and configures itself accordingly. My concern comes in for times that my link to the internet is down, but I still want to stream movies and music inside the house. That is one example, and I’m sure others would have better ones, but it’s a concern. I need to do a little more digging and see if the device will just come back up under it’s old settings, assuming nothing should change if it can’t talk to it’s master. I’d feel a little more comfortable if I could run a local version of the cloud management application, but nonetheless, this is still worth looking at.

After doing a lot of reading online, I’ve finally built a new OpenAFS cell in my home network. This involved reading piles of terribly out-of-date documentation and HOWTO’s, and a lot of frustration on my part. I help administer OpenAFS at work, but I’d never set up a new cell before, and the process of working out the right Kerberos keys and setting up the servers with bos were throwing me for a loop.

Eventually, I found the guides hosted on spinlocksolutions.com, which have a very clear walkthrough, with up-to-date explanations on how to setup a Debian-based OpenAFS cell, as well as a Kerberos 5 domain and an OpenLDAP server. They can be used with Ubuntu Server almost without alteration. This is good, as I have a passing knowledge of Ubuntu from previous projects.

However, there were two gotchas I ran into when doing an install with Ubuntu Server:

  1. The Ubuntu installer had added my FQDN to /etc/hosts with an IP address of 127.0.2.1, which makes the afs-newcell script lose it’s mind when trying to create the CellServeDB entry it needs.
  2. The Ubuntu apt-get delivered copy of the afs-newcell perl script has an error in it, that it doesn’t add the -noauth flag when creating the dafs server. You will see errors about not having permissions to create the dafs server. Simply edit the script and add the ‘-noauth’ flag.

Once afs-newcell runs properly, everything else goes exactly according to the documentation – afs-rootcell creates things as it’s supposed to, and the various Debian/Ubuntu packages to configure PAM work as expected.

I’m a long time Aperture user. I’ve been in the habit of saving my Master images (be them .NEF, .TIFF, .PSD or .JPG) to secondary disks or NAS devices for years. This lets me keep my Aperture library, containing the metadata about the images, the preview thumbnails, and the project organization relatively small. (1.4 TB of images, 97 GB library). However, I’ve been running without a secondary file server for months now, which I was using for a backup of the primary image collection, and I’m not terribly comfortable without a reliable backup. CrashPlan is running on my machines at home, but literally 8 months remain until all my images are backed up to the cloud. I would also prefer to have backups at home.

This has lead me to do a bunch of research on distributed and replicated network file systems. The idea being that you buy a number of relatively cheap computers to be used as file server nodes, you join them in some sort of namespace, and then you let the servers act as redundant copies of each other in case one is down with hardware issues or needs to be patched, or whatever. The popular options seem to be the RedHat backed GlusterFS, the network file system side of Hadoop, and Lustre, the somewhat failed distributed file system that Sun purchased and Oracle seems to have killed off. In the developmental space, there is Ceph, which looks promising but really isn’t ready for prime time.

Of course, none of these support OS X natively, and have lots of their own internal problems. Hadoop can only be accessed over it’s JSON/REST calls, or via a FUSE plugin; Lustre development is dying and similarly needs to be shared via FUSE; GlusterFS is modern and supported, but you have to re-share the namespace via SMB or NFS if you want to use it on your Mac, and the syntax for growing pools of storage seem archaic. Ceph is dynamic, POSIX compliant, and apparently awesome, but it’s not done, and there’s again no native OS X support, so it’s FUSE or SMB/NFS time.

I don’t want to set up a cluster of nodes just to have the one that is serving over SMB die, and have all my connections get reset. I want to access the file systems natively, and in a way that Aperture can actually use the space in the way the file system designers intended. And, much to my surprise, it seems that all roads are pointing me to OpenAFS, which I just so happen to use at work.

To start, it doesn’t do some of what I want, which is the automatic server-to-server replication that you see in Ceph or GlusterFS. But apart from that, it seems to rock along nicely. Volumes can be moved between servers, or even partitions on servers, while they are in active use by clients. All authentication is done over Kerberos or IP ACLs. Backups are automatically run nightly, and with a little scripting, volumes can be balanced between nodes without much effort. But the best of all is that it’s “native” to OSX – there’s a kernel-level driver to install, sure, but once that’s done, the global AFS namespace shows up under /afs – and by global, it covers every publicly available AFS installation in the world, if you want it to. It’s been around for 25+ years, and it just works. There are robust clients for OSX, Windows, Linux and Solaris, which are all the machines I’ll ever run it on.

Even better is the idea that if I were to partition my photo repository properly, I could move lesser used volumes to nodes with larger, slower disks, and move recently used files to faster ones – perhaps SSDs. With a working knowledge of the AFS command set, perl and find, this could even be done somewhat automatically, where every 30 minutes or so “hot” volumes could be moved to faster storage.

A lot of this is still a pipe dream – I need to replace my desktop before I turn my eye to storage solutions – but planning is King, right? Comments or questions, please sound off in the comments.

I now have OpenSolaris (b151a), Mac OS X Lion (10.7.2) and netatalk v2.2.1 working together in harmony. What follows is a brief setup guide:

Download, build and install a current version of BerkleyDB. I used 5.2.36, the current version on Oracle’s website.

  1. cd /tmp
  2. tar xvzf db-5.2.36.tar.gz
  3. cd db-5.2.36/build_unix
  4. ../dist/configure
  5. make && pfexec make install

Next, install gettext, from the IPS repo. If you are still using OpenSolaris like I am, you will need to unset the opensolaris.org repo, as it no longer responds.

  1. pfexec pkg unset-publisher opensolaris.org
  2. pfexec pkg install gettext

Next, download build and install the current version of netatalk, using the bdb you installed earlier:

  1. cd /tmp
  2. tar xvf netatalk-2.2.1.tar
  3. cd netatalk-2.2.1
  4. pfexec ./configure --with-bdb=/usr/local/BerkeleyDB.5.2
  5. make && pfexec make install

You are almost done. Create your ZFS filesystem for AFP sharing, setup your config files, and fire up the daemons:

  1. pfexec create -o compress=on tank/timemachine
  2. pfexec chown -R username /tank/timemachine
  3. pfexec chmod -R u+rwx /tank/timemachine
  4. echo "/tank/timemachine \"Time Machine Backups\" allow:username options:tm" >> /usr/local/etc/netatalk/AppleVolumes.default
  5. pfexec /etc/init.d/netatalk start

Netatalk should fire up a cnid_metad process as well as an afpd process, and you can now connect to the OpenSolaris server with afp://hostname. Once it’s mounted, Lion can use the volume “Time Machine Backups” as a TimeMachine disk, as it runs the full suite of AFP3.3 commands, including the fast/force sync stuff to make sure the writes happen even if the network connection is severed, which allows TimeMachine to resume backups.

As a word of warning: DO NOT kill -9 THE afpd OR cnid_metad PROCESSES. Use SIGTERM or SIGHUP, or just use the init script. If you kill -9, you run the very real risk of killing the CNID DB backend, which keeps track of all the Apple specific file relations, which make things like aliases work. Also, try very hard not to use hard links in AFP volumes – it’s a world of pain waiting for you.

OpenSolaris, Lion and netatalk 2.2.1

So, I’m working on a netatalk build at home on my OpenSolaris boxes – both a personal project as well as a skill set builder for future opportunities. Thankfully, a lot of the pain of netatalk builds has been resolved in the 2.2 branch, and the 2.2.1 release claims to support Lion and TimeMachine. So, after 30 minutes of futzing around, I’ve got it built and installed, but I’m stuck (for the evening) on the TimeMachine flags – getting an error on the OS X side of things that I don’t have the proper “read, write and append permissions”, which is very much a netatalk config issue. When I’m a little more clear headed, perhaps over Thanksgiving weekend, I’ll see if I can built it the rest of the way. Or, I’ll turn to RHEL, which I know a lot better these days.