simplifying
For those that have paid attention over the years, you'll notice I don't use the free services available for blogging, picture sharing, or email. To do this I run my own server and applications to host these things. While doing this gives me extreme abilities to customize things, it also takes my time to keep it all running. In the last couple years I've been looking at ways to maybe offload some of my work, even it meant paying for some services while I was at it. The only problem this leaves me with is the Terms of Use of these services. All the services out there pretty much state they guarantee nothing, can shut you off whenever they want, and won't promise a thing. Those are even the pay ones!
So, while I solve these problems both cost effectively and with a way to preserve my data, I will outline how I did it before and how I've changed it here for everyone to read.
The first is the ability to virtualize services around the Internet and make them all appear under my domain names. For instance, if you pull up this website (http://www.mikeb.org), or email me at my domain, you are interacting with some things I control to ultimately hit my servers.
To do this, I have named running on a VPS (virtual private server) at a company JVDS. My VPS is just a FreeBSD jail. I have that machine's IP registered as a name server in the root servers, and then slave servers at two different locations on an OpenBSD and FreeBSD machine owned by two different friends. Network Solutions and GoDaddy both now support free DNS hosting, so the very first thing I did was shut off named and move the domains to their respective registrar's name servers.
DNS was about the simplest thing I was managing, but it's one less thing I guess.
Next, I ran a web server on the VPS with a very simple virtual host config for each domain that looked like;
<VirtualHost *:80>
ServerName www.mikeb.org
ServerAlias mikeb.org *.mikeb.org
DocumentRoot /usr/local/www/data/mikeb.org
RedirectMatch ^/(.*)? http://www.disturbed.org/lifetype/blog/the-game-of-life
</VirtualHost>
That just simply pointed everyone going to the website to my home webserver where the blog was actually hosted. This functionality is offered for free at GoDaddy, but $12/yr at Network Solutions. So, I went to DynDNS and used their free WebHop service for the non-GoDaddy domains to duplicate it.
I was already using DynDNS to give the house a predictable address, so this was just another host to add to the list.
One thing to note, DynDNS offers URL cloaking where they don't change the URL in the user's web browser when loading the redirected site. You can use it for free with advertisements, or no ads for $11.50/yr. NetSol includes this in their $12/yr, and GoDaddy again does it for free. I added it to Lori's blog since her domain is GoDaddy hosted. Check it out in the sidebar under shiny things.
Apache turned off, named turned off, now came postfix.
I used to route email through the VPS to my house. Unfortunately most cable modem providers block in/out bound port 25, so the VPS had to route mail on a non-standard port. Thankfully the transport table in postfix does this easily with;
mikeb.org smtp:www.homeaddress.org:26
MX records were setup for primary delivery on the VPS, and backup MX on the other two machines. The VPS then had a virtual alias table forwarding mail to all the right places. In the case of my domains everything went to my primary email address, but other people have theirs forwarding to gmail, cox, yahoo and other places.
Once delivered to the house I could run spamassassin on the traffic, use procmail with some intelligence around active email addresses and spam level to filter even more, and then deliver locally.
Once it was delivered locally I could use imap to view the mail. I used dovecot as an imap server, along with Thunderbird and Squirrelmail to actually view the mail.
A few months back I changed my procmail to forward email to gmail at the same time it was delivered locally. This gave me a backup of all email in case gmail goes away. I setup gmail to recognize my email addresses and allow me to send mail as them instead of gmail itself.
Then once a month I would load up Thunderbird with both my local servers and gmail loaded via imap. In this setup I would copy Sent mail back to the local server. Unfortunately this still had a manual step, and I was still maintaining two mail servers.
So I've moved this to Google Apps now. I setup an account, proved I owned the domains, and then changed their MX records to Google. Unfortunately, after repointing all my email directly to Google, which in turn removed all my spam filtering, I was getting massive amounts of spam. Google does a good job putting these in the Spam folder, but it has way too many false positives for me to wade through that. To give you an idea; my domains receive over 55 pieces of email an hour (measured 24 hours a day over multiple weeks), and less than 2.5% of it is valid email.
So, for $3/yr, I signed up for Postini to filter my mail. That's where those numbers on mail/spam came from. Their web interface sucks compared to the rest of Google, but they are extremely good at filtering spam.
Now the last step is to backup the mail. Since I used to route the mail through the house, I then needed to download it from Google. For this, I'm using fetchmail with this config;
poll pop.gmail.com with proto POP3
user 'me@mydomain.org' options ssl fetchall mda '/usr/local/bin/procmail -Y -f %F .procmailrc.personal'
That .procmailrc.personal reads;
YEAR=`date +%Y`
:0
/usr/home/mikeb/.mail/personal/all-mail-$YEAR
I obviously had to enable POP in Google Apps, and then set it to leave the mail in the inbox. I am specifically delivering to procmail so I can just bypass using the local MTA and sort the all-mail per year without impacting other mail on the server.
So, there you have it, I've stopped paying for the VPS on JVDS at $15/mo and moved to Postini for $3/yr. That's saving $177/yr I can use for other things like shutting off the blogs and photo gallery software and moving them somewhere else.
I was able to stop maintaining postfix, apache, named and FreeBSD on the VPS. I could stop helping run named and postfix on my friend's servers. I also shut off dovecot, squirrelmail, holes in the firewall, and moved from postfix to the default FreeBSD sendmail install all at the house.
I already dealt with fetchmail and procmail for other email, so I really didn't add anything that I have to maintain.
Woohoo!
Do you really get SPAM that often that isn't already caught by the gmail SPAM filters? I have almost no spam ever. Been using it for a long while.
As for photos you can pick one of a couple of services (smugmug/flickr/picasaweb/etc) then just keep a local copy that is backed up at home too. You never have to worry about losing that stuff.
Now for the blogs I haven't found a good way to keep that backed up just yet. The biggest issue is a site that has good templates, plugin access, and being able to backup the blog inputed text so you can easily bring it back up. I guess if you have database access all you have to do is dump the database and you can restore it simply so maybe that is it. HMMM back to the drawing board so I don't have to pay for this hosting crap too.
I pay 30/mth for a nice VPS and a dedicated server.
My spam really was out of control. Postini has continued to be awesome for me. I also would end up with false positives in my Spam folder. The whole setup was pretty bad.
I do plan on using smugmug with iPhoto being the authoritative source for all data. Of course, my new Mac is being backed up via Time Machine over the network.
You and I were just IM'ing and poking around with WordPress.com/VPS WordPress installs and the backup options. It looks like that might be a good route for us too.