Previously, I was backing up my websites password-less ssh and rsync, but I have decided to change that for a couple of reasons such as passwordless ssh can have its security problems, and I dont trust hardware. Using S3 I can securely keep as many full backups as I’d like, not have to worry about hardware failing, all for pennies a month. Currently I backup my desktop with JungleDisk and thought I may as well consolidate all my backups to S3.
To accomplish this goal of backing up your website you will need 2 things, SSH access and s3sync which is a handy little S3 client written in ruby.
- Download and setup s3sync from http://s3sync.net/wiki. Once you untar the file open up s3config.yml (may be named s3config.sample.yml, just rename it) and this is where you set the path to your certficates, youAWS access, secret access keys, these can be found in “Access Identifiers” section of your AWS account.
- Now lets test this out. S3sync needs to know where its configuration file is so we just export the variable S3CONF with the path to your s3config.yml file.
[user@machine s3sync] export S3CONF=/home/bront1/s3sync
To make sure everything is working do a simple command to list your buckets.
[user@machine s3sync]$ ./s3cmd.rb listbuckets
bucket1
bucket2
bucket3
Yay we have s3 connectivity!
- Next we dump the databases, tar and gzip everything up.
[user@machine] mysqldump -u user -pPassword --all-databases > /home/user/backup.sql
[user@machine] tar -cf /home/user/backup.tar /home/user/public_html /home/user/backup.sql | gzip > /home/usr/backup.tar
- Copy the backup to your S3 bucket using s3sync’s s3cmd.rb
[user@machine s3sync] ./s3cmd.rb put bucket-name:folder/target_name.tar.gz /home/user/backup.tar.gz
- Success! You have now backed up your website and database to your S3 account.
S3sync has its own sort of built in rsync to make incremental backups, but I prefer to make keep my own daily backups. Heres the script I use to create daily backups of my websites and files. It creates a backup with day, month, and year in the filename and keeps backups for the last 10 days or so. (Note: The sed and awk commands are really messy due to my lack of sed/awk knowledge. This script is run everyday at 3am using a cron job with the output written to a log.
Crontab:
0 3 * * * export S3CONF=/home/user/s3sync; /home/user/backup.sh >> /home/user/backup.log
backup.sh
#!/bin/bash
export S3CONF=/home/user/s3sync
cd /home/user
TIMESTAMP=`date +%m%d%Y`
echo "$TIMESTAMP :: Backuping up the databases"
mysqldump -u user -pPassword --all-databases > /home/user/backup.sql
echo "$TIMESTAMP :: Bundling all the files up"
tar -cf /home/user/backup_`date +%m%d%Y`.tar public_html backup.sql
gzip -f /home/user/backup_`date +%m%d%Y`.tar
echo "$TIMESTAMP :: Copying backup to S3"
#we use full path because this script is running in a cron job
/usr/local/bin/ruby /home/user/s3sync/s3cmd.rb put bucket:folder/backup_`date +%m%d%Y`.tar.gz /home/user/backup_`date +%m%d%Y`.tar.gz
echo "$TIMESTAMP :: Cleaning up"
rm -f /home/user/backup_`date +%m%d%Y`.tar.gz
rm -f /home/user/backup.sql
echo "$TIMESTAMP :: Checking for old backups"
#check how many backups are saved
num=`/usr/local/bin/ruby /home/user/s3sync/s3cmd.rb list bucket:folder | wc -l`
#we save at least 10 days of backups
#12 is checked for due to other crap s3cmd prints out
if [ "$num" == "12" ]; then
echo "$TIMESTAMP :: Deleting old backup"
#i know there is a better way to check this, i just dont know how
last=`/usr/local/bin/ruby /home/user/s3sync/s3cmd.rb list bucket:folder | sed -e 's/-//g' | awk '{printf("%s", $0 (NR==1 ? "" : " "))}' | awk '{print $2}'`
/usr/local/bin/ruby /home/user/s3sync/s3cmd.rb delete bucket:$last
else
echo "$TIMESTAMP :: No old backup to delete"
fi
echo "$TIMESTAMP :: Done"
S3 is Amazon web service’s cloud storage solution offering very cheap price-per-gb storage and bandwidth which makes this a very cost effective backup solution. Currently I use JungleDisk for mounting S3 buckets as drives and to schedule backups, and s3sync to backup my files in various unix enviroments but I dont have anything in between to where I can easily manage my S3 files. That is where Cloudberry Explorer for Amazon S3 comes in. I have been using this for the past couple of hours or so and to say the least I am impressed.
Here is what I’ve noticed while using Cloudberry. 
- First off, this software is completely free. That is a huge plus because what I was using before, CrossFTP, you had to pay 25$ for the pro version to get S3 support and I had a whole slew of gripes about it.
- The UI is very clean and responsive.
- Tabbed! Everything is better with tabs, especially if you are dealing with multiple uploads and/or multiple S3 accounts
- Very easy to upload and manage files with the intent of serving them on the web
- Cloudfront integration
- The ability to bill the requestor for a file for the S3 usage. I messed around with this for a bit and I think its just a flag that is set on the file? Not sure how to implement it.
- Plugins to Microsoft Powershell
Though through using this there are a couple things that could be added that would be nice.
- In the “MyComputer” pane, shortcuts to Desktop, My Documents, etc would be nice.
- Also labels for each of the drives I have. I have so many drive labels I often forget which one is a HDD, network drive, or DVD drive.
- Refresh the S3 pane after I copy a file to a bucket so I can see that file there after it is finished copying.
Overall, this is a pretty solid S3 client and I plan to keep on using it.
Cloudberry Explorer for Amazon S3
Shared hosts are a great way to get started on the web whether you want to make a blog, start a forum, or you want to start developing your own applications, and did I mention shared hosting can be really cheap? That low monthly bill doesn’t come without a price, and no, I’m not talking about dollars and cents here, I’m talking about service. There is a huge trade off when getting such a service so cheaply. I see a lot of people who rely on these shared hosts for their businesses, client websites, and a lot of mission critical applications and it is just a flat out bad idea.
When not to choose a shared host:
- If you rely heavily on email
- If speed and reliability is important
- If you expect your data to be backed up
- If you expect tech support to know what they are talking about
- If your online business is important to you
If these sort of things are really matter to you then spending the extra 10-20$ a month for a reliable VPS. It may not have all the bells and whistles, but you get a lot more in return. It may not have all the bells and whistles a shared has offers with unlimited bandwidth and a billion other things, but if you are at the point to where one or more items in the above list affects you then than going this route should be worth it.
As for myself, I am sort of an uptime freak and obsessed with the health of my machines so I dont really mind spending a little extra money to fulfill these obsessions of mine. Even with that being said, there are still appropriate times when using a shared host makes the most sense.
- You want to start a blog/forum/photo album/etc
- You want to learn new web technologies (shared hosts are great to mess around with code)
- Cheap and great way to get started on the web
- If you want email with your domain
Both solutions have their pros and cons, I suggest looking at your needs and expectations and choose which way to go is right for you.
Photo credit: JohnSeb
A couple of weeks ago I setup iptables on my Slicehost VPS. After a few days of watching it I was suprised at how many connections were being denied. This machine has only been public for a couple days or so and I only had one domain pointing to it too. Its been a few weeks now and the information I have gathered is pretty interesting, take a look.
Top 10 countries based on number of denied hosts:
Top 10 ports based on number of denied connections:
Here is a mash up I made using the Google Maps API. This representation only shows around 500 of the denied hosts but I think its a good view of where all the connections come from throughout the world. Now to only AJAX‘ify this thing and have a live view of where the denied hosts are coming from in the world.
Biturl is still a very young website being officially announced on January 30th of 2009. I created this service for myself above anything because I was tired of these slow, bloated, url shortening services that in my opinion all just suck. My entire inspiration behind this project is “no bullshit”. All I want to do is enter a big url and get a tiny one in return. I really never expected anyone to use it, but a couple people on twitter have picked it up which led me to believe, maybe I got something here.
Here are some quick side by side comparisons of biturl’s site design and functionality versus the other url shortening services out there. All the clutter aka “bullshit” and unnecessary areas of the site are marked in red squigles.
Tinyurl
vs 
Bit.ly
vs 
Tr.im
vs 
Poprl
vs 
Hex.io
vs 
And probably the worst offender of all is Ow.ly. Their front page its very simple and I would like this service but….
when a url is forwarded they put this ridiculously obnoxious toolbar on top of the site you want to view. Invasive much?
Though there are a couple sites who’s design I do like such as is.gd and twurl. My intention with biturl isnt to attract users, or make money, or even really make a great service with a ton of features, it was to make a simple fast service, and to keep it that way. A lot of people think its just yet another url shrinker and it is to some extent, I just wanted to point out the differences in biturl versus the others.
There are certain parts of the day where we are the most productive. I consider these times of the day for me as being “in the zone”. This time of day occurs around 2-3am, this is when I feel total zen and that I am “at one” with my work. At this time is when all distractions are tuned out and I wont stop until I finish what I am working on. Only my keyboard, 4 or 5 
putty windows, and a browser have my full attention.
I’m a firm believer in working in our natural flows and rhythms, aka being in “the zone” or basically just working when you feel like it. When you feel like working or accomplishing a certain task, this is not a forced choice, its something you want to do. And when we do things we want to do, it makes us happy. If you are happy about working, then 1) you are happy about your work, and 2) the end result of your work is of higher quality.
Everyone has their own rythms and times of the day where they are most productive. I strongly encourage you to find yours whatever it may be, at 7am, 2am, or maybe you work best when you are hungry. A happy engineer is a good engineer.
Ah the fun of naming servers, but what on earth are you going name them and how are you going to come up with 5, 10, 100, 500 different names? At my previous job and at home I used BSG callsigns to name machines. Apollo, starbuck, hilo, boomer, hotdog, etc.
If you need some help naming machines, maybe these ideas will help you out.
Battlestar Galactica Callsigns
- Apollo
- Starbuck
- Helo
- Boomer
- Hotdog
- Athena
- Racetrack
Star Trek Characters
- Picard
- Riker
- Kirk
- Worf
- Data
- Spock
- Crusher
- Chakotay
- Tuvoc
- Dax
- Kaelas
- Odo
Simpsons Names
- Bart
- Lisa
- Homer
- Marge
- Maggie
- Moe
- Carl
- Lenny
- Barney
- Skinner
- Flanders
Rectal Foreign Bodies
This is the #1 upvoted and accepted answer via stackoverflow.
- Bottle
- Jar
- Lightbulb
- Tube
- Apple
- Bananna
- Icepick
- Flightlight
- Pen
If you need to name ALOT of machines, I suggest going for river names or mountain names. Almost endless amount of names you could use. Of course you could go with generated machine names, but actual names are easier to remember and a lot more fun.
[Via Stackoverflow]
I have created a AWS EC2 AMI with a couple source dedicated servers installed on it. Lately I’ve been using EC2 to start on demand game servers for whenever my friends and I want a private server to play on with the settings we all like.
This image is based on Amazon’s Fedora Core 8 32bit Official image. The latest HLDS update tool and Day of Defeat Source, Counter-Strike Source, and Team Fortress 2 games are installed with example server.cfg files for each game.
AMI Id: ami-7906e110
Manifest: fc8-srcds-01/image.manifest.xml
readme.txt:
This is the source dedicated server EC2 image.
Brought to you by Adam Bronte
Find updates and information at http://brontesaurus.com
All source ds servers are in /home/steam
Installed Games:
=================
* Team Fortress 2
* Counter-Strike Source
* Day of Defeat Source
Running Day of Defeat Source Server
========================
Dods Config Files: /home/steam/hlds/orangebox/dod/cfg
To start the server:
* cd /home/steam/hlds/orangebox
* ./srcds_run -game dod
Ex: ./srcds_run -game dod -secure +maxplayers 24 +exec server.cfg
Running Team Fortress 2 Server
========================
TF Config Files: /home/steam/hlds/orangebox/tf/cfg
To start the server:
* cd /home/steam/hlds/orangebox
* ./srcds_run -game tf
Ex: ./srcds_run -game tf -secure +maxplayers 24 +map cp_dustbowl +exec server.cfg
Running Counter-Strike Source Server
========================
CS:S Config Files: /home/steam/hlds/cstrike/cfg
To start the server:
* cd /home/steam/hlds
* ./srcds_run -game cstrike
Ex: ./srcds_run -game cstrike +maxplayers 24 +map de_dust +exec server.cfg
VPS hosting done right that is.
I am some-what hard to please, especially when it comes to hosting and technology in general. I hold web hosting companies to standards I would expect myself to meet if I were in their shoes. I dont want unexpected outages, unexplained server performance, random machine reboots, and probably my #1 pet peeve with hosting companies is often I feel like I monitor their servers better than they do. Is it really that hard for companies to tell their customers what is going on?
The other day I received a lovely text message from my pingdom monitoring of biturl (the site hosted on my slice) notifying me that it was down. Probably 30 seconds later I received an email from slicehost letting me know what’s going on.
The server your slice was on became unresponsive to our monitoring at 04:05:50 UTC and an emergency reboot was required.
Slices were restarted and are running, however we will be monitoring this machine for further trouble.
Please note this was a server issue and your Slice activity was not a causal factor.
We’ll notify you should further action be required.
We apologise for this downtime and please contact us if you have any questions.
Slicehost Support
[39001]
—-
Slicehost Support
support@slicehost.com
The downtime only lasted 10 mintues but it could of lasted an hour and I wouldn’t have minded. Instead of me thinking “WTF?! why is this down“, and going through the process of opening a ticket, talking to a support guy who doesn’t know anything, and never getting a definitive answer on what happened, they simply sent a courtesy email to me. Downtime is inevitable, but when it does happen I appreciate being notified on what is going on. It’s these little gestures that go a long way in a host and really sets them apart from other hosts.
Today marks the last day at my fulltime job. This economy has hit everyone pretty hard. We have already had one set of layoffs in the end of 2008 and just had a second round of layoffs in the begining of Febuary of 09. To say the least, it sucks. The company downsized a significant amount of people and I had to see some great friends/co-workers/engineers go. Fortunately for me I was given the option to stay on for another month to help with a big release we have been trying to get out. This gave me some to figure out what I was goign to do, update my resume, pay off any bills I had etc.
A lot of people in my place I know would be freaking out right now and frantically trying to get a new job in this economy, but I’m not too upset. I have felt “stuck” in my position there for quite some time and I was doing everything I could to possibly find a better fit for me within the company. The work that was given to me was very boring and none of my day to day tasks really challeneged me or made me think and overall I was very unhappy.
For me, getting laid off right now is a blessing in disguise. On one hand I’ll have to watch my budget a lot closer, but on the other hand I get to focus more on what I enjoy doing and attempt to make a living out of it. I am really looking forward to my time off.
0
Back to top
