Brontesaurus Blog

The night before I left to travel for the holidays, I was very excited to receive my beta invite for Liquidweb’s new cloud hosting platform, Storm on Demand. I have been in the beta invite queue since Liquidweb first announced this new venture. To say the least, my excitement has not yet been let down.

Since I first signed up, I’ve been able to play with everything and the best way I can describe it is it’s a mix between Rackspace cloud and Amazon EC2. Liquidweb strongly pushes their support and service as does Rackspace and also include a backup feature but have also taken note on VPS’s CPU and memory and bandwidth pricing from Amazon. Then of course, they have thrown a few features of their own in too.

These are some of the features I’ve run into so far:

• Instance Backups
  These are entire images of your Xen VPS. You can pay for backup space by the GB at $0.10/gb a month, or you can buy bulk backup space sort of like Amazon’s EBS starting at $10/mo for 100gb. Backups are executed daily and you can retain up to a maximum of 90 days of backups.
• Extra Instance Addons
  Storm on demand offers managed hosting for your virtual servers as well as CPanel and Fantastico pre-installed on your server.
• Ability to scale up
  This is something I think Rackspace got right and where Amazon is lacking. Sometimes you just need more memory or more CPU to a single instance. Scaling out is usually the right thing to do, but for small to medium sized deployments, that just adds extra headache.
• Easily create custom server images
  This can all be done through the management console very easily and quickly. You can also launch a new from your saved image. Pricing for storing server images is $0.15/per gb per mo
• Firewall
  Like Amazon’s EC2, you can create your own  firewall settings and apply them to each instance and have multiple different sets of firewall rules etc. This takes one less hassle out of setting up a new instance and dealing with iptables accross multiple instances.
• Virtual Networks
  I really like what Storm did here with the networking. It allows you to create virtual networks between instances. This is pretty useful because you can isolate different machines. On Rackspace cloud and EC2 (though I guess you can isolate them with VPC, but that is more of a VPN product), basically all the instances on your account are networked together.
• Better pricing for large instances
  This is where I think Storm on Demand is a great value. Hardware is getting dirt cheap, and the whole point of having your server in the cloud is to cut costs. I think the huge value here starts at the 8gb instances for $200/mo. On Rackspace this same setup costs $350/mo and on EC2 is costs $248.20/mo (standard on-demand).

When a instance is created it basically is assigned certain number of CPU’s based on the size of the instance. Each virtual CPU is a 2.2ghz Xeon. They don’t advertise this on the Storm on Demand front page, but is displayed when you create a new instance. 2gb instance has 1 VCPU, 4gb 2 VCPU’s, 8gb 4 VCPU’s, 16gb 4 VCPU’s, 32gb 8 VCPU’s.

The control panel is very slick and responsive.

No host is perfect, so I want to talk about some of the downsides I’ve run into so far.

Major Issues:

1. No API
   Seeing as this isn’t even available to the public yet, it’s not really a big deal. The API is actually done but is in testing now and will probably be released in a couple weeks.
2. No full DNS
   There is some sort of sudo DNS. All it seems to do is create an A record pointing to your instance IP. What about subdomains or MX records or CNAME records I possibly need for my apps and/or customers? This is something I really like about Rackspace’s cloud is that they give you full DNS support that you can do anything you want with.
3. Weird/confusing billing
   They state that all charges are hourly and “Any terminated services are charged only for the time they were active.” but on the bottom when you create a instance they state “Payment due today, for service now through 2010-01-15″. I think they charge you a pro-rated usage for the rest of the month because I have a $38 charge on my credit card from Liquidweb from what I’m assuming was from the first test instance I created. They charged me $38 but when those resources were canceled and weren’t used up, my account was credited the $38 for future usage. I have no idea why on earth they would decide to do something like this. It basically is false advertising with all their “pay as you go” stuff, when it’s clearly not pay as you go. I should be charged at the end of each month for resources used during that month.

Minor things:

1. Only 64-bit images
   You don’t really need 64-bit support unless you need > 4gb ram. Using 64-bit libraries also uses up a bit more memory. While this isn’t a deal breaker because you can install 32-bit libraries with ease, I would just like to see some other choices out there.
2. Instance names require a fully qualified domain name
   You can’t just name your instances one word, it has to be a full domain

Overall impression

I’m always excited to see new hosting technologies come out and see what competitors come up with next. I think Storm’s big advantages are their price’s of larger instances and the ease of use of some key features such as backups and creating server images. I’m also excited to see what their API is going to look like. My biggest peeve of Storm so far is why I was billed $38 for something that I’m suppose to pay for only what I use by the hour.

I’ve been debating back and forth for a while, weighing the pros and cons for a shared host vs. doing things myself. On one hand a shared host all the software and firewalls, etc are taken care of for you and its cheap but support isn’t that great. While on the other hand choosing a VPS, I have to manage everything myself and its a bit more money, but I have control over everything, it is much faster, and so far, Slicehost’s support is the best I have ever seen in a hosting company. So I said screw it, I’m migrating to my Slicehost VPS. I was planning to move to Mosso’s new cloud servers, but I have 6 months pre-paid so might as well use it to my advantage anyways.

Now I can proudly say I am hosted at Slicehost!

I have made a new pingdom monitor for this new host here or click the pingdom button at the bottom of the page. The data from my lunarpages host is still available. I am interested how these two compare in the upcoming months.

Previously, I was backing up my websites password-less ssh and rsync, but I have decided to change that for a couple of reasons such as passwordless ssh can have its security problems, and I dont trust hardware. Using S3 I can securely keep as many full backups as I’d like, not have to worry about hardware failing, all for pennies a month. Currently I backup my desktop with JungleDisk and thought I may as well consolidate all my backups to S3. 

To accomplish this goal of backing up your website you will need 2 things, SSH access and s3sync which is a handy little S3 client written in ruby.

1. Download and setup s3sync from http://s3sync.net/wiki. Once you untar the file open up s3config.yml (may be named s3config.sample.yml, just rename it) and this is where you set the path to your certficates, youAWS access, secret access keys, these can be found in “Access Identifiers” section of your AWS account.
2. Now lets test this out. S3sync needs to know where its configuration file is so we just export the variable S3CONF with the path to your s3config.yml file.

   [user@machine s3sync] export S3CONF=/home/bront1/s3sync

   To make sure everything is working do a simple command to list your buckets.

   [user@machine s3sync]$ ./s3cmd.rb listbuckets
   bucket1
   bucket2
   bucket3

   Yay we have s3 connectivity!

3. Next we dump the databases, tar and gzip everything up.

   [user@machine] mysqldump -u user -pPassword --all-databases > /home/user/backup.sql
   [user@machine] tar -cf /home/user/backup.tar /home/user/public_html /home/user/backup.sql | gzip > /home/usr/backup.tar

4. Copy the backup to your S3 bucket using s3sync’s s3cmd.rb

   [user@machine s3sync] ./s3cmd.rb put bucket-name:folder/target_name.tar.gz /home/user/backup.tar.gz

5. Success! You have now backed up your website and database to your S3 account.

S3sync has its own sort of built in rsync to make incremental backups, but I prefer to make keep my own daily backups. Heres the script I use to create daily backups of my websites and files. It creates a backup with day, month, and year in the filename and keeps backups for the last 10 days or so. (Note: The sed and awk commands are really messy due to my lack of sed/awk knowledge. This script is run everyday at 3am using a cron job with the output written to a log.

Crontab:

0 3 * * * export S3CONF=/home/user/s3sync; /home/user/backup.sh >> /home/user/backup.log

backup.sh

#!/bin/bash
 
export S3CONF=/home/user/s3sync
 
cd /home/user
 
TIMESTAMP=`date +%m%d%Y`
 
echo "$TIMESTAMP :: Backuping up the databases"
mysqldump -u user -pPassword --all-databases > /home/user/backup.sql
 
echo "$TIMESTAMP :: Bundling all the files up"
 
tar -cf /home/user/backup_`date +%m%d%Y`.tar public_html backup.sql
gzip -f /home/user/backup_`date +%m%d%Y`.tar
 
echo "$TIMESTAMP :: Copying backup to S3"
 
#we use full path because this script is running in a cron job
/usr/local/bin/ruby /home/user/s3sync/s3cmd.rb put bucket:folder/backup_`date +%m%d%Y`.tar.gz /home/user/backup_`date +%m%d%Y`.tar.gz
 
echo "$TIMESTAMP :: Cleaning up"
rm -f /home/user/backup_`date +%m%d%Y`.tar.gz
rm -f /home/user/backup.sql
 
echo "$TIMESTAMP :: Checking for old backups"
 
#check how many backups are saved
num=`/usr/local/bin/ruby /home/user/s3sync/s3cmd.rb list bucket:folder | wc -l`
 
#we save at least 10 days of backups
#13 is checked for due to other crap s3cmd prints out
if [ "$num" == "13" ]; then
  echo "$TIMESTAMP :: Deleting old backup"
 
  #i know there is a better way to check this, i just dont know how
  last=`/usr/local/bin/ruby /home/user/s3sync/s3cmd.rb list bucket:folder | sed -e 's/-//g' | awk '{printf("%s", $0 (NR==1 ? "" : " "))}' | awk '{print $2}'`
 
  /usr/local/bin/ruby /home/user/s3sync/s3cmd.rb delete bucket:$last
else
  echo "$TIMESTAMP :: No old backup to delete"
fi
 
echo "$TIMESTAMP :: Done"

Shared hosts are a great way to get started on the web whether you want to make a blog, start a forum, or you want to start developing your own applications, and did I mention shared hosting can be really cheap? That low monthly bill doesn’t come without a price, and no, I’m not talking about dollars and cents here, I’m talking about service. There is a huge trade off  when getting such a service so cheaply. I see a lot of people who rely on these shared hosts for their businesses, client websites, and a lot of mission critical applications and it is just a flat out bad idea.

When not to choose a shared host:

• If you rely heavily on email
• If speed and reliability is important
• If you expect your data to be backed up
• If you expect tech support to know what they are talking about
• If your online business is important to you

If these sort of things are really matter to you then spending the extra 10-20$ a month for a reliable VPS. It may not have all the bells and whistles, but you get a lot more in return. It may not have all the bells and whistles a shared has offers with unlimited bandwidth and a billion other things, but if you are at the point to where one or more items in the above list affects you then than going this route should be worth it.

As for myself, I am sort of an uptime freak and obsessed with the health of my machines so I dont really mind spending a little extra money to fulfill these obsessions of mine. Even with that being said, there are still appropriate times when using a shared host makes the most sense.

• You want to start a blog/forum/photo album/etc
• You want to learn new web technologies (shared hosts are great to mess around with code)
• Cheap and great way to get started on the web
• If you want email with your domain

Both solutions have their pros and cons, I suggest looking at your needs and expectations and choose which way to go is right for you.

Photo credit: JohnSeb

A couple of weeks ago I setup iptables on my Slicehost VPS. After a few days of watching it  I was suprised at how many connections were being denied. This machine has only been public for a couple days or so and I only had one domain pointing to it too. Its been a few weeks now and the information I have gathered is pretty interesting, take a look.

Top 10 countries based on number of denied hosts:

Top 10 ports based on number of denied connections:

Here is a mash up I made using the Google Maps API. This representation only shows around 500 of the denied hosts but I think its a good view of where all the connections come from throughout the world. Now to only AJAX‘ify this thing and have a live view of where the denied hosts are coming from in the world.

VPS hosting done right that is.

I am some-what hard to please, especially when it comes to hosting and technology in general. I hold web hosting companies to standards I would expect myself to meet if I were in their shoes. I dont want unexpected outages, unexplained server performance, random machine reboots, and probably my #1 pet peeve with hosting companies is often I feel like I monitor their servers better than they do. Is it really that hard for companies to tell their customers what is going on? 

The other day I received a lovely text message from my pingdom monitoring of biturl (the site hosted on my slice) notifying me that it was down. Probably 30 seconds later I received an email from slicehost letting me know what’s going on.

   The server your slice was on became unresponsive to our monitoring at 04:05:50 UTC and an emergency reboot was required.

   Slices were restarted and are running, however we will be monitoring this machine for further trouble.

   Please note this was a server issue and your Slice activity was not a causal factor.

   We’ll notify you should further action be required.

   We apologise for this downtime and please contact us if you have any questions.

Slicehost Support

[39001]
—-
Slicehost Support
support@slicehost.com

The downtime only lasted 10 mintues but it could of lasted an hour and I wouldn’t have minded. Instead of me thinking “WTF?! why is this down“, and going through the process of opening a ticket, talking to a support guy who doesn’t know anything, and never getting a definitive answer on what happened, they simply sent a courtesy email to me. Downtime is inevitable, but when it does happen I appreciate being notified on what is going on. It’s these little gestures that go a long way in a host and really sets them apart from other hosts.

Amazon EC2 just rolled out a new sudo pre-payment plans for all its instance sizes. With this new plan you pay a flat up front fee for a one or three year term. In exchange for this pre-payment you receive discounted per-hour price. A lot of people are using EC2 as a permanent web host and dont need to scale up like other services with possibly hundreds of instances being created or terminated at any given time. So what Amazon did here was give their consistent users a break with this new pricing structure.

This is a table with the pre-payment fees and what discount you get with them.

Actual costs. Pre-payment 1/3 year vs none.

Currently the reserved instance pricing is only available for Linux/Unix instances.

[Via Techcrunch]

A couple months back I signed up for the GoDadddy Grid Host beta. It was too great of a deal to pass up, 5$/mo with 100gb of space and 1TB of bandwidth (basically what MediaTemple gives you).

For this review I set up a test blog that syndicated around 20 different feeds. 37k posts and 2.7k comments later, I think I have a good base site to test this service with.

It has been 4 months, and so far my impression is “meh”. It seems to be plagued with the common problems that all grid hosting is struck with, one being mysql as the bottle neck. Everything I ran that did not require  a database back end was lightning fast, significantly faster than any shared host out there.

Here are the results of a benchmark I ran using a script that would test strings, dates, file system i/o, encryption, dates, images, arrays, and objects.

Ever wonder who owns/hosts some of the most popular blogs? Well I put together a list for you incase you are wondering. This list is taken from Technorati’s top 100 blogs page and I used whoishostingthis.com to find out who is hosting a particular blog.

The top 100 changes a lot so this may not reflect actual standings.

1. Huffington Post – Datagram
2. Engadget (Webblogs Inc) – America Online
3. TechCrunch – Media Temple
4. Gizmodo (Gawker Media) – Datagram
5. Boing Boing – Priority Colo
6. Official Google Blog – Blogspot (Google)
7. Lifehacker (Gawker Media) – Datagram
8. Ars Technica – CNET Networks
9. Daily Kos: State of the Nation (Ars Technica) – Voxel
10. Smashing Magazine – Net Build GmbH
11. BloggingStocks (Webblogs Inc) – America Online
12. Mashable – Rackspace
13. Gadling (Webblogs Inc) – America Online
14. Parent Dish (Webblogs Inc) – America Online
15. Seth’s Blog – Typepad
16. Postsecret – Blogspot (Google)
17. TMZ (Webblogs Inc) – America Online
18. ReadWriteWeb – Media Temple
19. Cinematical (Webblogs Inc) – America Online
20. Joystiq (Webblogs Inc) – America Online
21. Gawker (Gawker Media) – Datagram
22. I Can Has Cheezburger – Wordpress
23. TV Squad (Webblogs Inc) – America Online
24. The Unofficial Apple Weblog (Webblogs Inc) – America Online
25. CNN Political Ticker – Layered Tech
26. Perez Hilton – Voxel
27. Download Squad (Webblogs Inc) – America Online
28. The Daily Dish – Six Apart
29. Wired: Threat Level – Akamai
30. The Caucus – New York Times
31. Slashfood (Webblogs Inc) – America Online
32. Wired: Gadget Lab – Akamai
33. Gigazine – K-Opticom
34. TreeHugger – Pair Networks
35. Autoblog (Webblogs Inc) – America Online
36. Wired Science – Akamai
37. The Pioneer Woman Ree Drummond – Liquid Web
38. Think Progress – Reliance Globalcom
39. Talking Points Memo – Rackspace
40. Kotaku (Gawker Media) – Datagram
41. WSJ Washington Wire – Akamai
42. VentureBeat – Virtual Networks
43. Michelle Malkin – Diamond Alignment
44. The Corner – National Review
45. dooce – SourceDNS
46. Luxist (Webblogs Inc) – America Online
47. Epicenter – Akamai
48. Consumerist – (Gawker Media) – Datagram
49. The Sartorialist – Blogspot (Google)
50. Ben Smith’s Blog – Irides LLC
51. Parjamas Media – Joyenet
52. ProBlogger Blog Tips – Server Beach
53. Bits – New York Times
54. The Underwire – Akamai
55. Hot Air >> Top Picks – Diamond Alignment
56. Crunch Gear – Media Temple
57. Apartment Therapy – Slice Host
58. Danger Room – Akamai
59. Neatorama – Softlayer
60. GigaOM – Wordpress
61. News Busters – Razor
62. Crooks and Liars – KLOS Technologies
63. Blog di Beppe Grillo – ITnet S.P.A
64. Google Blogoscoped – Germany
65. Political Radar – Six Apart
66. Gothamist (Gawker Media) – Datagram
67. How to Change the World – Six Apart
68. Geekdad – Akamai
69. Salon: Glenn Greenwald – Raging Wire
70. Daily Blogging Tips – CyberLynk
71. Seeking Alpha – 9D Interactive
72. Spinner.com (Webblogs Inc) – America Online
73. Chrisbrogan.com – Local Launch
74. Engadget Mobile (Webblogs Inc) – America Online
75. Freakonomics – New York Times
76. Microsiervos – Media Temple
77. kottke.org – The Planet
78. Zen Habits – Slice Host
79. Valleywag (Gawker Media) – Datagram
80. MAKE Magazine – IH Networks
81. Popwatch – Six Apart
82. Yanko Design – Liquid Web
83. /Film – Media Temple
84. Mac Rumors – Softlayer
85. Marginal Revolution – ?
86. Scobelizer – Layered Tech
87. Stereogum – Tier Zero
88. Celebrity Baby Blog – Layered Tech
89. Defamer (Gawker Media) – Datagram
90. Deadspin (Gawker Media) – Datagram
91. Smitten Kitchen – California Polo Store, LLC
92. Just Jared – Tier Zero
93. Copyblogger – HCOTP LLC
94. Global Voices Online – RimuHosting
95. Wired: Game|Life: – Akamai
96. Pharyngula -Rackspace
97. Gred Mankiw’s Blog – Blogspot (Google)
98. Power Line – Softlayer
99. delicious:days – Domain Factory
100. Jalopnik (Gawker Media) – Datagram

Blogs I think should be on the list:

1. Torrent Freak – NoZone
2. Coding Horror – Crytal Tech

I am always shopping around for new cool online services and hosting. Yesterday I was looking at Godaddy (because Diggnation drills Godaddy into my brain) and I saw that they are offering a new grid hosting plan. I am kind of anal about website up time so I decided to take a look. Currently their grid hosting plan is in beta with a ridiculously low price of of 5$/mo with minimum of a 3 month commitment. The plan comes with 100gb of space, 1TB of storage, and unlimited CPU cycles. This was just too good of a deal for me to pass up and at least try out. Grid hosting can be kind of expensive and I have seen  some bad reviews and up time regarding some popular grid hosting companies, like MediaTemple.

Current Plan Details:

• Windows/Linux
• PHP/10x MySQL/ASP/1x MSSQL
• 100gb Storage
• 1TB bandwidth
• Unlimited CPU cycles
• 50 FTP Users
• 100 Email Addresses
• 100mb Total Email Storage
• Total DNS control
• 25 Sub domains
• Unlimited Addon Domains

If you are really looking for grid hosting it usually means you have 1 high traffic site or a couple high traffic sites. I know 99% of the hosts out there offer OMGUNLIMITED everything but I really would rather have stability and scalability over bells and whistles. I have hosted with godaddy years ago but eventually left because they some what restrict you and what you can do and I found better offers out there that could suit my needs better. But I have noticed they have gone through a lot of changes and opened things up a little more, but still not as much as others.

Pros

• Cheap grid hosting (so far, depends on what the final price is)
• Reliable (from personal experience and they guarantee 99.9% up time)
• Fast
• Total DNS control (big plus esp if you want custom DNS records)
• Control panel has gone through a lot of upgrades, I really like the disk space/bandwidth graphs
• MySQL backup/restore function (they didn’t have this back when I use to host with them, I actually used it this time and it worked flawlessly)

Cons

• Only unsecured FTP access to upload files
• No SSH (i am a command line junky, let me effin edit the file on the server instead of copying shit back and forth)
• The control panel still sucks, they have like 5 different panels for each product, cant it all just be in one FFS?
• They are missing some features like perl, ruby, python but they are apparently in the works.
• No MySQL command line access

The cons are pretty big for me but I am hoping that the stability will trump the cons.

I set up what to help me test this service is basically a huge blog spam site taking in around 20 or so RSS feeds. This is wordpress 2.7 with no caching and constantly updating. In about 2 days I already have 1500+ posts. I will post updates as I get time to benchmark and mess around with this a little more.

Test Blog