Chuck Burt’s Blog

This is an idea that has been scratching at me for quite some time. The biggest reason: I need to find something like this!

The problem

Everybody has growing file collections. Some of us definitely have more than others—my photo collection alone is reaching 80 GB! A single recording session can easily reach 20 GB. Acquired media aside (purchased/downloaded music, movies, etc—as it’s replaceable), everyone has a need for an elegant solution to back up this data.

There are a number of products and services popping up that attempt a solution to this problem, but none do it quite right. I have to give Apple a lot of credit here—they figured out a way to make backup so simple, the masses can finally do the right thing! Time Machine is not perfect for everyone, but it’s a huge step in the right direction.

The purpose of this post is not to discuss software solutions, but rather the hardware that accompanies it. While the right software is what gets people backing up, it always requires the right hardware¹ to make it work.

The current solutions

Online

Online services (aka “cloud” storage) allow users to upload their data to online servers. Some of these include backup software and all tout the benefits of being failure-safe and secure.

Pros

• Safe: Redundantly hosted on large server farms so the data itself is safe and secure.
• Portable: You can usually access your data anywhere you have a net connection.
• Sharing: Many of these services let you share certain folders with colleagues, friends, etc.
• Flexible/Scaleable: If the service has storage limits (some are unlimited), you can scale your costs with your direct backup needs.

Cons

• Speed: Backing up a lot of data online is constrained to internet speeds, and residential upload speeds are typically 2Mbps or less. This makes it very poor for users with large data needs and any time a user needs to access some of the data there is a big lag time.
• Software: A number of these solutions lock you into using their software. Often, their software is limited in big ways, e.g. not being able to backup external hard drives (unless they remain attached all the time).

Hardware

Hardware solutions consist of external hard-drives, file servers, and more intelligent products such as the Drobo (which is extremely cool).

Pros

• Fast: Accessing your data through a wired connection—or over a local wireless network—is much faster than through the internet. This makes it ideal for large amounts of data and all around much more accessible.
• Secure: You have absolute control of the security settings, so it’s as secure as you want to make it.

Cons

• Redundancy: Unless you are savvy enough to be able set up a RAID solution, or shell out for a Drobo, making external hard-drives redundant is a real pain.
• Flexibility/Scaleability: With the exception of the Drobo none of these solutions making scaling very easy.
• Physical Security: External hard drives, file-servers, and the like can all be easily stolen if you ever experience a break-in. Server farms often have very tight security: id badges, fingerprint/retina scanners, armed escorts, and all kinds of crazy cool precautions.
• Disaster:² Electronics are extremely susceptible to everything that comes with a house fire: heat, smoke, water, and chemicals. Backing up off-site is always a good idea.

Summary

The cons of both online and hardware solutions exclude any one solution from being ideal for me. I am a heavy data user that wants fast access and redundant, off-site backup. There is nothing that easily and elegantly fills this need. Sure, I could purchase “cloud” storage and manually back things up, but that requires quite a lot of time. I want something I can set and forget in the Time Machine sense.

The ideal solution

This leads me to what I think is the ideal solution to meet all of these needs. There would definitely be challenges in doing it right, but I think the need would make the investment worthwhile.

The Hardware

First, there is a hardware device that acts as the primary solution. This hardware device needs to redundant, easy to use, and smart.

• Like the Drobo³, it must be easy to scale with hot-swappable drive bays and it needs to “take care” of making the data redundant.
• It needs a variety of physical access methods: USB 2, FireWire 400/800+, eSATA, and Gigabit Ethernet.
• It must be cross-platform and therefore accept a variety of protocols such as AFP, Samba, and NFS, and advertise itself using Bonjour and Avahi. Along the same lines, it must be compatible with any file-system thrown at it, such as NTFS, HFS+, and ext3, or even ZFS, WinFS, and ext4.⁴
• It requires an easy-to-use and cross-platform compatible user-interface. A small web application would probably be best.
• Finally, it needs rsync set up as a nightly process.

The Cloud

Behind the hardware device, sits the ever-present protection of an online backup solution. It’s only for the event of a truly catastrophic event.

• It must be inexpensive. It doesn’t need lots of sharing features (though they are nice) and it needs to be priced in such a way that the user is ok with the on-going expense. Probably no more than $10/month, if that.
• It must be scaleable. Since the hardware component must be easy to scale, the online service must be able to keep up, with little or no direct effort from the user.
• It must be absolutely secure. When it comes to security, perhaps nothing is truly absolute, but it must be as secure as possible so that people can trust it. In addition, the data must be physically secure: it should be stored in more than one geographic location so it is impervious to natural disasters.
• It can’t have weird limitations such as a maximum file size of 5 GB That said, perhaps a little intelligence on the hardware side could get around a limitation like that…
• It must be stable and have very limited downtime.

Wrap-up

A solution like this—done correctly—would solve backup problems for some time to come. It gives users access to data quickly, redundant local backup, and off-site protection. The beauty is that the software is irrelevent: users can use any software solution they like, e.g. Time Machine, SuperDuper!, Retrospect, FlyBack, or anything else.

It might not be a product that everyone in the world would need, but it would be great for power users, professional or prosumer photographers, and just about anyone else that doesn’t have needs quite as high as video editor. Most importantly for me, I would buy one.

¹ In this case, “hardware” can mean online backup services as well.
² As someone that has woken up in the middle of a house fire and lived to tell about it, this is a bit of an ever-present threat in the back of my mind. A fire is a terrible thing, but losing all of your photos on top of it is icing on the suck cake. Plus, you never, ever, want to worry about grabbing things like that when evacuating. We are fortunate we live in a time where we can store things like photos off-site and still have access to them!
³ I swear, I’m not being paid for this. That said, offers are always welcome.
⁴ Obviously, it should be as forward-looking as possible. That said, if any potential future file-systems have specific hardware requirements that are unrealistic, they would have to be considered on a case-by-case basis.

Nice.mp3

I have always had random product ideas. My father once told me that he had seen many product ideas he’d had turn out to be real products 5-10 years later. I have the same affliction, however since many of my ideas are internet related, it’s often only 6 months to a year before I find the idea in some form of application.

I suppose a lot of people have ideas, but what sets the entrepreneur apart from everyone else is the knowledge, motivation, and courage to follow through on one of them–and stick with it until it either fails or succeeds. The fact that I’ve seen so many of my ideas turn out to be real later means I’m either very plugged in, or perhaps a bit unoriginal, but either way I have them and very briefly get excited about them.

I am going to blog about these ideas.

The first reason to do this is that it gives me an outlet for the ideas. I’m hoping that an outlet will help me put them to rest as my mind often obsesses on what it feels is the next big thing. Right now, I want to stay focused where I am.

The second is a bit of hope that the idea is so unoriginal, it actually already exists and I just haven’t heard about it yet. As a side to this, it might help those entrepreneurs out there trying to decide whether to go for it. If I’ve blogged about it, it’s something I would use so here is at least one consumer!

Finally, and by far the most self-serving of the bunch, I get a bit of personal gratification from having “called” an idea well ahead of time. Like the idea of Political Base over a year and a half before they launched. Or Fire Eagle a good 6 months before they were even announced (though many others had similar ideas too).

One final note: while most of the ideas are things that I really think would make a good product, there will be a few splashed in that are humorous only. Like the ‘Spaint and Soyda ideas I helped dream up in college. I’ll let you decide which ideas are serious.

Um.mp3

Tipjoy is a neat new company that aims to make it easy for people to reward content producers. The premise is simple: content producers embed the tipjoy button on their website (as I have done on the left) and any person that clicks it agrees to “tip” 10 cents. Alternately, a surfer can use tipjoy’s bookmarklet to tip any website–even if the site owner has not signed up.

The idea is profoundly different than any model that exists now. This is what makes the idea so neat, but is also their biggest challenge.

Presently, the vast majority of independent financial gain that comes from publishing content is based on advertising. Generally speaking, this advertising is a good thing (though I may never admit I said that) for a few reasons: it increases brand awareness, it takes financial burden off of the end-user, and most importantly it allows the content producer to keep going.

Without delving too deeply into the world of web advertising, there are typically three things for which a content producer might get paid:

• Impressions (ie, how many times the ad is viewed)
• Clicks (ie, how many times the ad is, erm, clicked)
• Actions (ie, how many times a user that clicked that ad went on to buy something, or sign up for a subscription service)

Different forms of advertising tend to reward based on different actions. If the advertiser is paying based on actions only (such as most affiliate programs), the content producer receives no financial gains unless their users spend money on the advertiser’s product(s) directly–or to a lesser extent if the user gives up their personal information to the advertiser to become a lead.

Tipjoy’s service fills an interesting void by allowing an end-user to reward a content producer much more directly. Under the advertising model, a user could click on an advertisement, but it’s very difficult for users to know whether this even helps the content producer. If the advertiser is paying for impressions, then it can be argued that the user’s presence on the site is reward, but it’s still very difficult to know whether this is the case. By giving users the option to more directly reward good content, it can inform the content producer of the true perceived value of their content–and thus their work–rather than a much less tangible number such as number of visits.

The inherent challenge of this is that it relies on users to pay for things they would otherwise get for free. If a user tips one great blog post per day, this adds up to $36.50/year that s/he otherwise would not have had to pay. Arguably, the user probably received much more value than that from all of that great content, but it’s a problem of motivation. It requires the content consumers to have a sense of altruism in their day-to-day surfing.

Due to this challenge, I don’t know that Tipjoy would work as a sole form of income. I do think that it works nicely beside already existing forms of income and also for content producers that don’t look to make a living off of their content, using more of a “donation” style of model. Whether this will generate enough revenue for Tipjoy to stay in business remains to be seen.

In the meantime, those of you that enjoy my content–and feel ever-so-slightly altruistic–feel free to leave me a tip. To my fellow content producers, watch out as you may have some tips coming your way.

As Robert A. Henlein wrote through his character Lazarus Long, “If tempted by something that feels ‘altruistic,’ examine your motives and root out that self-deception. Then, if you still want to do it, wallow in it.”

Dreamhost is awful. They were always a bit slow, but hey, they are cheap (cough, inexpensive).

As you probably know if you’ve visited my web site, one of the primary reasons I keep a site is to have a place to upload my photos. Sure, I could use Flickr, or Picasa Web Albums, or even SmugMug like Elizabeth does.

The issue of image ownership and usage rights aside, the bottom line is that I’m a control-freak. I like doing things my way. This means that I want full control over the display, linking, searching, protection, and usage of my photos. Though the set up of my photo gallery is far from perfect, it’s my own.¹ I’d eventually like to build my own photo gallery software, but that is a bit speculative right now.

Back to the point, on my previous host my photo gallery crawled. In a large collection of photos, say over 100–which is small by my photographic tendencies–it would often take 10+ seconds to load a single page. At first, I thought it was just because of the sheer number of photos. When I paired down the galleries, I noticed that it was still quite slow. It was almost embarrassing to send out links to something I should take pride in–a gallery of wedding photos for a friend for example! The performance would get even worse with more users.

As I had done a number of customizations to the gallery software, I immediately assumed I had made some mistake somewhere that was causing slow performance. Though I can hack my way through code and make customizations, I’m a product manager, not a developer. Thus, I set about doing some tests. The first test was to remove all customizations from the code–I reverted it back to an “out of the box” installation. There was a slight performance increase from doing this, but the site was still very slow. I then slowly added back my customizations one by one, first my overall appearance, then my custom breadcrumbs, then Google Analytics. Finally, I added back the (rather large) scripts that controls the “Lightbox” effect. Ah ha! There was a noticeable performance decrease in using these scripts.

My first assumption was that it was purely the size of the scripts at play. Thus, I set about one more test: I disabled the Lightbox scripts, but left an exorbitantly large quantity of unused scripts embedded in the page. My theory was that if the size of the scripts were at play, there would be little to no performance increase. I was wrong. Clearly, something else was at play here. At the time, I probably should have also realized that file transfer from my site had always been speedy–something anyone that had downloaded a straight file (ie, music mix-tape) from my site could attest.

This left me one conclusion: the extra database queries required to make the Lightbox effect work were slowing things down.

Sometime later, I realized that it was not just my gallery that had performance problems. After installing a blog, a wiki, a project management tool, and other miscellaneous open source projects and scripts, I realized every application that depends on the database was slow. Mind-numbingly slow. Dare I say, dial-up slow!?

Finally I bit the bullet and contacted Dreamhost’s customer service. I always prefer to figure things out on my own rather than dealing with the hassle of customer service channels–I like the learning experiences involved–so this was a rather big deal. After a quick response, they immediately tried to place blame on the software, on me, on anyone other than themselves really. They even had the audacity to try to up sell me! I figure that as a web operator that has a “peak” of 50 users in a day across ALL sites, there was no reason I should be too big to fit in a shared hosting package.

Dreamhost has grown too fast for their own good.

Why not, they are offering one of the cheapest packages in hosting around? I finally decided to bite the bullet and switch providers. After a fairly exhausting search, I settled on GeekStorage. They don’t offer nearly as much space or bandwidth for the price, but I like working with the owners. I worked with them 5 years ago and up until they were acquired by a less than reputable company.

The experience with GeekStorage has been pleasant (barring one customer service snafu with a friend I recommended them to). Migrating websites is not a fun task, but the head geeks at GeekStorage have been accommodating and helpful. The performance increases alone have been well worth it! Their pricing is also not bad. Not nearly as much space/bandwidth as some other hosts, but I value performance and working with people I trust much more. Plus, at those rates they are not likely to oversell their resources.

So please, enjoy the improved performance around here. Browse around some photos without wanting to poke your eyeballs out with a big stick. And watch for some improvements to the site coming in the next few months.

www.songstowearpantsto.com/

We're having a superbowl party, woohoo!

While the preceding fact is true, I'm mainly testing Pownce's Event functionality.

What

Superbowl Party

Where

My place

When

Sun, February 3 2008 at 5:30 p.m.

As anyone that follows my Google Reader Shared Items knows, I enjoy sharing a lot of news from TechCrunch.

This one is absolutely preposterous. Sony BMG has announced that they will now [finally] be selling DRM free music. This makes them the last of the “big four” to wake up and smell the coffee.

With one small catch. In order to purchase said music, you must go to a store, choose the album you want, buy a special card for $12.99, bring it home, log on to a special website, and voila!… you may now download your DRM free album.

At that point, why wouldn’t you just buy the album on CD where you get the uncompressed full version? If you have a computer with internet access, it’s not much of a stretch to presume you would have access to a CD drive that can rip your music. The major reason I haven’t hopped on the digital music bandwagon is the compression issue.

This begs the question, “Do record companies really wonder why people are compelled to steal music?” If they continue to make it this difficult to legally purchase digital music, nobody is going to start buying any time soon.