Upgrade Path from Mediasmart Home Server to Something Stronger?

I've got another question for you guys, hoping you can help. Thanks again for all the free tech support by the way

Anyway, I'm currently running an old HP Mediasmart as a home server. I use it to centrally store my files, backup the machines in the house (at the image level), and do a small amount of media streaming. It's old, really old. I upgraded both the RAM and CPU a few years back, and have steadily swapped the drives out for larger ones (I'm up to about 2TB of space now). But it's starting to show its age, and I'm running out of space again.

Considering I'm now running a business off of this thing as well as housing my music files, I figured it might be time to consider upgrading.

So I'm looking to do a few things.

  • Store personal and business files.
  • Protect against disk failure with some form of RAID like functionality.
  • Stream media files to the Xbox.
  • Backup/Image the desktops and laptops in the house.
  • Provide remote access to the files if possible.

It would be nice if the system was capable of doing an offsite backup as well, but I can use a service like LogMeIn Backp or something for that if needed.

This Western Digital Sentinel is what I'm looking at so far.

Do you guys have any suggestions for something better/different to do what I'm after?

I figure I'll install this or whatever else you guys talk me into in conjunction with a complete refresh of my main desktop PC as well. I currently have a whole host of scheduled SyncToy backups going back and forth between the desktop and the server, and blowing the whole thing away and starting fresh seems like a good idea.

Presumably, you'd rather not build it yourself, you'd rather just buy something?

Does it need to be just storage, or do you want it to be a computer, too?

edit: okay, you want streaming, which is a form of computer functionality -- do you want anything else that requires a program?

And, for remote access, what's your intended use pattern? Just to grab your files on a laptop, or are you going to be doing it a lot?

I can build it myself if that's the way to go, I just figured that with server licenses and whatnot a pre-built would be as cost effective.

Storage is the main function, but I would like it to handle some computer functionality too. For instance I'm currently running LogMeIn Hamachi as a service on my WHS, along with PlayOn and a few other streaming services. I might not need Hamachi if this unit has built in remote access, but better safe than sorry.

For remote access, I'm currently working from three separate locations - primary and two satellites - on desktops in all three. I've been keeping my most used files in Dropbox so I have access to them in all locations, as I pretty much use them daily. I'd prefer to be able to keep them on the server and work on them from any of the locations. I set Hamachi up for that purpose, and it works, but it is atrociously slow - which is why I switched back to Dropbox.

EDIT: I just signed up for Crashplan to take advantage of the Carbonite switch offer. I see that you can't actually back up network share drives, which I didn't realize. That being the case I'll definitely need whatever hardware solution I go with to have computer functionality, as it looks like I'll need to run Crashplan locally on that machine.

I have a Synology Diskstation
http://www.newegg.com/Product/Product.aspx?Item=N82E16822108065

The model i use is not being sold anymore, but they have newer models. I paid like $200 for it.

Pros
I can stream music to the playstation.
I can stream media to my nexus 7 tablet.
I have two 1tb drives in it. One drive is a mirror image of the other. This is automatic.
I had this for about 3 years and have had no problems.
I have setup access levels so my devices can only access some files.
Pretty easy to use
Has a bunch of other features

Cons
Slow saving anything on it
Uses its own file system, might be linux. If the device fails its going to be a pain getting to my files.

At work, we just put in a set of Synology Diskstation DS411+II (4x3TB in each, about 9GB usable space with a failover drive in place), one for primary use, the other to backup the first one to. They also support backing up to Amazon S3.

We've gotten good read/write from them, the OS has been a pleasure, and altogether, we spent about $1200 a pop on them.

They support VPN, FTP, plus they have the ability for easy package installs (media sharing, virus protection, etc).

Was quite easy to setup for our needs, but we're just doing file sharing, auto-backups, and the MySQL database for internal use.

I would highly recommend looking into something like this.

Synology looks pretty good, and I like the ability to use any drive off the street in it, that's a plus over the WD Sentinel, which requires proprietary drives.

The WD Sentinel on the other hand is running Windows, with which I'm more familiar.

A bit of research seems to indicate that I can install Crashplan on either system with a bit of tweaking. And they both have a VPN-like remote access solution, and stream media.

So it looks like I'm at $834 for the 6TB Western Digital Sentinel, or $950 for the Synology 412+ and two 3TB drives. Is there anything I'm missing in the comparison?

I would highly recommend you pick a device that allows you to have a failed drive that puts the device in a degraded state as opposed to crashing it and hosing your data, it's well worth the investment.

That's definitely my goal. I have that now with the WHS since I have duplication turned on, and I don't want to take a step back in functionality.

Both the WD Sentinel and the Synology use RAID of some stripe, so my limited understanding is that both of them accomplish what you're saying. Is that not correct?

Only certain raid setups accomplish that. Some are build for mirroring data, some for speed via striping it across drives, and there are other setups as well. I recommend a quick glance at Wikipedia for a better understanding.

I completely expect Malor or someone to give a better clarification of it, though.

There are a zillion flavors of RAID, but only two or three that you'd actually want.

  • RAID-0 is striping a volume across disks, with no redundancy. This lets you join smaller disks into one larger volume, but if either disk fails, you lose all data. Worse than a single disk. Avoid this.
  • RAID-1 mirrors a volume across two or more disks; the same data gets written on all the volumes. This is hardly ever more than two disks, but once in a great while you'll see more disks in a RAID-1. You have to lose all disks to lose your data.
  • RAID-0+1 stripes two disks and then mirrors the stripe. With 4 disks, this is identical to RAID-10 in terms of overall reliability and loss of space, but honestly, it's just a bad idea, and you should never use it. Always use RAID-10 instead.
  • RAID-10 sets up mirrors of disks (normally pairs), and then stripes the mirrors. This is better than RAID-0+1, because both drives in a given mirror have to fail for you to lose the array. With RAID-0+1, the loss of any single disk on each side of the array will kill the array. So, say you've got ten drives, with RAID-0+1 you'd have two stripes of five, mirrored -- lose one drive in each stripe, and you're toast. With RAID-10, you have five mirrors, striped, and only if both disks in the same mirror fail do you lose everything. RAID-10 is also probably the fastest overall RAID stripe method, but you get only half your total drive space.
  • RAID-5 sets aside one disk for parity. If you've got five drives, you end up with the storage space of four drives, and if any one of them fails, the fifth can be reconstructed from the other four. If you lose a second drive, before you reconstruct a new one, you lose the array.
  • RAID-6 is just like RAID 5, but it uses two disks for parity, so you have to use six disks to get the space of four. In exchange, you can survive the loss of two drives without downtime.

I think you already know this from your phrasing, but for other readers: RAID IS NOT A BACKUP. RAID prevents downtime from drive failure. It can accidentally function as sort of a backup if you happen to lose a drive, but there are a zillion other ways to lose data, from controller failure to software bugs to simple fat-fingering. Only real backups protect you from this kind of data loss. You have a much, MUCH higher chance of just deleting data you shouldn't have than you do of losing a drive.

Typically, most people want either RAID-10 (for speed), or RAID-5 or 6 for space efficiency. The parity calculations slow down drive access a fair bit. Linux software RAID is pretty goddamn amazing, however, and if you're willing to use Linux, you can do software RAID-5 or 6 that runs lickety-split.

If you build the machine yourself, you can almost certainly do it cheaper than something like a Synology. There are three main options for OS; Linux, FreeNAS, and Windows.

With Linux, you'll probably need to drive it from the command line. The Linux md tools are pretty easy, as command-line utilities go, but they're really, really intimidating if you've never worked with Unix. There's a lot you need to understand about how the system works before you can build, format, and run a disk array very well. I'm confident you can puzzle your way through it, with help, but it will be a puzzle, and it will be frustrating. In exchange, you've got a full-featured server machine, and you can do anything you want with it. Run your own DNS, run your own DHCP, run a local mail server, run a VPN host, whatever you want, man.

I haven't used FreeNAS, but it seems fairly well-liked. It's based on FreeBSD, has a nice web GUI, and supports ZFS, which is a super-advanced filesystem that's got all kinds of cool reliability features in it. It's better than anything on Linux. I'm not sure you can get a Crashplan client for FreeBSD, though, while I think one exists for Linux. And you should be able to make a FreeNAS system into a server as well, but that part of it is likely to actually be harder than doing it with Linux, and keeping up on patches is a real PITA if you use the Ports system. (which is how you install most third-party software.)

If you want to use Windows, you'd probably want to use the Intel softraid controller, which comes, now, on most motherboards. This still uses the CPU for all the calculations, but it hides everything from Windows, so it runs pretty fast, and works pretty well. (Windows has a software RAID, but it is awful.) A dedicated controller is faster and more featureful, but WAY more expensive, and not that much better. We've typically got tons of CPU power we're not really using anyway, so running a RAID on the CPU works out quite nicely on modern machines. This should give you the exact same protection you'd have on FreeNAS or Linux, and while the last benches I saw on Intel softraid were quite a bit slower than Linux software raid, it doesn't usually need to be that quick for a home network server. The network is usually the bottleneck, not the disk.

Something like a Synology will be turnkey, and you won't have to think about much except what drives to buy. In exchange, the available software features are typically pretty limited. They may even run Linux inside, but they'll probably try to hide that from you so that you can't break anything.

Just to give you an idea of what I did for myself: my server is running 8 1TB drives in a RAID-6, so I have 6TB of drive space available, and can lose any two drives without losing the RAID. I do daily backups to another volume that's mounted only long enough for the backup, and then offsite backups of the (subset of) data I really care about.

Ok, after talking to Mono and reading what you guys wrote, I'm thinking that rolling my own might be the way to go. At least partially.

If I pick up something like this HP ProLiant ML110 G7 Tower Server, throw WHS 2011 on it, and throw four big drives in it, I should have the best of all worlds, right?

It has a SATA Raid Array in it which looks like it lets me do RAID 0, 1, and 1+0, so judging by Malor's post I'd want to go with RAID 1. I could put the business info on the RAID drives, and then media and what not on the other two (assuming the controller only handles one pair of drives).

I'd prefer something that has more than 4 drive bays, so I may research a bit more, but what do you guys think of this path in general?

Well, I did some digging, and honestly, that looks like a pretty sh*tty RAID controller. Only Raid 0, 1, and 1+0, as you found on your own.

Why not just do something like:

  • This ASRock H77 motherboard, with six SATA ports, $80
  • A Pentium G850. Dual core, 2.9Ghz, HD2000 graphics, $80.
  • Two of these RAM sticks, $36, 8 gigs total. You could shave off another $15 by dropping back to four, but a little RAM is good in a server.
  • This 430W Corsair power supply is bigger than you need, but it'll support almost any graphic card if you add one, and it's $25 if you claim the $20 in rebates. $45 otherwise.

Then you just need a case, drives, and some SATA cables, and you're golden. I'm not too up on inexpensive cases, so hopefully someone else can chime in.

That'll give you a zippy CPU with decent onboard graphics, quite capable of being an HTPC, and the Intel chipset will give you RAID 5 and RAID10, which are the most common types. I'd kind of nudge you toward buying 3 drives and setting up a RAID5 -- that way, you get the space of two disks, for the price of three, instead of one disk, for the price of two. You can go up to 6 drives on that motherboard, but I don't like going past 5 with RAID5... it gets too risky.

Ok, so I take your list there, add six SATA cables, and three or four drives (probably 3TB each) and find a case and I'm good to go?

Any particular form factor I'd need for the case to ensure that the motherboard would fit? I've upgraded or replaced every single piece of computer hardware over the years including CPU, with the exception of the motherboard, so I have no idea what to look for in fitting a case to a motherboard or vice versa.

Thanks Malor. A "Buy these things and put them together, moron" list is pretty much exactly what I was looking for

Yep, pretty much.

That's an ATX motherboard, so basically any ATX case is fine. Don't buy mini- or micro-ATX, those are much smaller. If you want a really small case, that can be done, but not with that specific motherboard.

How important are size and noise for your case selection? Are you planning to keep it in the living room, or can you put it somewhere that nobody will hear it?

The motherboard will probably come with 1 or 2 SATA cables, so you should only need to buy about four. But they're really cheap, if you get them somewhere like Monoprice... the extra two will disappear in the shipping cost.

Watch Newegg carefully on shipping, they tend to be a bad place to order cables and cases from, because of that. A cable is often like $2, but then they charge $6 shipping per item. And it often costs $50 to ship a case from them, so Amazon is often a better source.

Noise isn't terribly important, it's in my home office so I'd prefer it not be a jet engine, but it also doesn't need to be an HTPC. Size isn't terribly important either, in fact larger with more drive bays would probably be better.

At the moment I'm reviewing these three from Newegg, though your point is well taken and I may end up buying them from Amazon if they carry them.

Rosewill REDBONE Black SECC Steel ATX Mid Tower Computer Case

Rosewill BLACKHAWK Gaming ATX Mid Tower Computer Case, come with Five Fans, window side panel, top HDD dock

Antec Nine Hundred Black Steel ATX Mid Tower Computer Case with Upgraded USB 3.0

It looks like Newegg's shipping is reasonable on all three cases you linked... $5/$3/$10, respectively. Not too awful.

Corsair cases are quite popular here. You could consider the 550D, which is pricey ($120 AFTER rebate) but supposedly very quiet. I've never actually touched one.

That $45 case looks okay, though, and the $80 difference will cover a lot of drive space.

I've added the $89 case (for USB 3.0 support), a blu ray drive, three 2 TB Seagate Barracuda Green drives, WHS 2011 and a couple of SATA cables to the Newegg cart.

Cart price hovering at $797.46.

My finger hovering on Buy button.

Wife's pen hovering over the divorce papers if I take on another project rather than buying a ready made.

Ah well, what's life without a bit of risk. I'm going to go ahead and buy this package this afternoon, barring anyone offering any last minute tweaks to the build.

Well, it's not a gigantic project, if the wife's really upset -- about four hours start to finish, usually, and then probably some tweaking, but there'd be as much tweaking in any unit you bought.

Those drives aren't very fast, but they don't really need to be. They'll probably keep gigabit ethernet saturated, most of the time. And they're nicely cheap and quiet.

Don't forget a cable for the Blu-Ray drive. That's an OEM drive, and may not come with one.

edit: oh, if you want to make the machine quieter, you can add an aftermarket CPU cooler and aftermarket fans to the case. You don't need to worry about that now, but if you get it all hooked up, and it's too loud, first thing to try is tweaking the BIOS to enable fan speed adjustment, which will usually cut down noise quite a bit, and then replacing the cooler and fans if it's still too loud.

D'oh, literally placed the order five minutes before your post, without a cable for the Blu-Ray.

It uses a SATA cable though, right? If so I should be ok, I ordered three for the drives, and you mentioned the case would probably come with one or two. I think I have some old SATA cables lying around too.

I told the wife I was ordering it, and I could actually hear her rolling her eyes from 30 miles away. I blamed you guys for talking me into it.

The motherboard will probably have one or two. If all else fails, you can pop down to a Best Buy and grab a couple, though they'll be grossly overpriced.

The one thing to be careful about is mounting the motherboard. Most motherboards will have nine holes; count the holes, then manually count the brass standoffs in the case, making sure there are the same number, and then hold the motherboard directly above the case bottom and eyeball every hole to make sure it has a standoff. The only thing you're really likely to screw up in a build these days is getting a brass standoff into the bare circuitry of the board, instead of into a hole.

Also remember that there are TWO power connections on the motherboard, not just one. There's the big long one, and then a smaller one near the CPU. Most motherboards have an 8-pin plug, with four holes covered. If your lead from the supply that says CPU has four pins, just plug it into the empty holes. If it has 8, pull the little plastic shield out, and plug in all 8. It doesn't matter which one you have; none of the modern CPUs pull enough power to need more than 4 pins. So it works exactly the same either way.

Other than that, it's really quite easy to build modern PCs. If you can handle legos, you can easily do this.

If you don't mind throwing Win8/Win12 (shoot me right?) on new hardware then the replacement for Disk Extender is built in to those. I have a 5 3TB drives running on 2012 using ReFS that has replaced my v1 WHS. I still have the WHS but it is now a backup for the 2012 box (DATA only not OS) and it still handles the nightly backups of my clients since there isn't really anything else that replicates the ease of automatic backups and bare-metal restores as WHS.

Ok, built the server this afternoon. Posted fine, and installed Windows Home Server 2011. I've got a 500gb drive with the OS on it, and 3 2TB drives, which I initialized as GPT (GUID) drives. Now I need to set them up as a RAID array.

Following MonoCheli's instructions, I go into Server Manager, Storage, Disk Management, right click one of the drives and select New RAID 5 Volume.

It shows me all three disks, each with 1907600 MB capacity. I add all three, and it shows me:

Total volume size in megabytes: 3815200
Maximum available space in megabytes: 1907600
Select the amount of space in megabytes: 1907600 maximum

I knew that RAID 5 would drop my three 6 TB worth of drives to effectively 4 TB, and that's fine. The total volume size of 3815200 seems to match up with that, but the maximum available space number seems to be dropping it again, down to only 2 TB.

Mono remembers having this issue before, but can't remember how to fix it. I've done a quick google, but quickly get lost as this is all new to me.

Any suggestions from the gurus?

You probably don't want to do the RAID with the Windows software. Windows RAID is terribly slow. Rather, you want to build the RAID with the Intel tools. It still uses the CPU for all the grunt work, but it's far more efficient than the Windows flavor of RAID. I think what you want to run is called "Intel Rapid Storage Technology", but I can't check for sure, because I only have two drives on this machine.

From Windows' perspective, the 3 2TB drives become a single, 4TB SCSI drive, with driver software that takes a fair bit of CPU time. It has no idea there's any RAID going on.

edit: You might have to remove all partitioning info from the disks before the Intel tool will RAID them. I haven't actually done it, so I don't remember for sure. Let me see if I can find some docs online, I'll edit this again in a minute.

second edit: I can't find a clear set of instructions in a quick search, but the key seems to be putting the drives into RAID mode in the BIOS. Leave the boot drive in AHCI mode, but put the three RAID volumes into RAID mode, and then the tool should give you an option to create a RAID volume.

Ok, looks like that was it Malor. Scrapped Windows RAID and ran the Intel Rapid Storage utility. It allowed me to create a RAID volume which shows the full 4 TB.

After creating it, it told me I'd need to partition it through Windows, but the Windows tool wouldn't let me partition it. It tried to get me to make a simple volume out of it.

I looked a bit closer in the Intel tool, and it showed the volume as "Initialized: No", with a hover tip indicating it needed to be initialized. I kicked that process of ten minutes ago, and it's still sitting at 0%. MonoCheli had told me there was a process that would take 8-10 hours before I could use the disks, I'm assuming that this is it.

I'm going to let this run overnight, and hopefully it will all be ready for me in the morning.

Thanks for the help.

Yeah, that might be it. Many RAID controllers now allow you just use a RAID even while it's being initialized, but this is freebie stuff that's on the motherboard, and it may not be that sophisticated. If you're going to bed anyway, no biggie, might as well let it run.

As far as Windows is concerned, it just looks like a blank disk... it doesn't see three disks, it only sees one. So it should work just like a large blank hard drive would. It's been awhile since I've done this, but I think you first make it a volume, and then you create partitions... most frequently, a single partition that spans the whole thing, though you can obviously chop it up any way you like.

Been a while since I partitioned anything either. I'll probably just leave it all as one.

Hit this in the morning, add Crashplan, figure out how to change the workgroup name to the group my other machines are on (it's locked for some reason) - or whether I need to bother - and I'm ready to start creating users and moving my stuff over.

Holy smokes. Nine hours later and I'm only at 21% initialized!

Go ahead and try to partition it. It'll probably work.

It does take a really long time to initialize modern drives; the rate that they can accept data seems to roughly double for every ten-times increase to the amount they can store. Within ten or twenty years, it may take weeks to fully initialize RAID volumes.

(and I don't mean 'interface speeds', which have wildly outstripped the ability of magnetic drives to read and/or write data -- I mean the actual physical speed of reading and writing bits onto the platters. They get bigger at a much higher rate than they get faster.)

Within the Disk Management tool of Server Manager I'm showing the entire RAID volume as Disk 0 (the name Volume_0001 doesn't show up anywhere). But when I right click it, there's no option to partition it, only to create a New Simple Volume.

I assume that means either: 1) it's not ready to partition yet, or 2) it's been longer than I thought, and I'm in the completely wrong spot to partition the drive

Not sure about the media part, but Drobo has great storage solutions. Maybe a bit on the expensive side though.