Life in a start-up
So I work in a start-up.
I actually founded it 4 years ago, we've been doing pretty well, but we're still small enough (10 people) that I still get the fun calls in the middle of the night that the server is down.
Last night was a night like that.
3am wake-up call, servers are not responding. ISP claims no knowledge until I've escalated 2 levels to get to someone who actually knows what they're talking about and admits they're having an issue with a router.
Make a big pot of coffee as I realize this is going to be a long morning with only 3 hours of sleep.
We're a real-time business, tying into shopping carts all over the place so all of our customers are affected by this, of course.
I wake up the infrastructure team (eh...guy) and he helps me troubleshoot. One of the developers was up working still from the night before helping me out as well. I think he passed out though, because he's no longer responding to IMs.
Packet loss goes up & down and all I can do is stare at it, send out notifications to customers and pray that the ISP networking team actually knows what they are doing. I have my doubts.
After no word from the ISP in question for 1 hour, I get mad and email the CEO/Chairman. He calls me back within 15 minutes, telling me that he'll have an answer for me within 15 minutes.. (pretty impressive, actually, but he's new, so I'm sure he'll get over that in a few months).
15 minutes later, the CEO calls back with no answer and is begging for another 15 minutes. Left with no other choice, I say yes.
5 cups of coffee and 6 hours later, the issue is resolved.
It's 9am in the morning and I've been working for 6 hours. The sales guys just got to the office and are calling me with questions.
It's going to be a long day.
Sorry for the rambling above, but I know I'm not alone on GWJ working in a small company, thought this thread could be a good way of sharing experiences, joys and whine about life, or the lack thereof, in the world of small businesses.
"This is way, way more bad boy than you're gonna be able to handle." - Tommy Gavin on Rescue Me.
Hope your day gets better, Magnus. Coffee is your friend.
Mercury's been in retrograde most of the week. It's like a full moon with a side of kicked-in-the-nuts. -- H.P. Lovesauce
If it's any consolation, these kind of cluster-uh-bombs are just as entertaining from the ISP side.
Danjo Olivaw Lives
The only good thing about days like this is that I don't feel bad about taking a 2-hour lunch break and play some games.
That does make me feel a little better.
The explanation I got from them was that 16 servers on our subnet was taking up most of the traffic and there was a problem with the router. They took those 16 servers offline and that resolved the issue. I can only imagine (but I'd much rather not) what was on them.
"This is way, way more bad boy than you're gonna be able to handle." - Tommy Gavin on Rescue Me.
Xbox Live Blog
My home on the web
There are definitely headaches working with a smaller company. It seems like the worst is dealing with other companies for services. I always love the internet dance, where every company involved blames every other company. Then large corporations can seem unwilling or unable to deal with us promptly... unless it's a sales call. For all the stress and problems, I love it, though. I've had the chance to sell out before, but I can't see going to work for somebody else again. I'd write more, but I have to run and put out another fire...
XBox Live: PoppinfreshGWJ
LobsterMobster wrote:
Screw the coffee, bust out the Jack Daniels!
Xbox Live: Trachalio
Twitter wrote:
I'm seriously considering it. 2-minute downtime again just now. ISP responding with "huh? No...really? That shouldn't happen".
2-minute downtime managed to generate 5 incoming phone calls to my cell while I was on hold with the ISP.
"This is way, way more bad boy than you're gonna be able to handle." - Tommy Gavin on Rescue Me.
Xbox Live Blog
My home on the web
We once had a power failure in our main datacenter, followed by a UPS failure the week after our head ops guy quit. We rolled over to the backup databases and started running the disk integrity checks on the masters. Halfway through that the power company "fixed" the power problem, dropping our power again. That was fun.
Funny/scary fact of the day: the CEO & Chairman of this ISP uses AOL email
I'm not going to name the ISP, but for the love of God, this is not a small-time ISP.
"This is way, way more bad boy than you're gonna be able to handle." - Tommy Gavin on Rescue Me.
Xbox Live Blog
My home on the web
We use speakeasy.net. We've had trouble with other ISP's, but Speakeasy has been consistently great, and they seem to have a fairly large footprint.
XBox Live: PoppinfreshGWJ
LobsterMobster wrote:
Start up story:
The company powers decided to change the company name at the same time we were going through our IPO. Hundreds of thousands of pages were impacted. Directory names, redirects weren't working, our pushes freaked out, and because of a typing error, on the day that Wall Street analysts were visiting, we dropped every single radio station. There were thousands. Every one of them crashed. Should have been an easy fix. Backup drives had failed. No recent back ups. What should have taken us a couple of minutes, ended up taking hours to fix. I think that was the day I actually had a 20 hour day. (Usually they were only 12.)
I miss the excitement and energy around start up companies, but damn, I don't miss the hours.
*Legion* wrote:
That reminds me of the time that a startup I was working at, where somebody tripped over a powerchord the day of the IPO and managed to take down the webserver. Management solution? Install a camera.
It's 7:30pm, I just opened a beer. It's been a looooooong day.
"This is way, way more bad boy than you're gonna be able to handle." - Tommy Gavin on Rescue Me.
Xbox Live Blog
My home on the web
Fedaykin98 wrote:
So..is there an English 101 class around here somewhere for Swedes?
/Sigh. Can't believe I typed that.
"This is way, way more bad boy than you're gonna be able to handle." - Tommy Gavin on Rescue Me.
Xbox Live Blog
My home on the web
Don't feel bad, you're totally metal.
Fedaykin98 wrote:
I'd hope that the company in question has died a horrible death by mismanagement by now.
I never minded piracy. Anyone who minds about piracy is full of sh*t. Anyone who pirates your game wasn't going to buy it anyway! -Warren Spector
I miss my startup days. The first year was loads of fun. Long hours, lots of work, but it really felt like we were going somewhere. Towards the middle of the second year, we had grown to over 100 employees... a significant portion of which didn't seem to actually DO anything. Too much management, no actual sales... and suddenly, we had managed to burn through $40Mil in VC without selling a damn thing.
I was one of the 20 people they kept on a skeleton crew, as they tried to turn things around... We actually managed to sell more in those last 6 weeks than they had in the previous year, but it wasn't enough. The board elected to shut us down, and sell off the remaining assets, rather than risk any more bridge financing on us. I can't really blame them, considering how much they were already in the hole.
Ah well, thems the breaks!
Skriss: lvl 70 undead mage - Bloodhoof
Kyrator: lvl 63 night elf rogue - Blackhand
Thadryn: lvl 70 draenei mage - Blackhand
I have two things in common with magnus: I run a very small business, and my boss is a Swede. (Well, second-generation Swede.)
Sounds like an ugly day, bro. Glad it's over.
If I didn't drink, Crom would laugh and cast me out of Valhalla when I die. Peer pressure I can handle, but not when it comes from Crom. -Lobo