How do I cut you? Let me count the ways.
August 9, 2011 2:36 PM Subscribe
You probably knew that much of the physical Internet consists of fiber optic cable. However, you probably didn't know just how many ways it can be broken. via
I am acutely aware of how fucking impossible it is to splice the damn stuff together. Ugh.
posted by GuyZero at 2:42 PM on August 9, 2011 [1 favorite]
posted by GuyZero at 2:42 PM on August 9, 2011 [1 favorite]
"Lastly, never under estimate a Southern gentleman with a backhoe and a shotgun."
Wasn't this also a Lynyrd Skynyrd lyric as well?
posted by BigHeartedGuy at 2:47 PM on August 9, 2011
Wasn't this also a Lynyrd Skynyrd lyric as well?
posted by BigHeartedGuy at 2:47 PM on August 9, 2011
Train derails, digs up fiber in right of way. Train load of hexa-poly-death-anate, which is leaking. Civil authorities hose the cars down, unless it explodes, and it always takes a few days. Not a quick fix. You had geographic diversity, right?
Similar to the story in the link, homeless people staying warm around their burn barrel in MN under a bridge melts the fiber. Repair has to call the cops because homeless drunks are throwing bottles at the repair crew for for fun.
Had a farmer modify his wireless radio by running a couple hundred feet of cat5 for POE to move his radio up to the barn roof. Cable on the ground was left to close too the fence, and his donkey ate the cable. Spec for installations around donkeys requires at least ten feet clearance.
Once had a person driven off a repair site by hyper-territorial wild turkeys.
Field work. Oy!
posted by dglynn at 2:57 PM on August 9, 2011 [5 favorites]
Similar to the story in the link, homeless people staying warm around their burn barrel in MN under a bridge melts the fiber. Repair has to call the cops because homeless drunks are throwing bottles at the repair crew for for fun.
Had a farmer modify his wireless radio by running a couple hundred feet of cat5 for POE to move his radio up to the barn roof. Cable on the ground was left to close too the fence, and his donkey ate the cable. Spec for installations around donkeys requires at least ten feet clearance.
Once had a person driven off a repair site by hyper-territorial wild turkeys.
Field work. Oy!
posted by dglynn at 2:57 PM on August 9, 2011 [5 favorites]
Working in telco, you hear a lot of the same jokes, like, every day.
These include such gems as "Hey, Stan [everyone in telco is named Stan], did you get your haircut?" "No, Roy [wait, some of them are named Roy], I got them ALL cut!"
Also, there is the joke that if you're going to go out hiking in the mountains, you should bring a little piece of fiber optic cable with you. That way, if you get lost, you can bury it under a little dirt or leaves, and a backhoe will right along.
posted by ernielundquist at 3:03 PM on August 9, 2011 [18 favorites]
These include such gems as "Hey, Stan [everyone in telco is named Stan], did you get your haircut?" "No, Roy [wait, some of them are named Roy], I got them ALL cut!"
Also, there is the joke that if you're going to go out hiking in the mountains, you should bring a little piece of fiber optic cable with you. That way, if you get lost, you can bury it under a little dirt or leaves, and a backhoe will right along.
posted by ernielundquist at 3:03 PM on August 9, 2011 [18 favorites]
We're pretty much middleman so we don't have that much fiber of our own to deal with, but anything affecting our customers or vendors means trouble for us too. Especially when say one of our primary grade A vendors goes down. That means calling every customer and telling them to route away or expect severe quality degradation for a while.
The big horror story I've heard about is from before my time, but it was when the redundant power at our data center turned out to be not so redundant. Oops.
(My boss also tells a story of when he used to work for a giant international retailer that had pretty much its own data center. Every couple of months they would run a test of the diesel power backup. One time though, somebody had forgotten to fill the diesel tanks. There were rather quite a few frantic calls and overtime and unhappy execs.)
posted by kmz at 3:07 PM on August 9, 2011
The big horror story I've heard about is from before my time, but it was when the redundant power at our data center turned out to be not so redundant. Oops.
(My boss also tells a story of when he used to work for a giant international retailer that had pretty much its own data center. Every couple of months they would run a test of the diesel power backup. One time though, somebody had forgotten to fill the diesel tanks. There were rather quite a few frantic calls and overtime and unhappy execs.)
posted by kmz at 3:07 PM on August 9, 2011
I'm just here to point out that since the bulk of the Internet backbone is fiber, and because fiber optic cable is generally bundled and carried from point to point by a tube of some sort, as such, the Internet actually can be reasonably accurately described as a series of tubes.
posted by eschatfische at 3:08 PM on August 9, 2011 [3 favorites]
posted by eschatfische at 3:08 PM on August 9, 2011 [3 favorites]
Had a subcontractor cut through over 400 strands of 50/125 OM3 at a site. Said he couldn't see why all this blue-green stuff was here, and he couldn't find any amperage going through it, so it seemed safe to cut. He used a sawzall. He cut it in two locations, both running from a central building to satellite buildings on the site.
He had to stand on a ladder to cut it. Said it was in the way of his painting.
posted by Sternmeyer at 3:08 PM on August 9, 2011 [7 favorites]
He had to stand on a ladder to cut it. Said it was in the way of his painting.
posted by Sternmeyer at 3:08 PM on August 9, 2011 [7 favorites]
BigHeartedGuy: Wasn't this also a Lynyrd Skynyrd lyric as well?
...
Lastly, never under estimate
a Southern gentleman with a backhoe and a shotgun.
He doesn't trust government
and shoots up internet cables just for fun
...
Nope, you're thinking of someone else. That wasn't Skynyrd.
posted by filthy light thief at 3:09 PM on August 9, 2011
...
Lastly, never under estimate
a Southern gentleman with a backhoe and a shotgun.
He doesn't trust government
and shoots up internet cables just for fun
...
Nope, you're thinking of someone else. That wasn't Skynyrd.
posted by filthy light thief at 3:09 PM on August 9, 2011
I know a vice-president of special projects engineering can ruin a fiber optic cable while demonstrating how flexible the cable is.
I also know that an errant operator foot can snap a fiber optic cable that has been run underneath a desk along the floor and not in metal conduit.
posted by infinitewindow at 3:26 PM on August 9, 2011 [1 favorite]
I also know that an errant operator foot can snap a fiber optic cable that has been run underneath a desk along the floor and not in metal conduit.
posted by infinitewindow at 3:26 PM on August 9, 2011 [1 favorite]
As part of my job we do a lot of vertical drilling in public right-of-ways. My saying goes something like this:
- Hit gas or electric and we'll all be dead
- Hit fiber and we'll wish we were all dead
posted by Big_B at 3:26 PM on August 9, 2011 [3 favorites]
- Hit gas or electric and we'll all be dead
- Hit fiber and we'll wish we were all dead
posted by Big_B at 3:26 PM on August 9, 2011 [3 favorites]
One incident that sticks out in my mind was during mud slides in Utah a few years back. We were trying to repair a cable across a ravine that was literally over a quarter of a mile wide and filled with raging water. Every piece of equipment and truck we had was stuck in mud up to the axels. We launched a couple of Sea-doos and a boat to try and pull the cable across the ravine to make the repairs, but no luck. We finally had to shoot the cable across with a line gun.
Worth reading for this sequence alone.
posted by EatTheWeek at 4:24 PM on August 9, 2011 [1 favorite]
Worth reading for this sequence alone.
posted by EatTheWeek at 4:24 PM on August 9, 2011 [1 favorite]
Sometimes I think I miss having a job where I get called out to fix things that crazy people have done. But I'm actually much happier just reading about it on the Internet.
posted by jeffamaphone at 4:54 PM on August 9, 2011 [1 favorite]
posted by jeffamaphone at 4:54 PM on August 9, 2011 [1 favorite]
I thought the physical Internet consisted of a cloud. That's why so much of the Internet industry is in San Francisco; if they need more Internets they can run out and grab some fog, and they don't have to rent a helicopter to do it.
posted by madcaptenor at 5:11 PM on August 9, 2011 [1 favorite]
posted by madcaptenor at 5:11 PM on August 9, 2011 [1 favorite]
What a group of Einsteins!
posted by rottytooth at 5:13 PM on August 9, 2011
posted by rottytooth at 5:13 PM on August 9, 2011
Sometimes I think I miss having a job where I get called out to fix things that crazy people have done.
I don't miss this. I really don't miss doing contract work where I'd be wishing for some Pepto Bismol before I opened the door to see what fresh horror awaited me. Sometimes I'd swear that half the commercial wiring done in this world is done as a practical joke on whoever the next poor bastard is that has to come work on it.
posted by empyrean at 6:07 PM on August 9, 2011
I don't miss this. I really don't miss doing contract work where I'd be wishing for some Pepto Bismol before I opened the door to see what fresh horror awaited me. Sometimes I'd swear that half the commercial wiring done in this world is done as a practical joke on whoever the next poor bastard is that has to come work on it.
posted by empyrean at 6:07 PM on August 9, 2011
Empyrean's got it right.
Half of everything seems like it was done in an emergency on St. Patrick's Day evening, and was going to get "cleaned up later". Ha.
posted by dglynn at 6:29 PM on August 9, 2011
Half of everything seems like it was done in an emergency on St. Patrick's Day evening, and was going to get "cleaned up later". Ha.
posted by dglynn at 6:29 PM on August 9, 2011
Not fiber, but I witnessed the world's most anal cable repair ever. Seems there was phone trouble in our building. So, the phone company decides they need to dig a hole and re-splice the 100 pair (or some such) cable that goes into the building. First, they call Julie who dutifully marks out where the Comcast coax is buried. Second, the hole digging team appears to strip the sod and excavate. Sadly, the Comcast coax was buried a good 1-2 inches down and got cut.
Time passes.
The second hole digging team appears and digs the hole. And to their credit, they got the hole EXACTLY right. The cable was exactly in the center.
Time passes.
The line tech appears early one morning, and spends more time making the hole PERFECT. Right angles at all corners. Floor laser flat. Then he spends the rest of the entire day redoing the splice. Perfectly. One of those deals were all 100 pairs are laying akimbo and in a big mushroom looking pile, and 6 hours later, the cable was the exact right length. Then there was the 15 layers of sealing tape.
There is no punch line. I was just amazed at the perfect union of anal retentiveness combined with the absolute correct profession. And Comcast's ability to bury a cable functionally above ground.
(OK, if there is a punch line, it is that the three phase power lines for the building were under this guy's ass the entire time he did the splice. I hope he was wearing the rubber underwear...)
(Actually, not Comcast. Whatever crappy-ass local cable company that was here when the cable was laid. And to *their* credit, the coax was some beefy shit. Damn near an inch in diameter, bigger than RG11.)
posted by gjc at 6:53 PM on August 9, 2011
Time passes.
The second hole digging team appears and digs the hole. And to their credit, they got the hole EXACTLY right. The cable was exactly in the center.
Time passes.
The line tech appears early one morning, and spends more time making the hole PERFECT. Right angles at all corners. Floor laser flat. Then he spends the rest of the entire day redoing the splice. Perfectly. One of those deals were all 100 pairs are laying akimbo and in a big mushroom looking pile, and 6 hours later, the cable was the exact right length. Then there was the 15 layers of sealing tape.
There is no punch line. I was just amazed at the perfect union of anal retentiveness combined with the absolute correct profession. And Comcast's ability to bury a cable functionally above ground.
(OK, if there is a punch line, it is that the three phase power lines for the building were under this guy's ass the entire time he did the splice. I hope he was wearing the rubber underwear...)
(Actually, not Comcast. Whatever crappy-ass local cable company that was here when the cable was laid. And to *their* credit, the coax was some beefy shit. Damn near an inch in diameter, bigger than RG11.)
posted by gjc at 6:53 PM on August 9, 2011
I was just amazed at the perfect union of anal retentiveness combined with the absolute correct profession.
Yeah, there are two types of people in the world. Those who can crimp and splice cable, and those who cannot.
...to see what fresh horror awaited me.
Aww, that was the best part! I loved opening up a machine and seeing hot glue and duct tape everywhere, or opening a server closet to have dust and pigeons fly out. It means I get to bill a lot of hours.
posted by jeffamaphone at 7:07 PM on August 9, 2011 [1 favorite]
Yeah, there are two types of people in the world. Those who can crimp and splice cable, and those who cannot.
...to see what fresh horror awaited me.
Aww, that was the best part! I loved opening up a machine and seeing hot glue and duct tape everywhere, or opening a server closet to have dust and pigeons fly out. It means I get to bill a lot of hours.
posted by jeffamaphone at 7:07 PM on August 9, 2011 [1 favorite]
Had a subcontractor cut through over 400 strands of 50/125 OM3 at a site. Said he couldn't see why all this blue-green stuff was here, and he couldn't find any amperage going through it, so it seemed safe to cut.
I facepalmed myself so hard I cut a fiber optic cable outside of my house.
kidding of course; they won't bring FTTH to my neighborhood. *sob*
posted by davejay at 7:43 PM on August 9, 2011
I facepalmed myself so hard I cut a fiber optic cable outside of my house.
kidding of course; they won't bring FTTH to my neighborhood. *sob*
posted by davejay at 7:43 PM on August 9, 2011
While on the subject of squirrels, submarine fiber optic cable is frequently attacked by the Teredo Worm.
posted by thewalrus at 10:04 PM on August 9, 2011
posted by thewalrus at 10:04 PM on August 9, 2011
The big horror story I've heard about is from before my time, but it was when the redundant power at our data center turned out to be not so redundant. Oops.
This is not so far fetched - just this week a big facility which has its own generators failed to start-up the generators when the power cut. I'm assuming there was a UPS which was intended to buffer the period between dramatic loss of city power and generator startup (given that 1 megawatt+ diesel generators take a while to warm up), and the UPS entirely drained itself.
posted by thewalrus at 10:07 PM on August 9, 2011
From Neal Stephenson's 1996 work on the FLAG cable system:
Dr. Wildman Whitehouse and his 5-foot-long induction coils were the first hazard to destroy a submarine cable but hardly the last. It sometimes seems as though every force of nature, every flaw in the human character, and every biological organism on the planet is engaged in a competition to see which can sever the most cables. The Museum of Submarine Telegraphy in Porthcurno, England, has a display of wrecked cables bracketed to a slab of wood. Each is labeled with its cause of failure, some of which sound dramatic, some cryptic, some both: trawler maul, spewed core, intermittent disconnection, strained core, teredo worms, crab’s nest, perished core, fish bite, even “spliced by Italians.” The teredo worm is like a science fiction creature, a bivalve with a rasp-edged shell that it uses like a buzz saw to cut through wood – or through submarine cables. Cable companies learned the hard way, early on, that it likes to eat gutta-percha, and subsequent cables received a helical wrapping of copper tape to stop it.
posted by thewalrus at 10:10 PM on August 9, 2011
Dr. Wildman Whitehouse and his 5-foot-long induction coils were the first hazard to destroy a submarine cable but hardly the last. It sometimes seems as though every force of nature, every flaw in the human character, and every biological organism on the planet is engaged in a competition to see which can sever the most cables. The Museum of Submarine Telegraphy in Porthcurno, England, has a display of wrecked cables bracketed to a slab of wood. Each is labeled with its cause of failure, some of which sound dramatic, some cryptic, some both: trawler maul, spewed core, intermittent disconnection, strained core, teredo worms, crab’s nest, perished core, fish bite, even “spliced by Italians.” The teredo worm is like a science fiction creature, a bivalve with a rasp-edged shell that it uses like a buzz saw to cut through wood – or through submarine cables. Cable companies learned the hard way, early on, that it likes to eat gutta-percha, and subsequent cables received a helical wrapping of copper tape to stop it.
posted by thewalrus at 10:10 PM on August 9, 2011
I'm curious how these cuts get repaired. I mean, it's optical cable so you can't just splice it together like you can with electrical cable, can you? Do you have to like melt the two ends together?
posted by cirrostratus at 9:24 AM on August 10, 2011
posted by cirrostratus at 9:24 AM on August 10, 2011
cirrostratus, check out
http://www.youtube.com/watch?v=PuO_IEn1X0Q
http://www.youtube.com/watch?v=ciwe4EEb-n4
Or
Splicing with Larry (RIP)
http://www.youtube.com/watch?v=bu_ion0912U
http://www.youtube.com/watch?v=jBzZ_f6QaUA
posted by fake at 10:03 AM on August 10, 2011
http://www.youtube.com/watch?v=PuO_IEn1X0Q
http://www.youtube.com/watch?v=ciwe4EEb-n4
Or
Splicing with Larry (RIP)
http://www.youtube.com/watch?v=bu_ion0912U
http://www.youtube.com/watch?v=jBzZ_f6QaUA
posted by fake at 10:03 AM on August 10, 2011
I've had about a dozen instances where a client provided us with the meticulous details of their diverse datacenter connectivity, separate LECs, physically different feeds into the building, etc., all terribly expensive to build out, only to have a backhoe a couple of miles up the road prove all that fiber was running through the same conduit.
@sternmeyer: you killed the guy where he stood, right?
@sailormom: squirrels are always, *always* dicks.
posted by kjs3 at 12:45 PM on August 10, 2011
I've heard enough generator horror stories to make sure I do my nightly rounds properly. Fuel level? Check. Generator water circulation? Check. No problem (I hope please don't fail oh fuck I don't want to get fired christ don't die you stupid generator)!
posted by tmt at 2:41 PM on August 10, 2011
posted by tmt at 2:41 PM on August 10, 2011
« Older A Dark Day for Independent Music and Film | Supermodel Superfugly Newer »
This thread has been archived and is closed to new comments
posted by kmz at 2:37 PM on August 9, 2011 [4 favorites]