After a big upgrade to the infrastructure of our LAN at work, I was feeling peppy and full of that wholesome oh so good “can do” sensation that comes with seeing an idea brought to life and followed to completion. Not to go into too much detail (Me? Ramble on about nothing? Never!) but since I joined the ranks of the very few fine gentleman that I work with, I did nothing but constantly “disrupt” the workplace. (to use one of those awesome fucking buzzwords that make tech managers salivate and regular folk like you or I froth at the mouth with incoherent rage the likes of which might serve purpose to trigger as resonance cascade and touch off a firestorm that engulfs the atmosphere akin to the likes imagined by Dr. Oppenheimer before the first test fire of a nuclear weapon)
By “disrupt” (mwuahahaha I’m a sadist I really am!), I mean that I came in, didn’t like what was going on technologically around the office and started to “improve” things. First to go was the analog phone system that was 2 corded 4-line desk phones and 2 cordless phones shared among 6 employees. I took some of my limited knowledge of Asterisk from my youthful days of setting up severs in my bedroom and set up a FreePBX server and deployed phones to everyone (with the generous help from the boss man who paid for everything!). Soon after, I set up a centralized PXE imaging server to do our OS installations on new machines to speed that up. A well, a coworker of mine got the harebrained idea that he wanted to build an off-site backup business. We hashed out a lot of details and did a lot of experimenting until finally we got everything up and running and eventually he merged it into our employers business in trade for a room to put the server racks in and power to run them with. So then we became a “datacenter” of sorts!
Oh it’s been a blast, migrating to a virtualized cluster platform, redundant storage, I built a router using pfsense, pretty much the vast majority of it all running on open source software much because of my badgering the idea man to use it. Indoctrinating him, traditionally a Windows man, into the world of Linux/UNIX and FOSS. With one fell swoop I became a server administrator. What I’ve longed to be since days of my youth! Not that I get to stop repairing your grandma’s computers and answering telephones, but oh well. It’s not a whole lot to manage anyhow, and I’ve written some decent monitoring scripts to help me keep everything in line. That and a slew of RSS feeds to the pulse of the security world so that I know when to patch my shit, it’s mostly smooth as butter!
“Not to go into too much detail”
So then we took a look at something else to be improved as it stood, we had a single 250 foot ethernet cable that ran from our server room, across the ceiling, down a wall, along another wall, and feeding into 24 port switch like a super tall, sideways U. From this single line we had the 24port switch with lines out to 6, 8-port desk switches we plug in-repair computers into, our workstations, and our VoIP phones. On top of that, to try and help our latency and bandwidth situation, we had our PBX and the PXE image server off of the 24port switch in a mobile rack in the back work area instead of in the server room. There was yet another line that came off the 24port switch, back up along the ceiling and down to the front desk to feed 3 other machines, another VoIP phone, and some printers. Upwards of some 50 network endpoints at any given moment all feeding through a single 250 foot run of ethernet back to the server room where our NAS and Domain controller were housed. If this sounds abundantly stupid well it was. Here’s kind of what our LAN network was like:
So I’ve been griping about this for some time, but the boss man didn’t want us to run a shorter length (about 85 feet) which would cause a column of cable to be visible to the store front. What we really needed was some 20+ drops. But finally, after complaining about it for ages, we came up with a game-plan to put in a nice looking piece of conduit to hide the cable.
The Almighty Fence Post
What sort of conduit, you ask? Why a $17 dollar PVC fence post! Best yet, given the view from the store front looks like a painted square steel structural column from the ceiling down behind the half-wall. No one even notices it! I’m so proud of my co-worker who did a lot of the heavy lifting on this job by opting for this wonderful solution, and making the home depot runs to get the materials we needed. He deserves a lot of props for implementing my cooky (yeah, a stable network is cooky right?!) ideas. So we took several thousand foot rolls of ethernet, cut them down into 100foot runs (oh my god the cable measuring/spooler machine was SO fun to use!) and strung a much shorter bundle and filled up a patch panel! So we’re running 3 drops per work bench now, one that feeds the desk switch that customer computers get plugged into, one for the VoIP Phone, and one for our workstation. So we’ve now managed to physically separate our VoIP segment, our Work segment, and the customer PC segment all away from each other into a unique switch for each back in the server room’s network rack. To boot, we managed to put the PBX and PXE image server in the rack next to the LAN NAS and Domain controller where they belong!
Are we there yet?
Yes, yes, okay sorry. I’m so bad at going on a tangent when I try to post about something and end up going into detail about something completely different but in some background way related to the original thing that got me inspired to put up a post in the first place!
Anywho, so yes all of this network upgrade stuff got me in a great mood to DO STUFF. So what did I do? What I suggest every absent minded tech worker does and CLEAN THEIR FUCKING WORK AREA. Mine was pretty bad, granted I was literally the ONLY employee who’s workbench didn’t have drawers to put stuff in so my top shelf was cluttered with cables, adapters, hard drives, you name it. What’s more with a concrete floor you tend to forget about sweeping and boy did I need to do that. Zip ties, screws, dust clods, old stripped out computers. Yeah it was bad. So I took a couple of paid hours to overhaul everything. I needed to put in my new cable runs to my bench afterall! Pulled everything off the shelf; swept up the landfill; managed to procure a plastic drawer thing to shove under my desk and put everything inside of that in a neat and organized fashion; and terminate/plug in my fancy new cables! That isn’t what I wanted to write about, however!
*sigh* Alright, and?
In the process of un-cluttering everything, I found an old friend of mine! I once owned a Cisco Linksys E2000 home router, one of the first “high end” home routers I ever purchased with my own money rather than find in a scrap heap or make out of an old Pentium box and some Linux. It served me well for years, a stalwart little black and grey slab, happily blinking its LEDs and routing my packetz. At some point I got it in me to flash the thing with a custom firmware. The one I settled one that had the right set of features I wanted was TomatoUSB. In particular, the mod of TomatoUSB by Shibby. Oh what a great thing this was. I suggest anyone who has a capable home router flash some sort of Linux-based firmware upon their device and unlock the potential of their hardware. Little MIPS processor on the 2.6 Kernel doing all sorts of things Cisco would have LOVED to charge me some $500 more to put into their software. VLANs, OpenVPN tunnels, traffic QoS, unlocked WiFi region and transmit power *grin* (Sorry FCC, but fuck you there’s too much damned WiFi around me to not use Chanel 14!)
But sadly back during the dawn of Heartbleed, the night I came into the knowledge that an update to my beloved Tomato firmware existed that contained the patched binaries for OpenSSL I got a bit antsy. Now it was 3am, I was a bit boozed up and sleep deprived (On a work night too gosh I’m such a good employee) and for some reason I had it in my head that I owned an E3000 not the E2000 I actually owned. So what did I do? Check the model number? FUCK no, download the E3000 firmware and flash that fucker right onto my beloved little router because DON’T HAX ME BRO was flooding through my mind about having SSL exposed to the Intarwebz ripe for exploitation! Then, the obviously predictable happened. Reboot bloop annnd
Well fuck me here I am at 3am half falling asleep trying to figure out why the router isn’t booting up and all the lights are flashing in a way they never have. Fuckit go to bed maybe it’ll fix itself by morning.
The next day
Well woke up to the horrifying sight of the poor little router that could still blinking a terrible death throe. I knew I’d done and fucked up no internet, time fast approaching that I needed to get my ass to work, and a poor room mate who now had to go the rest of the day without the internet! Oh what a great sysadmin I am!
So, on to work I went router in hand hoping I can figure out a way to unbrick my little friend. Using my downtime wisely to Google around, coming up with a couple of methods I could use but sadly, none of the things that did not require me to crack the enclosure open managed to work. So I saw a post on how get a serial connection to the motherboard by way of some exposed pins on one of the ethernet ports. Followed some pictures to craft a neat connector to push in and try to touch the pins using some careful razor blade work and a ribbon cable but just couldn’t get it right. On top of that, I needed a special serial adapter. We had some serial-USB adapters but none supported 3.3v. So I managed to order a 6 dollar adapter with some pin headers and waited patiently. In the meantime, I took home a brand new ASUS AC1750 router to take the place of my old one, at least (hopefully) for a time.
Finally, the USB adapter arrived and I once again tried to get the thing working using my nifty little ribbon cable creation but it was of no avail. So, crack the thing open I did. Now I’m not afraid of voiding a warranty, that’s not the case and I’ve a very good track record of not frying bare circuit boards despite never using a static wristband, but I still worried about it. This thing was kind of sentimental to me! It wasn’t just a router. It was my router. Even if it had the arch nemesis Cisco’s logo etched right onto it. Like a branded slave I freed it from the bluish-green empire and empowered it with the freedom of Linux!
So with my trusty soldering iron in hand, I got to tacking on some cables to an internal set of exposed pads that make up the serial port (separate from the one hidden under one of the ethernet ports). A few terrible solder joints later, I had frankensteined cable with a header on one end to plug into my serial adapter.
But it didn’t work. I didn’t know why, spending way too much time trying to get it to work until eventually I gave up and set the whole thing not even reassembled, aside so that I could get back to working on my actual job.
Fast forward several months.
So we’ve come full circle again! I was feeling peppy, cleaned my desk, and found the carcass of my once pal the E2000, covered in dust and missing the screws that once held the enclosure together. Just some plastic, poorly soldered cable, and a silent circuit board pitifully staring up at me its LEDs dark.
No, I had to give it another shot before I laid it to rest forever! I’m smart, why shouldn’t I be able to figure it out!? So I set to work, got the little adapter, got GNU screen up on my workstation, found the power adapter in the bunch of cables I’d neatly put away into my fancy new drawers, and plugged the bad boy in. Still nothing right back where I was before. So I went to investigate until it dawned on me
The serial adapter has a GRND, TX, RX, and 3.3V power pins (as well as a 5V but I didn’t use it obviously). The router, the same I matched the GRND, the 3.3V the TX to TX on the adapter, and RX to RX on the adapter but OH FUCK. OF COURSE! Why would I plug RX to RX and TX to TX So I swap’d em TX to RX (transmit to receive!) and RX to TX (receive to transmit). How stupid could I be!!! Low and behold I had garbage printing in my console! In my enthusiasm, I noticed as well that the GRND cable had snapped away from the blob of solder holding it on the pad. Well I wanted to be safe and make sure that was connected (I honestly don’t have an intimate knowledge about serial interfaces, and just assumed these things needed to be there!). Got it re-soldered, plugged it in the same way annnd nothing. WHAT?! Why! What did I do wrong!! So being the crazed technician that I am recreated the condition that it last was working with and unplugged the GRND cable from the USB adapter. Then at last text in my console!!! I fiddled around with it until I found the proper baud rate and was presented with a beautiful sight a prompt for the router’s BIOS.
Now we’re talkin’!!
Ohhh yeah, I was living large. I didn’t care why it worked, I just cared that it did work! Finally after all of that time having now grown accustomed to the fancy ASUS AC1750 router chugging away in my home network, pushing huge amounts of WiFI bandwidth to my AC enabled Nexus 5, I felt determined to get this old friend back on his feet, even if I knew now that I was no longer going to employ it as my main internet gateway, I’d find a task worthy of it for certain!
Few commands to get the TFTP client working, and I pushed the latest version of the Shibby-modded TomatoUSB firmware to the device. Rebooted and the second most beautiful sight of the day a Linux root # prompt. The skies parted, the angels sung a chorus as life was once again given breath within the dear E2000!!
Best part about it I had never cleared the NVRAM, so it just magically had all of my previous configurations from that very fateful day in which I turned my faithful little device into a brick. THE FEELS FEEL THEM THEY ARE SO HAPPYYYYY!!
Well that’s it just wanted to put that out there.