Falling down the data hoarding k-hole

I am becoming a data hoarder.

I think I’m genetically predisposed to hoarding, or maybe it was just where I grew up, but every time I saw the show Hoarders on TV, I always thought that like every other person I knew in Indiana had a house that looked like that.  I’m not saying that all of my relatives kept boxes of dead kittens and uneaten food stacked from floor to ceiling.  But I never knew anyone in Elkhart that had one of those minimalist zen apartments with white walls and floors and no furniture like you’d see in Dwell magazine.  (Well, except for Larry’s place in Bloomington.  He did have a cannon, though.)

I’ve mentioned this a million times before elsewhere, but I think if I never would have left Indiana, I feel like I’d have the basement of a ranch house filled with some collectible obsession, like action figures or toy trains or something like that.  The year I returned to IUSB and was living at home, I had no money, but somehow locked into a comic book habit, and got this crazy idea that I’d someday own every single Spiderman comic. I probably got about 10% toward that goal before I gave up. Since then, there have been an endless line of reboots and relaunches and reprints, and if I would have knocked up that girlfriend and gotten stuck in Elkhart, I’d be able to tell you all about them.  But, that obsession passed, and I moved on to albums, then books, and now… data.

I have been obsessed with keeping all of my digital life archived. I think part of that is because there are a few big gaps that I can’t get back. For example, I never saved that much email. I save everything now, but when I was working on Summer Rain, which fictionally took place in 1992, I had about 500 emails from all of 1992 saved, probably about a tenth of how many I actually received.  And a lot of my conversation back then was on bitnet or VAXPhone, which was not archived.  And I have almost no photos from then, maybe a dozen.  I’ve probably taken a dozen photos of my cats this week.  And although I started keeping most of my incoming mail and all of my outgoing mail in 1996 when I started at Speakeasy, I lost all of my mail from 2000.

A constant wish of mine is that I’d somehow stumble upon an archive of old material that I didn’t know existed. When DejaNews first came on the scene, it suddenly uncovered a ton of old usenet posts I made in college, going all the way back to 1990.  I spent a lot of time on usenet, especially in 1991 and 1992, and it’s fun (and cringe-worthy) to look back at the stupid computer questions I was asking back then, or the lists of CDs I was trying to sell on the alt.thrash newsgroup. But more than the actual content, I simply enjoyed that rush of suddenly uncovering this hidden archaeology of the recent past, and finding all of these old bits of my past.  I’ve often said that it will be amazing if they every invent a search engine to find yourself in the background of others’ photos, because when I worked in Times Square, I must have photobombed thousands of tourists.

Anyway, I have been paranoid about backing up my machines since the 2000 incident. I use an external drive to clone my laptop drive, plus I use CrashPlan to back everything up to the cloud. But lately, I’ve had the issue that I’ve been accumulating too much stuff.  My new computer has a 500GB drive, but now I’ve got an 18MP camera that shoots video, and I’m scanning pictures and documents, and I keep downloading stuff and buying more music.  So, I decided I need an external data library, too.  And I started adding more storage.

This is the current data hoarding situation, as of this week:

  • A 4-bay USB3/SATA drive enclosure.  It’s basically a case with a power supply, hot-swappable drive bays, and a backplane that makes all four drives appear when you plug in the single connection.
  • 2x2TB Western Digital Red drives. I have those set up as a software RAID-1 in OSX, so when they are plugged into the Mac, they appear as a single 2TB drive that has roughly twice as fast read speed, and if one drive dies, I am not screwed.
  • A 3TB external that I use to back up the RAID.
  • A 2TB external that I use to back up the laptop.
  • A 500GB SSD external I use as a scratch drive for video editing.
  • A 1TB Western Digital NAS that I don’t use for much, but it’s there.
  • A 1.5TB external that’s connected to the NAS.

I also have an endless number of old, small, and/or obsolete IDE and SATA drives from dead computers, at least three semi-functioning computers with drives in them, two work computers, whatever is in my PlayStation, and an ever-increasing number of thumb drives and SD cards. And every time my life feels incomplete, I’m usually buying more USB thumb drives and stashing them in camera bags so I’ll have them on vacation, because there was one time in Germany when I wanted to watch a movie on my laptop but play it on the hotel TV, and I couldn’t find a big enough USB drive.

As the RAID fills, I have two bays open.  I’d eventually like to add 2x4TB to that and RAID it, too.  And I keep thinking about building a real NAS to use instead of the crappy one, so I can do stuff like run an iTunes library from it, but it’s not a big deal right now.

A k-hole I’ve fallen down now is hoarding sites. There’s a whole reddit on it, but there’s a lot of people who torrent and search and download stuff like crazy. Most of the people doing it are looking for stuff like recent movies, anime, e-books, music, and porn. I’m more interested in weird stuff, though: impossible-to-find movies, PDFs of oddball things, old zines, that kind of stuff.  For example, I’ve been collecting a ton of UFO-related PDFs. Most are things like FOIA request documents, Project Blue Book things and the like.  Or I found a site that was a very complete collection of internal documents from a certain church started by a science fiction writer, which I will not name so they don’t firebomb me.

There’s some strange stuff out there.  And there are a few people looking for it, searching corners of the web for open directories, folders of stuff left unlocked on servers, dumps of data.  It’s usually pictures, sometimes loose MP3 files or porn, but sometimes it’s pure craziness.  The whole thing reminds me of how ham radio people search the airwaves for stray signals, transmissions of automated numbers stations or radio checks. It’s the same, but downloadable.

Anyway, I’ll leave you with this.  Go search google for this and get started:

intitle:index.of "parent" -inurl:index.of

Fridge Pack is a Registered Trademark of the Coca-Cola Corporation

I think if I had one chance to make one single change to my entire life from birth to now, I would find a way to back up every single sent and received email and bitnet conversation I had from the second I got an email account in the fall of 1989 to present.  I would also figure out some way to index and search this crap efficiently, but I can figure that out later.  I have bits and pieces of email from college, some near-complete archives, some important, but there’s ones I wish I could read now.

The biggest problem was that our accounts in school had quotas.  VMS accounts had a 2000-block quota, which if I remember correctly is 2000 * 512 bytes, or about a meg.  (Don’t start with the megabyte versus mebibyte shit – this isn’t wikipedia.)  Anyway, that meant that on a weekly basis, I was making a judgment call over what to keep and not to keep, not knowing in 20 years that the email I got from some random person in a flamewar would be important, because said person would become the third-largest selling children’s author and have a movie done by Disney, or they would become a terrorist, or whatever.  And those judgment calls were usually made when I went over quota.  And I was probably drunk, too.

The bitnet thing also bothers me.  I used to spend all day at work having these running conversations with various bitnet buddies, usually other people who also worked campus jobs answering phones or driving a desk.  Bitnet was sort of like IM but ten years earlier, a way to trade lines of chat with another person, but without the fancy AOL-inspired UI and smileys.  And here’s the problem.  Okay, I was friends with this girl in 94, 95.  In my head, I was more than friends with her; she looked like a 20-year-old Sean Young and didn’t know she was drop-dead gorgeous.  And she was funny, and a lot of fun to talk to, and we spent like six, eight hours a day talking about nothing while I sat in the basement of the Support Center, telling people that there really wasn’t an ‘any’ key on their computer.  And we hung out a few times, but nothing serious.  There were complications, like she was a devout Christian, maybe a Pentecostal or something.  And she went to church like five times a week.  And she lived with her parents.  But the biggest complication was that I was too chickenshit to do anything about it.  That’s pretty much the story of my last couple of years at IU, and maybe I want to write that story someday, like part of this stupid book about Bloomington I have been plonking away at for years and have recently been pushing around, but I think I need to stop writing about Bloomington and write a book about a bunch of guys filling out their brackets for an office pool for NCAA basketball, except the office is the laboratory where they engineered the AIDS virus and it’s run by Josef Mengele.  But I’m sure Jerry Stahl wrote that story as filler for Juggs magazine back in the 80s.  But I digress.

My point is, I have a lot of the emails I have from her.  But I have none of the bitnets, since you can’t log those.  And all of the emails were “hey, are you online?  bitnet me.”  So much for writing that story.

Also, for any (both) of you who read Air in the Paragraph Line #13, I have this story called “Burial Ground” that is kinda-sorta based on a relationship and breakup I had in 1993.  So this is a girl who I swapped a lot of emails with, a lot of pouring-out of the heart into the stupid EVE editor in VMS that I used to write my emails back in 1993.  And I had all of them in an archive, and I never went back and read them, because it was too painful.  I sort of told myself that after some set amount of time, or maybe when I got into another relationship and put it all behind me, I’d go back and read all of our emails.  And I gzipped them up.  And then I got like one email from her in 1994.  And when I was cleaning up my account because I had no god damned disk space because of that quota, I zipped up the unzipped email.  And it wrote right over the existing zip file without asking and deleted all of it.  And I didn’t notice it at the time – only like a year later, long after any tape backup would have gone away.

And I dated someone in 2000 that went to Cornell, and in the beginning of 2001, I bought a laptop and tried to rsync all of my data from my home PC to my laptop.  But I did the command backwards, and synced a blank hard drive on top of my PC.  I had a backup from 1999, and luckily all of my writing was backed up to a remote account.  But man, I wish I had that mail from 2000.

I also wish I had the mail I sent to people.  Because back in school, I was famous for writing these giant, rambling emails to various prospects, who probably deleted them without reading them, but I would love to still have them.  Starting when I moved my accounts to Speakeasy in 1996, I started religiously logging my sent mail, and aside from that 2000 blackout, have pretty much every mail I’ve sent since then.  And I keep talking about some way to slice that up into something interesting, but who knows a) if I will ever figure out how to read and edit all of it and b) if any of it is actually interesting to anyone aside from me.

Here is the oldest email that I wrote that I could find that was marginally interesting.  It was written to a guy who spammed everyone who was logged into a CS machine at a certain point of the night, looking for Douglas Hofstadter.

From: Jon Konrath 
To: William Winton 
Subject: D.H.'s email address
Date: Mon, 26 Oct 1992 00:34:51 -0500

William Winton writes:
 > I am looking for Douglas Hofstadter's Email address.  He works at Indiana
 > University's Cognitive Science Department.  If you know of him, please
 > send me his E-mail address.
 > Much appreciation (in advance),
 >     William Winton
 >     Internet:

Douglas Hofstadter doesn't exist, he is a bit of urban folklore here
at IU.  All of his work in the area of cognitive science was actually
written by an elisp program called spew-random-cogsci-jargon.el that
was developed for the emacs editor by Bill Perry.  The Cognitive
Science department is also a bit of a rumor, we are actually a vocational
school that specializes in truck driving and air conditioner technology.

-Jon Konrath, A.S. program, interior floor covering technology program