Falling down the data hoarding k-hole

I am becoming a data hoarder.

I think I’m genetically predisposed to hoarding, or maybe it was just where I grew up, but every time I saw the show Hoarders on TV, I always thought that like every other person I knew in Indiana had a house that looked like that.  I’m not saying that all of my relatives kept boxes of dead kittens and uneaten food stacked from floor to ceiling.  But I never knew anyone in Elkhart that had one of those minimalist zen apartments with white walls and floors and no furniture like you’d see in Dwell magazine.  (Well, except for Larry’s place in Bloomington.  He did have a cannon, though.)

I’ve mentioned this a million times before elsewhere, but I think if I never would have left Indiana, I feel like I’d have the basement of a ranch house filled with some collectible obsession, like action figures or toy trains or something like that.  The year I returned to IUSB and was living at home, I had no money, but somehow locked into a comic book habit, and got this crazy idea that I’d someday own every single Spiderman comic. I probably got about 10% toward that goal before I gave up. Since then, there have been an endless line of reboots and relaunches and reprints, and if I would have knocked up that girlfriend and gotten stuck in Elkhart, I’d be able to tell you all about them.  But, that obsession passed, and I moved on to albums, then books, and now… data.

I have been obsessed with keeping all of my digital life archived. I think part of that is because there are a few big gaps that I can’t get back. For example, I never saved that much email. I save everything now, but when I was working on Summer Rain, which fictionally took place in 1992, I had about 500 emails from all of 1992 saved, probably about a tenth of how many I actually received.  And a lot of my conversation back then was on bitnet or VAXPhone, which was not archived.  And I have almost no photos from then, maybe a dozen.  I’ve probably taken a dozen photos of my cats this week.  And although I started keeping most of my incoming mail and all of my outgoing mail in 1996 when I started at Speakeasy, I lost all of my mail from 2000.

A constant wish of mine is that I’d somehow stumble upon an archive of old material that I didn’t know existed. When DejaNews first came on the scene, it suddenly uncovered a ton of old usenet posts I made in college, going all the way back to 1990.  I spent a lot of time on usenet, especially in 1991 and 1992, and it’s fun (and cringe-worthy) to look back at the stupid computer questions I was asking back then, or the lists of CDs I was trying to sell on the alt.thrash newsgroup. But more than the actual content, I simply enjoyed that rush of suddenly uncovering this hidden archaeology of the recent past, and finding all of these old bits of my past.  I’ve often said that it will be amazing if they every invent a search engine to find yourself in the background of others’ photos, because when I worked in Times Square, I must have photobombed thousands of tourists.

Anyway, I have been paranoid about backing up my machines since the 2000 incident. I use an external drive to clone my laptop drive, plus I use CrashPlan to back everything up to the cloud. But lately, I’ve had the issue that I’ve been accumulating too much stuff.  My new computer has a 500GB drive, but now I’ve got an 18MP camera that shoots video, and I’m scanning pictures and documents, and I keep downloading stuff and buying more music.  So, I decided I need an external data library, too.  And I started adding more storage.

This is the current data hoarding situation, as of this week:

  • A 4-bay USB3/SATA drive enclosure.  It’s basically a case with a power supply, hot-swappable drive bays, and a backplane that makes all four drives appear when you plug in the single connection.
  • 2x2TB Western Digital Red drives. I have those set up as a software RAID-1 in OSX, so when they are plugged into the Mac, they appear as a single 2TB drive that has roughly twice as fast read speed, and if one drive dies, I am not screwed.
  • A 3TB external that I use to back up the RAID.
  • A 2TB external that I use to back up the laptop.
  • A 500GB SSD external I use as a scratch drive for video editing.
  • A 1TB Western Digital NAS that I don’t use for much, but it’s there.
  • A 1.5TB external that’s connected to the NAS.

I also have an endless number of old, small, and/or obsolete IDE and SATA drives from dead computers, at least three semi-functioning computers with drives in them, two work computers, whatever is in my PlayStation, and an ever-increasing number of thumb drives and SD cards. And every time my life feels incomplete, I’m usually buying more USB thumb drives and stashing them in camera bags so I’ll have them on vacation, because there was one time in Germany when I wanted to watch a movie on my laptop but play it on the hotel TV, and I couldn’t find a big enough USB drive.

As the RAID fills, I have two bays open.  I’d eventually like to add 2x4TB to that and RAID it, too.  And I keep thinking about building a real NAS to use instead of the crappy one, so I can do stuff like run an iTunes library from it, but it’s not a big deal right now.

A k-hole I’ve fallen down now is hoarding sites. There’s a whole reddit on it, but there’s a lot of people who torrent and search and download stuff like crazy. Most of the people doing it are looking for stuff like recent movies, anime, e-books, music, and porn. I’m more interested in weird stuff, though: impossible-to-find movies, PDFs of oddball things, old zines, that kind of stuff.  For example, I’ve been collecting a ton of UFO-related PDFs. Most are things like FOIA request documents, Project Blue Book things and the like.  Or I found a site that was a very complete collection of internal documents from a certain church started by a science fiction writer, which I will not name so they don’t firebomb me.

There’s some strange stuff out there.  And there are a few people looking for it, searching corners of the web for open directories, folders of stuff left unlocked on servers, dumps of data.  It’s usually pictures, sometimes loose MP3 files or porn, but sometimes it’s pure craziness.  The whole thing reminds me of how ham radio people search the airwaves for stray signals, transmissions of automated numbers stations or radio checks. It’s the same, but downloadable.

Anyway, I’ll leave you with this.  Go search google for this and get started:

intitle:index.of "parent" -inurl:index.of

The new camera and the march of progress

I could not sleep a week ago, and woke up in the middle of the night, nauseous from the heat. I went downstairs to sit on the couch with the iPad, and pulled the trigger on an Amazon points subsidized purchase of a new camera, something I’ve been eyeing for a bit.

This was the much-criticized Canon EOS-M, which is an odd cousin to Canon’s DSLR line. I wanted to get a replacement to my Rebel XS, and wanted to stay in the Canon ecosystem because of the half-dozen lenses I already own. I thought about waiting to get either a higher-end crop sensor DSLR (60D), or a lower-end full frame (6D).  But the EOS-M got stuck in my head, so that’s what I did.

The EOS-M is Canon’s first mirrorless, and is essentially the 18.1 MP sensor from their APS-C cameras, crammed into a tiny case the size of a point-and-shoot.  It’s mirrorless, so there’s no optical viewfinder, and no clacking mirror with each shot. The standard Canon interface is also gone, replaced with a touch-optimized UI designed for the nice capacitive-touch screen. The switches are simplified, and the flash is gone. It’s an odd hybrid, like putting a Corvette engine in a Chevette.

The thing is, the camera got a lot of bad reviews when it cost almost $1000.  People complained about the autofocus, the odd UI, and how Canon missed the mark. They subsequently dropped the price dramatically, and the bottom line is you can now buy something that theoretically takes pictures like an $1800 7D, for only about $250 for the body only.  The camera uses a screwy EF-M lens mount, but for about $60, you can get an adapter and use all of your EF and EF-S lenses.  (Or you can get a cheaper adapter and use any old FD or Olympus lenses, if manual focus is fine with you.)

I grabbed the camera with a 22mm prime kit lens, the adapter, and an extra battery.  My first impression is that it’s adequate for taking snapshots, and once you get the hang of the menus, it’s a competent substitute for a DSLR, as long as you lens up and get a decent zoom on it. The big advantage is it doesn’t look like you’re hauling around a DSLR, and the camera you have is always better than the camera you don’t have when you’re out and about. (Probably the reason my iPhone is my “best” camera these days.)  Of course, it looks somewhat stupid to have a 300mm zoom the size of a Subway foot-long hanging off of the end of a tiny thing as big as a deck of cards, but it works well.

One interesting thing to me is that this camera supports Magic Lantern. This group of hackers has created a new firmware that sits on top of the existing Canon software, and gives you tons of extra features and switches and configuration parameters, along with an alternate viewfinder and a second menu system. It’s essentially like jailbreaking a phone and adding a new UI alongside the old one, unlocking tons of features. The whole idea that a camera contains an entire computer is baffling to me, but the concept of reverse-engineering it and slapping in new features is absolutely amazing. I added Magic Lantern, a fairly easy process – you just download it, put it on an SD card, then tell the camera to update its firmware, and you’re done. The two features I like are “magic zoom” and focus peaking. Magic zoom shows a 3x enlargement of the focus point, which is good because I’m slowly going blind. Focus peaking shows a cross-hatched fill pattern in the viewfinder where the camera is focused, which is also helpful.

Oddly enough, Canon fixed the slow autofocus in a firmware update, which is also real Star Trek technology to me, the idea that what you bought isn’t what you get, and they can fix a broken feature on the fly. Also dumbfounding to me is the idea that Canon lenses can have firmware updates. Within the barrel of even a cheap image-stabilized Canon zoom lens is a RISC processor probably more powerful than the PCs I first used to learn how to program.

I don’t remember how much my first point-shoot digital camera cost in 2000 – I think it was around the same price, maybe a little more. But that camera (an Olympus D0460) took pictures that were 1280×960 and looked pretty cheesy, very poppy colors and obviously a low-end camera at the start of the digital age. The new one shoots at 5184×3456 by default, with incredible sharpness and colors, and it has a wide array of lenses to do everything from macro to high-speed zoom. It’s been 14 years, but it amazes me that basically the same price point can offer so much more now.

Also, the big thing this camera does that my other ones don’t is video. It’s capable of full HD video, 1920×1080 at 30fps or the more film-like 24fps. And because of the sensor size, it’s taking much more incredible video than a camcorder or phone could. I have no idea what I want to do with video, but I want to do something, and I’m scheming along those lines now.

Anyway, it’s a fun little toy. Now I need another vacation to Hawaii to get some more great shots there.


more return to blogging garbage

I saw a bunch of articles recently about “the return of the blog” and suddenly remembered I have a blog and I never update it, and maybe while I’m circling rudderless on this next book, I should maybe think about that.

I have all of these various “content boxes” to fill, and never know how to evenly distribute the random chunks of thought. Should I be posting ideas to twitter?  More pictures to tumblr?  “Serious” photos to 500px?  Meme photos to Facebook?  Stories here, or submit the stories, or expand the stories and push them into books?  And when I do all of those things, in some mystical, perfect combination, then what gets posted here?  News and info, or what I ate for lunch, or… what?  The anxiety and uncertainty over all of that makes me not post.  The only real answer is to write.

They are drilling a hole under the highway across from my house. There’s a large vacant dirt lot across the street, the immediate view under my third-story windows.  The power company has leased the land and has an armada of heavy machinery there now, large drills and generators and containers and backhoes and other unknown things, surrounded by a temporary fence emblazoned with the name of an industrial rental company every ten feet. During the day, they’re essentially drilling for oil sideways, running segments of pipe into this patch of mud and debris. I think they pump in water, or suck out mud, or something, the mess being sifted by a large machine that looks like if a dumpster had sex with a Sherman tank. The sound is not incredibly loud, but it’s loud enough, and constant. I think they will be doing it for a few more weeks. I hope they find some dead bodies, or a UFO. I’ve got in the noise-cancelling earbuds, which do little, and have some stupid new-age meditation music playing, because I have a splitting headache.  (I think it’s mostly allergies, though.)

I started my own social networking site this week, and then decided that was a stupid idea and closed it.  So now I’m sitting on the domain for and don’t know what to do with it.  Any of this stuff is a waste of my time though, and I should be writing. I have become more and more disillusioned with Facebook, not the actual software itself or the company, but the people I follow. I have some really good friends on there, and then a bunch of people who only post about Ebola and whatever NFL player did whatever to whoever last week. I often wish I could find my own clique or group out there, but the more writers I find online, the more I realize I’m army-of-one’ing it over here.

I have been piddling with this UFO cult book, and it’s going slow, so I keep throwing words into the chasm of this book that’s essentially a sequel to Atmospheres, but that has no tracks yet, no structure or theme or anything else. It’s fun to work on, though. The audio book for Atmospheres is done, awaiting approval, so hopefully I will have news on that in a bit.