Pirate Cinema

Note: I wrote this entry in late 2012 after finishing the young adult novel Pirate Cinema by Cory Doctorow, a book that explores, among other things, the consequences of criminal penalties for civil acts like copyright infringement. I was thinking about civil liberties and internet freedoms and what I’ve chosen to do with my life. I wasn’t quite sure what to do with the entry after writing it, so it sat for three years. Re-reading it now, the feelings and conflict I expressed still resonate.

I can get to feeling about Cory Doctorow the same way I often feel about Richard Stallman, the famous advocate for free and open-source computer software. Zealots. Troublemakers. Not everything is about The Man out to squash the little guy. Geez, I’m not evil just because I use Apple products! I respect your opinions on copyright and software, on free expression and privacy — but do you have to be so darn annoying about it?!

I suppose it is the same with anyone with a Cause. It makes the rest of us nervous, because we aren’t True Believers like they are. The long and short of it is that I have gotten older and supposedly wiser, and at some point I decided that the world is really complicated. Seeing lots of shades of gray makes it hard to get worked up about causes, which I suppose is why most people don’t.

I do care about this stuff. I care about it a great deal. I care about internet freedom, about privacy, about civil liberties, about remixing and free expression, about individual rights and blanket licenses, internet radio and everything else. I cared about it all through high school — I even took a summer course at UC Irvine in “Internet Law”, back when the whole field could be surveyed in 5 weeks! I followed mailing lists and message boards about the Napster and Kazaa court fights and the DMCA and the Communications Decency Act legislative battles.

When I had the opportunity in college to get an internship for my legal studies minor, I wrangled a placement at the Berkman Center for Internet & Society at Harvard Law School, the original epicenter for scholarship in the field of internet law. And when I got my first job out of college, it was at Berkman as well, working alongside some of my heroes.

I thought I would be a public interest lawyer. I thought I would live and breathe this stuff. And I met a lot of very smart people in this field, and I respect them a great deal. But the pace of it didn’t work for me, someone built for high pressure and quick wins. The law is slow and plodding, a years- and decades-long synthesis of law review articles, amicus briefs, books, conferences, and winding cases. It works for many people, and its very good work indeed, but it was not going to work for me.

Now I’m in the fast-paced world of technology, wrapped up in fancy new trends like “DevOps” and “continuous integration” and “infrastructure as code”. There is lots of energy and lots to do. I feel busy and driven, but not the way I felt about internet law. I don’t have the same purpose — I never feel like the work I am doing has the last possibility of fundamentally affecting our culture or reshaping our society. No one is going to jail for abusing deployment frameworks, and no one is passing legislation telling me how I can or cannot administer a web server. The stakes just aren’t very high.

No one is going to jail for abusing deployment frameworks, and no one is passing legislation telling me how I can or cannot administer a web server.

cory-doctorow-pirate-cinema-coverI just finished Cory Doctorow’s latest novel for young adults, Pirate Cinema. The main character is a teenager in a near-future Britain whose (illegal?) video downloading and remixing gets his family “banned” from the internet. This leads to all sorts of serious consequences — his father loses his (online) job, his mother can’t apply (online) for her disability benefits, and his sister fails out of school due to being unable to research and do (online) homework. Embarrassed and ashamed, our young cyber-criminal runs away from home and ends up in London. There he builds a new life after taking up with a loose collection of anarchists, activists, and other down-on-their-luck free-thinkers. Eventually he spearheads a a campaign to overturn the draconian laws that got him in so much trouble in the first place.

It’s all a bit contrived, and some things fall in to place much too easily. Certainly the homeless life is glamorized to a degree I find unsettling. But the fundamental fear expressed in this pointed critique is sound. Yes, Big Content (also known as the entertainment industry) pushes for laws that protect their commercial interests at the expense of our culture. Yes, the penalties for victimless crimes of copyright infringement continue to become more and more harsh. And yes, this leads us down some pretty slippery slopes.

And frankly, it is a whole framework of thought that I have managed to relegate to the dustbin at the back of my brain, a big ball of stuff I believe deeply in but have carelessly shoved aside.

Cory Doctorow’s novel has given me a bit of fire back, and for that I thank him. It also gives me hope. Doctorow postulates a future in which the internet becomes so central to everything we do that the general public has no choice but to stand up and take a stand against the government and corporate control that is clamping down on us from all sides. Doctorow makes a persuasive argument as to why this sort of control is not only wrong, it is destructive to a free society. And he aims to persuade young people who are the most affected by these changes but not always cognizant of them. He is telling them they should care, and I hope his message takes root.

As a young kid I taught myself how to edit videos long before I taught myself how to program. I can only imagine that if I was born ten years later, in the age of high speed internet and much more powerful computers, I would have been a “remixer” myself. Heck, when I was young I saw the world as amazing and wonderful, the future as bright and thrilling and open for the taking. In short, I was just the sort of person liable to get caught in one of these copyright webs and have my life completely ruined.

I can blame my current cynicism on all sorts of things — endless war, recession, the general ennui that comes with growing older. I wish I could recapture some of that excitement, some of that optimism I had as a kid. What gets to me about Pirate Cinema is that by the end I was rooting for our young heroes to succeed, yet all the while convinced that he was writing to us from his eventual jail cell. I’m only slightly ashamed to admit that as I read the final pages tears welled up in my eyes. Not because its the best book or the best story in the world, but because it reminded me about what I care about, and that things are worth fighting for, and that the bad guys don’t always win in the end.

For all that, it is not at all clear that our “hero” lives happily ever after. Doctorow acknowledges that youth can’t last forever, and even the most idealistic of us do have to grow up and face, dare I say it, a more complex, more nuanced world. It is up to each of us, then, to temper our wisdom with a streak of idealism, and to not forget about the causes and morals we hold dear.


Book of Mormon

I’m in San Francisco for a work conference and was able to snag a last-minute ticket to see the touring production of Book of Mormon at the beautiful Orpheum Theatre. I was sandwiched between a group of 47 lovely residents of an Oakland retirement community. The abundant crude humor in the show made this seating arrangement only slightly awkward.

The show was uproariously funny throughout, but was also laced with sad and even tragic stories. South Park veterans Matt Stone and Trey Parker did an amazing job of using humor to present a thorough and multi-layered social commentary. As I watched the show I kept wondering how much of the audience was grasping the deeply cynical and biting undercurrent of this show. I expect that many were not — and perhaps that in itself is just another joke, this time at the audience’s expense.

Early (and ongoing) humor plays off of Mormon beliefs and practices that are surprising or seem silly to non-Mormon audiences. But time and again the humor is predicated on absurd comparisons to more “mainstream” beliefs. For example, one song by a main character proclaims that “I believe that the Lord God created the universe.” The audience remains silent. He continues, “I believe that He sent His only Son to die for my sins.” No reaction. Then, “and I believe that ancient Jews built boats and sailed to America!” Uproarious laughter.

It makes no sense. It’s absurd. Who would believe that? Never stopping to think that the two preceding lines, from a non-Christian, would sound just as absurd. The verse ends with, “a Mormon just believes,” but isn’t that also mocking all others who “just believe” their own deeply held but never critically analyzed religious convictions?

A whole other class of criticism takes the form of upbeat and absurd songs that portray the suffering of the native Ugandans. In chipper tones and catchy music they tell of their children raped by warlords, family members dying of AIDS, and friends felled by preventable diseases like dysentery. The audience laughs along, but at times it becomes sporadic as particular lyrics hit a little close to home.

A memorable song dreams of the magical land of “Salt Lake City” as a place where the “goat meat is plentiful,” the “warlords are friendly,” there is a “Red Cross on every corner,” and “the people are open-minded.” We laugh, then we realize what we are laughing at — the longings of a poor child who can dream of nothing more than a slightly nicer version of the desolate village that is the only life she has ever known.

Interwoven with the stories of the “Africans” are a number of pokes at white Westerners — missionaries and others — who feel entitled to lecture native peoples on how they should be living, push them to change their lives in highly disruptive ways, and then abandon them to deal with the unexpected long-term consequences of the half-baked “improvements” on their own.

The overall theme of the Book of Mormon seems to me to be that well-intentioned, dedicated, and focused individuals can make a positive impact on the world around them, but that the strictures of organized religion often serve to get in the way of or actively subvert that impact. In that way I see it as a message of hope — our society’s accelerating move away from organized religious institutions need not imply that we will become less dedicated to improving the lives of our fellow man. Far from it. Without outdated doctrine and absurd rules getting in the way, real and lasting impacts may finally be achievable.

Many will no doubt disagree with this interpretation. People often tell me that my relentless positivity about our ability to make the world a better place is absurd. But I continue to take to heart John Perry Barlow’s words that groundless hope is the only kind worth having.

The Fallen of World War II

This is a gobsmackingly incredible visual exploration of deaths related to World War II and other conflicts.  You'll start watching and it just keeps getting more insane, and you can't stop until the end.  Well worth the 18 minute run time. I thought I understood the scale of the conflict -- to the extent anyone can -- but I had no idea.
The Engineer’s Lament: Two ways of thinking about automotive safety

Malcolm Gladwell’s look into how automotive safety recalls work, and don’t. But hidden inside is a look at how we examine and evaluate risk and where we place priorities. And the sometimes insurmountable gulf between what engineers see and what normal, emotional people see. And the consequences in safety for how we prioritize and deal with public safety issues. A good read.

A week of disappointment with the Apple Watch

I got up at 3am to pre-order the Apple Watch and mine arrived on release day. My justification was simple — I have found the FitBit activity trackers to be useful but limited, and a more comprehensive device seemed like a great upgrade.

Unlike the many glowing reviews, I have found Apple’s much-hyped new gadget to be nothing but trouble. My litany of complaints is vast, so I will focus on a few major pain points that might dissuade others from purchasing this device until the next version is released.

Third-party app support
Third party app support is universally poor. I have not yet found a single third-party app that works well, they are all slow to load, quick to crash, and often fall out of sync with their phone apps. Apparently when the watch goes to sleep the watch app loses connection with the phone app, so things can’t just finish processing or loading in the background. You constantly have to stand there like a dope staring at your wrist waiting for something to happen.

The “glances” (cards) in the watch app for quick updates are also problematic for the same reason. Even Apple’s built-in glances, such as weather and stocks, do not update in the background, so I often find myself seeing yesterday’s weather or an out-of-date version of my todo list. Because I can’t trust the data to be accurate, I find myself not even bothering to use glances.

Watch faces
If you want a “digital” watch face with additional data (“complications”) rather than a pseudo-analog one, there is only one option. The level of customization is actually quite limited — you can’t put the time in the middle, for example, and while you can show your next calendar event or the phase of the moon, you can’t show your daily step count or any other third party app data. The calendar complication can only link out to Apple’s built in calendar app, the weather complication to Apple’s weather app, etc. There is no third-party integration possible, so if you like using Dark Skies to know when it is going to rain or Things for task management or basically any other of the thousands of third-party apps, there is no way to integrate them into your watch display.

It is very difficult to tell the difference between a phone notification that is not actionable, a watch notification that can be tapped to get into an app, and the app displays themselves. The whole interface is confusing in that way — am I in a notification, an app, a glance? Will swiping work, or not? Tapping? It is complete inconsistent. And it is very easy to get lost or frustrated, tapping the screen repeatedly only to find nothing happening. Why tap repeatedly? Because sometimes in apps you need to tap multiple times to hit the tiny touch targets. And sometimes you hit the wrong one, end up somewhere else, and have no obvious way to get back.

General bugginess and unreliability
The built-in health tracking is extremely buggy. Sometimes it tells me it is “time to stand” while I am standing. Sometimes it tells me to stand when I’m in the car driving at 60 miles per hour. Sometimes it tells me I achieved a fitness goal while I’m in the middle of a run, knocking me out of my running app, which then crashes and will not reconnect with my phone, so I’m frantically navigating through the tiny app launcher while trying to keep up my pace. The scroll wheel (err, “digital crown”) gets mucked up and won’t turn until I run it under water. The maps app takes forever to update with my current location. Sometimes I get buzzes for notifications but then none display. Sometimes I send a text message reply and the whole watch freezes for 30 seconds. A couple times I’ve had to hold down both buttons to restart the watch because it got completely stuck.

If I’m going to wear a device on my wrist, I want it to integrate into my day. I want it to be effortless. I want it to show me the information I need when I need it. I don’t want to fiddle. I want the apps I already use to easily integrate and work well. I want to be able to hide the many apps that I don’t care about, making it easier to find the ones I do. I don’t want spurious notifications. I don’t want a watch that crashes.

The Apple Watch, in my experience, is a failure at its basic purpose. Even the buttons — there are two, one of which is dedicated to sending your friends drawings, which I will never do. There is no way to assign that button to something I might actually want to use, like a dedicated way to get to a single app, or back to the watch face.

Luckily, almost all of the problems I have run into are software related, so I can only hope that Apple will remedy them in software updates in the future. But will that be anytime soon? And will the updates work with this watch, or will I have to buy a newer model? In the case of the Apple Watch, it does not pay to be an early adopter.

After Jobs Dry Up, What Then?

An excellent distillation of one of the most important issues of the next decade.

Update (2015-03-29): This post sparked a few conversations with friends that have caused me to revise my opinion. I still believe that rising economic inequality, lack of opportunities, decreased social mobility, and poor measures of success focused on economic growth are all major problems for our society. I’m not convinced that technological progress is the major cause of this shift or that this technological age is fundamentally different from previous periods of “creative destruction.” That said, I do still worry about the types of new jobs that are currently being created, many of which in the “gig economy” offer very little economic security and no benefits.


Introducing Hygroscope

Hygroscope is a command line tool for managing the launch of complex CloudFormation stacks in Amazon Web Services.

CloudFormation is a tool for creating and managing Amazon Web Services infrastructure using code. A JSON-formatted template describes the state of a “stack” including such resources as servers, S3 storage buckets, and load balancers. Utilizing the AWS Virtual Private Cloud service, entire software-defined networks can be described and repeatably created, updated, and destroyed using CloudFormation.

CloudFormation is not without its pain points:

  • Templates must be written in JSON, which, in addition to being difficult for a human to read, does not support niceties such as inline comments and repeated blocks.
  • Launching CloudFormation stacks requires knowledge of the various parameters that need to be provided, and it is difficult to repeatably launch a stack since parameters are not saved in any convenient way.
  • There is no easy mechanism to send a payload of data to an instance during stack creation (for instance scripts and recipes to bootstrap an instance).
  • Finally, it is difficult to launch stacks that build upon already-existing stacks (i.e. an application stack within an existing VPC stack) because one must manually provide a variety of identifiers (subnets, IP addresses, security groups).

Hygroscope aims to solve each of these specific problems in an opinionated way:

  • CF templates are written in YAML and processed using cfoo, which provides a variety of convenience methods that increase readability.
  • Hygroscope can interactively prompt for each parameter and save inputted parameters to a file called a paramset. Additional stack launches can make use of existing paramsets, or can use paramsets as the basis and prompt for updated parameters.
  • A payload directory, if present, will be packaged and uploaded to S3. Hygroscope will generate and pass to CF a signed time-limited URL for accessing and downloading the payload, or the CloudFormation template can manage an instance profile granting indefinite access to the payload.
  • If an existing stack is specified, its outputs will be fetched and passed through as input parameters when launching a new stack.

The latest version of Hygroscope can be installed via RubyGems. The inline help documents each command and its options. The source code for Hygroscope and additional documentation is on GitHub, and a sample template that sets up a “bare VPC” is a good introduction to creating Hygroscopic templates.

Everything Is Awesome!

Ignore the hyperbolic headline and drink in the facts and figures that refute the general negative feeling about the state and outlook for America in 2015.


The Fault In Our Stars

This review does not contain any spoilers.

The Fault In Our Stars Book CoverI found this novel deeply affecting. I hear from time to time, as everyone does, about various loose relations who have cancer, or are in remission, or have succumb to the disease. I see the cancer stories that describe tragic but stalwart children, their caring and committed parents, the charities they found and causes they champion, the valiant way in which they battle with dignity, their indefatigable courage.

And it all feels like bullshit to me. I don’t have the experience or the pain to justify this feeling, but I feel it all the same. Cancer patients and their families and support networks are not magically heroes. Evolution run amok does not make one noble or immune from normal-person feelings.

I hate the language we use to describe illness. I hate how we so often glorify people suffering from cancer while simultaneously pitying them. I don’t know how to interact with or relate to people who are suffering from disease, and I think the structures we as a society have created make that interaction harder than it should be. When we hold people up on a pedestal due to circumstances outside of their control, we don’t allow them to be normal people with normal-people feelings and concerns and fears and needs.


The Fault In Our Stars is a book about kids with cancer, but it is quick to point out that it is not a “cancer book” full of the standard tropes and plot progressions. The narrator is a teenage girl stricken with a form of the disease that affects her lungs, kept in check via experimental medicines and various machines to assist her breathing. Hazel is weak and frail and still alive long beyond her predicted expiration date. Her Sword of Damocles hangs ever-present as Hazel goes about her daily life, which is entirely normal in as much as she watches trashy TV and goes to the mall, and entirely abnormal in that she has nothing to strive for, few friends, little direction and plenty of pain.

Hazel joins a cancer support group suffused with a macabre sense of competition to outlive and outlast. The reader quickly gets the sense that most of her life post-diagnosis has been like this. There she meets a fellow sufferer, a high school boy with a prosthetic leg and a fear of oblivion.

They form a bond, they share Experiences (capital-E) and pain and fear and philosophy and random poetry and video games. They are overly wordy and prone to soliloquy and sort of strange, but in other ways quite real. They have ways of dealing with cancer and life and parents that feel very authentic to me. Things take dark turns, then people get better, then they get worse again. The future is uncertain, except that it is completely certain — dying is the endgame, and sooner rather than later.

How do you live when so much of your waking time is spent worrying about and wondering about and trying to fend off death? It is horrible and tragic but sometimes brilliant and funny and often just numbingly depressing. It is living, it is not living, it is a disease that doesn’t make sense and isn’t supposed to and doesn’t magically make people heroic or different, a disease that doesn’t care in a world that doesn’t care in a universe full of lives and people that may not have any meaning at all.

Oblivion. How do you face it? What choices do you make, when you aren’t given a chance to go out in a blaze of glory or on your own terms, but instead only slowly, by inches, in pain and agony and sadness? What does it all mean? Why should it mean anything? And how do you deal with that, each and every day?

Deep questions. Dark questions. Real questions.

Thank goodness there is a happy ending. Hazel and her friends figure everything out and feel better and know that they have accomplished something real and lasting and memorable.

No, I made that up. Of course it doesn’t end that way. It can’t. It just ends when it ends, as we all do. A surprising ending, but not surprising at all.  Because that’s how endings are.  They come along when you least and most expect them.  Sometimes, right in the middle of a

Never Forget

On September 11th some t-shirts I had ordered arrived. One of them looks like this:

I bought it because I thought it was funny and clever, but when it arrived I was a bit dismayed.  I had forgotten about how the usage of the phrase “never forget” has changed since 2001.  I first heard the phrased paired with “never again” in reference to the Holocaust.   The message was clear: we must remember humanity’s past misdeeds, lest we repeat them.  Similar phraseology has been used around other genocides, and the unfortunate fact is that we do forget, and we do allow them to repeat — Armenia, Rwanda, Congo, and now the ISIS actions in Iraq and Syria.  The world has not decided on a shared mission of preventing genocide in all its forms, and in that way the phrase “never forget, never again” is comically sad.

Never forget the dinosaurs plays on that — on the one hand its funny, because dinosaurs, right?  On the other hand it does make you think.  A great civilization came before us, a huge civilization that rose and covered this planet, and then was wiped out in its entirety such that none but bones remain.  We should remember this, we should remember our fragility as a species, as a planet.  We should think about the costs of the things we do to our world, and to each other, and we should remember that there is no guarantee that we will survive.

But now “never forget” seems to mean something different, something more insidious.  We apply it to the national tragedy of September 11, 2001, when a small group of Islamist terrorists committed a great atrocity in New York City that killed nearly three thousand people.  It is a testament to the power of terror and the dangers of an open society that such a small group — 19 actors — could commit such a large crime, and one so symbolic.  It was terrorizing as intended, and it embarked our country and the world on a new political, economic, and military path that has reshaped our modern world at the dawn of a new century.

“Never forget” is the wrong phrase here — we should remember the tragedy and honor the fallen innocents, certainly.  But the phrase became a rallying cry for two wars of revenge and destruction that have resulted in far more lives lost while arguably doing little, if anything, to make America safer or the world a better place.  It is not a cry for our shared humanity, but is instead a statement of division and anger.  The phrase itself has been twisted, turned petty.

I’m uncomfortable with my silly little dinosaur shirt, but not because I think it is wrong to wear it.  I’m uncomfortable because it forces me to confront the many mistakes we made after 9/11, the opportunities we missed, the actions we took from a place of fear and anger and sadness that were the wrong actions, with the wrong consequences.  We have this one world, this tiny precious world, this world we must all inhabit together, but through thousands of years of societal evolution we continue to repeat the same mistakes, to commit the same tragedies.

So the shirt does serve its purpose, even if accidentally — perhaps it is more meaningful, more impactful than I ever would have thought.  In that two-word phrase, “never forget,” is so much wrapped up meaning.  It makes you think.  Maybe it makes you think that I’m an idiot who got it all wrong, but you’re still thinking about it.  I guess that’s worth doing, and maybe September 11th of each year is the time to do it.

People silently struggle from all kinds of terrible things. They suffer from depression, ambition, substance abuse, and pretension. They suffer from family tragedy, Ivy-League educations, and self-loathing. They suffer from failing marriages, physical pain, and publishing. The good thing about politeness is that you can treat these people exactly the same. And then wait to see what happens. You don’t have to have an opinion. You don’t need to make a judgment. I know that doesn’t sound like liberation, because we live and work in an opinion-based economy. But it is. Not having an opinion means not having an obligation. And not being obligated is one of the sweetest of life’s riches.

Paul Ford – "How to Be Polite"


Water for Elephants

This review contains minor thematic spoilers but reveals no major plot details.

water-for-elephantsSara Gruen drew me in to her story of Great Depression-era circus life.  Meticulously researched and augmented with real photographs, this historical fiction novel is captivating and engrossing.

The framing of the story through the eyes of a reminiscing geriatric provides lots of opportunities for additional reflection and contemplation on changes in our world, although few are seized. His interactions with other patients and nurses are an interesting addition but offer little payoff.

Early on we meet roustabouts, carnival barkers, sideshow freaks, dwarves, and animals, and we learn about the pace of life on a traveling circus with clear class divides.  The “backstage” reality is gritty and tiring and hard-charging, and fascinating to behold. In the back third the pace of the story greatly accelerates and the romantic angle takes center stage, leading to a too-neat conclusion with little time to reflect on the human cost.

I can forgive the frequent telling-not-showing (with the requisite dumb as a brick narrator needed everything to be explained), I can overlook the romantic obsession, and I can accept the very limited development of secondary characters, but the present-day ride into the sunset ending is too ridiculous to bear.  I choose to believe that the modern-day denouement is just a crazed vision in old Jacob’s head, and somehow that makes this book work better for me.


Magic for Beginners

Screen Shot 2014-07-11 at 9.15.17 AMKelly Link’s Magic for Beginners short story collection is one of the most difficult books I have ever read. I loved every story with one exception, one that I just couldn’t get through because I was so uncomfortable with the cats and witches and death, but every other story was captivating and enthralling. Maybe I’ll go back and try that Catskin story again.

She does funny things with time, this writer. It goes forwards and backwards and sideways and spins around and comes back again. She has a strange way with literary structure — stories loop in on themselves and make my head spin. Each story is a little universe, each one pulls me in, and then ends mysteriously and evaporates into the mist. Each story is a dream, a long dream, a dream that ends when you wake up with the sun beating down and you wonder if you could have just slept a little longer how it might have all worked itself out but you will never know because that is how dreams are.

There is some strange twisted logic in these stories, like they are all in a weird magical alternate universe very close to but yet very different from our own. It is all so very frustrating. But so very worth it.

Running GitHub Enterprise in Amazon EC2

Update (2015-03-29): GitHub now supports an EC2 appliance and this information is no longer accurate. It is useful only for historic reasons or general background when confronting similar challenges from other vendors.

GitHub’s hosted offering allows companies to run their own private GitHub appliance behind their firewall.  It is distributed as an OVF container that runs under VMWare or VirtualBox.  But what if you want to run it, along with your other infrastructure, on AWS?  Here is the (completely unsupported) way to do it!

The goal is to get the base GHE virtual appliance running on AWS so that we can install the latest GHE software package on top of it.  This package takes care of updating and configuring everything.  Once the software package is installed, the appliance behaves just like its on-prem cousins.

Break into the virtual appliance

First we need the virtual appliance in a form that can be moved into AWS.  Download the current virtual appliance from the GHE dashboard and find a way to get at it.  You may be able to just launch it locally in VMWare or VirtualBox, if you are able to get root, but I did not do this Instead I extracted the archive (it is just a tar file) to get at the VMDK disk image inside, and attempted to import it into EC2 using the AWS VM Import/Export tool.

This requires some fiddling, because you have to install the old EC2 command line tools and get all the options right, with some plausible guesses about what is inside.  Here is the command I ended up running:

ec2-import-volume /var/tmp/github-enterprise-11-10-320-x86-64-disk1.vmdk \
 -f vmdk -z us-east-1a -b agperson-ghe -o $AWS_ACCESS_KEY -w $AWS_SECRET_KEY

Once the import is complete (you can check the status with ec2-describe-conversion-tasks) I attempted to launch it — and failed due to an unsupported kernel.  But never fear!

Figure out what’s under the hood

If you don’t want to do this yourself skip to the end of this section where I tell you the secrets.

The VM import creates an EBS volume.  It may not be runnable, but it is mountable!  So start up a one-off Linux instance and attach the volume to it.  The data is stored in LVM, so you may need to install the lvm2 package and then run lvmdiskscan to see the volume group.

Run vgdisplay to get the name of the volume group (“enterprise”) and activate it by running vgchange -a y enterprise. Now you can mount the root volume:

mkdir /ghe
mount /dev/mapper/enterprise-root /ghe

Poke around in this volume a bit and you will establish that the virtual appliance comes with Ubuntu 11.10 Oneiric (wow!) and is 64-bit. With this information, we can launch an equivalent instance in EC2.

Setup an Amazon-happy instance

Launch a new EC2 instance using the publicly available community AMI from Ubuntu for 64-bit Oneiric (make sure you are using the released version — in us-east-1 I used ami-13ba2d7a). I chose an m3.large which is a good baseline based on GHE’s requirements. Make sure to attach a second volume for data or make the root volume large enough to hold all your repositories, and use SSD storage because it makes life better. Put your new instance in a security group that allows traffic on ports 22, 80, 443, and, if necessary, 9418 (the git:// port, which is non-authenticated so often not used on GHE installs).

When the instance launches, login as the “ubuntu” user and become root. Modify the /etc/apt/sources.list to point all archive stanzas at old-releases.ubuntu.com (including the security ones). Run an apt-get update && apt-get upgrade and wait a few minutes.

Now you need to copy over all of the files from the virtual appliance. You can either do this via SSH from the one-off instance you launched earlier, or detach the volume from that instance and repeat the steps to get LVM running and attach it to the new instance. Either way, use rsync to get everything important onto your new VM. Rackspace offers a helpful tutorial on doing this, including a good set of directory paths to exclude. I used their list and everything worked fine. The command I ran with the volume mounted locally was:

rsync --dry-run -azPx --exclude-from="exclude.txt" /ghe/ /

(and once I was satisfied, I ran it again without the “–dry-run” flag).

Bombs away!

Before rebooting, copy your SSH key into /root/.ssh/authorized_keys in case anything goes wrong (and take a moment to ponder who Ben is and why his “HacBook-Air.local” key is on our server!). Then restart the instance and, when it is done booting, visit it via HTTPS to see the beautiful GHE setup screen! Upload the latest software package and your license key and give it half an hour or so, and if everything goes well, you will have a fully-functional GitHub Enterprise instance in the cloud.

Note that after the software package installs you will no longer have root access to the server. A pity.

A few other important steps are left as an exercise to the reader — lock down access, setup SES or some other email sending capability, stay in compliance with your license, and take frequent backup snapshots! Good luck!


The Internet’s Own Boy

Aaron Swartz

I’ve written before (and on several more occasions) about Aaron Swartz, a complicated and amazing person and digital activist who I followed and loosely orbited for many years.  Aaron did incredible work for and on behalf of the Internet as a democratizing medium, and he caused me to frequently question my own life and career choices.  He was targeted by an overreaching federal prosecution due to some of his activism work on the edges of the law, and after two years of pressure and abuse at the hands of the federal government, he tragically took his life in January 2013.

The Internet’s Own Boy: The Story of Aaron Swartz is a documentary film by Brian Knappenberger that traces Aaron’s life, his successes and failures, his political action and digital activism, his run-ins with the law, and his too-soon death.  Along the way several internet luminaries, journalists, activists, congresspeople, and other smart individuals weigh in and provide context.  I knew much of what the film presented but I still found it compelling.  While clearly opinionated, this film does a good job of portraying who Aaron was, what he believed in, and where things went so very wrong.

Many people feel that technology and politics together are too complicated, too confusing, and too inscrutable.  Many in government dismiss technology experts and inventors of things that have fundamentally changed our lives as mere “nerds”.  Aaron lived his whole life thoughtfully and fully, and his story is one that is approachable to anyone, technological or not.  The things he fought for are important, and they are comprehensible, and they should not be dismissed.  He showed how we can use technology and the internet to make this world a better one, and what we should do to stop others from using it to make the world worse.

The movie is imperfect, as was Aaron.  And it does a few funny things with time and ordering that slightly distort some of the major events in Aaron’s saga.  But on the whole it is thoughtful, and it is powerful, and it is worth watching.

You can view The Internet’s Own Boy: The Story of Aaron Swartz in its entirety for free on the Internet Archive.

European Conning

Meghan and I are currently attending a conference on a small island (!) in the Côte d’Azur — better known in English as the “French Riviera.” If this one is the norm, European-style tech conferences are quite a bit different from their American counterparts! Breakfast is early (7:30am), sessions run all day, and every evening includes dinner and drinks that start around 8:30pm and go past 11. The next morning, we wake up and start again.

Did I mention that every lunch and dinner is three courses and includes copious amounts of wine? And did I further mention that the wine is produced on this very island’s vineyards? Because yes, that’s a thing. And if you aren’t eating lots of paté and frequently emptying your glass, people start to wonder if you are ill.

We cut out early — it is only 10:30 — so Meghan can put the finishing touches on the talk she is giving tomorrow about user experience design. We also took a bit of time in the afternoon to wander around the island, which is quite a sight to behold — a few pictures above are a preview of the forthcoming album.

So far we have made friends with some Canadians, met some French and Italians, and had dinner with a contingent from Belarus. We learned that last year’s conference was held in a circus tent in Warsaw, so take that American conference centers!

It is fascinating to see people of all different languages and cultures come together to discuss their shared interest in technology. The opening speaker said that he hoped that everyone here would learn something new and then spread that knowledge by teaching it when they got home. I am reminded of how easy I have it as a native English speaker who never has to worry about a lack of documentation, examples, or online help. It is also interesting to hear from people in countries where there is much less appetite for working with modern, fast-moving languages and frameworks. Many of the people here really are ambassadors for and teachers of these technologies.

We are here for a couple more days, and then we will take a (ferry + bus + train) ride back up to Paris to spend some more time exploring that city’s wonders.