Geeking Out

Capistrano multistage deploy configuration stored in a YAML file with MultiYAML

I spend a lot of time working on deploying a variety of software applications smoothly to different environments. A tool central to my workflow is Capistrano, an SSH-based deployment framework written in Ruby.

In its Ruby-ish way, Capistrano’s multistage functionality requires stubbing out different Ruby files for each stage — staging, production, etc. In our environment, I decided it was better to instead store all of the per-stage configuration in one single configuration file, and I chose to do it in the simple YAML format.

There are several advantages to this approach:

  • The file format is straightforward and can be modified both by humans and scripts, including automatic updates from a central source of truth.
  • There are fewer configuration files, and within the single configuration file there is much less repetition of configuration, because we can use YAML’s built-in anchor/alias functionality.
  • It strongly encourages storing deployment logic in the deploy.rb file and hooking tasks using Capistrano’s before/after callback functionality, rather than building stage-specific tasks.

The module I built is inspired by Jamis Buck’s original Capistrano multistage module, as well as Lee Hambly’s prototype YAML multistage extension, which was never packaged and is no longer maintained.

My capistrano-multiyaml module is available on GitHub along with documentation, and can be installed via RubyGems.

As it gets easier for one member of a group to destroy the entire group, and the group size gets larger, the odds of someone in the group doing it approaches certainty. Our global interconnectedness means that our group size encompasses everyone on the planet, and since government hasn’t kept up, we have to worry about the weakest-controlled member of the weakest-controlled country. Is this a fundamental limitation of technological advancement, one that could end civilization? First our fears grip us so strongly that, thinking about the short term, we willingly embrace a police state in a desperate attempt to keep us safe; then, someone goes off and destroys us anyway?

Bruce Schneier

Chilling.

Serious steel

Japanese Santoku KnifeThis week I bought my first “adult” kitchen knife, and I’m excited. As I have gotten a bit older and just a tiny bit wiser, I have started to realize that the matched set is less important than the single item of quality. In this case, that means getting rid of a block set filled with fifteen matched knives of all shapes and sizes — most of which I never use — in favor of three or four really nice cutting implements that I will use every day.

I’m not obsessed with having the best of everything — that involves a level of time commitment and monetary outlay that is at odds with my lifestyle — but I am interested in quality. I lean heavily towards very good single-purpose tools rather than mediocre multi-function devices.

When it came time to purchase a new television of a larger size, I settled on one that was not the most expensive — not by a long shot — but was very good at being a television. That’s it, just a television. No “smart” features, no whiz-bang 3D or other fads, no super-fancy speakers or internet-connected doodads or motion-control thingamabobs. It is just a television, and I am very happy with it. While peripherals and set-top boxes and remotes will change, I think this television will stand the test of time.

My first adult knife is a 7″ Japanese Santoku. I love holding it and feeling its weight and balance in my hand. I am looking forward to some serious chopping and dicing in the near future. I enjoy spending time in the kitchen, and perhaps this new high-quality tool will push me to up my food prep game a bit.

Geeking Out

Data privacy and security in 2013: Cloudy!

On Friday I attended a Data Privacy Day (a real thing!) panel co-sponsored by HUIT and Harvard’s School of Engineering and Applied Sciences called “The Intersection of Privacy and Security“. The panelists were noted Harvard technology graybeard Scott Bradner, always interesting professor Salil Vadhan, and SEAS computing director Steve King.

After some brief introductory remarks by the panelists about balancing privacy and security, the floor was opened. I seized the opportunity to ask about something that has been much on my mind lately: how to make sensible personal choices about data privacy (and security!) in an age of highly-connected devices heavily depending on third-party hosted services.

Or to boil that down a bit more: Let’s say I have a phone, a tablet, and a laptop, a pretty common set of devices these days. And let’s say I use them all constantly. And these devices are tracking what I read and listen to, who I talk to, where I go, what I buy, and every email, chat, and text I send and receive. They are syncing this data between each other and up to an amorphous “cloud” service, where my data is being collated, cross-referenced, sold to marketers, and stored forever.

Given this fact situation, how can I, as an individual, make sensible privacy and security trade-offs, when in order to get the maximal value out of these devices, I must cede control of my data — both the privacy of it and the security of it — to a third-party vendor such as Google or Apple?

A variety of answers were given, none of them entirely comforting. From Bradner, first, came the cynical view — pay in cash, forego loyalty programs, do not use cloud services, and assume everything you store online will be there forever. This is a valid answer, and rock-solid from a data privacy perspective, but I don’t consider it very practical.

His next suggestion was an interesting one, and that was to look for natural alignments — is the corporation I’m entrusting with my data looking out for the same things as I am? His example, backed up by King, was Google’s track-record on fighting invalid data requests from governments and safeguarding customer information. They do this both because that information is valuable to Google, and because customer confidence in Google is also valuable to their bottom line. This raises some interesting and difficult questions — with a company as far-reaching and often secretive as Google, how can we know their actions and track their intentions? For how long will my interests align with Google’s, and when they inevitable stop aligning, how can I erase my digital life from Google’s clutches?

Professor Vadhan I believe was the one to bring up some of the regulatory remedies. Data privacy laws, when well crafted, could help to protect individuals from corporate data misuse, and perhaps even some types of government data misuse. Europe has tried several approaches to this, with mixed success. But such regulation is not on the docket in the United States currently, so that solution doesn’t provide any immediate guidance. And, Professor Vadhan admitted, he clicks through every terms of service notice and privacy agreement without reading it, just as we all do.

In my view, and seemingly that of the panelists, there is no clear path forward at present for this problem. For now we must all work to inform ourselves about risks, balance the trade-offs, and make decisions that we are comfortable with. So maybe I will use the CVS loyalty card, but not link it to a credit card. Or I will use Google’s Gmail service, but not Google+. This is complicated, time-consuming, and frankly difficult — Facebook’s privacy settings, for instance, shift frequently in unexpected ways, often without notice. Opting out of online services’ choices about how to use our personal data is becoming more and more difficult — perhaps because they see it as their data.

With no easy answers on individual data privacy, we can only muddle on as we have been doing, and hope for clearer, easier choices in the future. Meanwhile, the data we share ends up in unexpected places. The only silver lining, in my view, is that I’m not convinced that putting something on the internet does necessarily mean it will be there “forever”. The internet does seem to forget, or, if not forget, at least the constant deluge of new data seems to moderate and bury the old, in ways that can only be good for our lasting well being.

RIP, Aaron Swartz

Cory Doctorow’s moving tribute to Swartz, 26, who was recently found to have committed suicide. I never knew Aaron, but I’d occasionally see him around Harvard. I recognized him because I followed his blog, digital activism, and standards-making work since I was in high school. He was a brilliant and driven thinker and doer in the digital law and public policy space. He also helped create (or at least rewrite) the early Reddit, crafted the Creative Commons license framework, and helped build the RSS specification. That’s a lot to accomplish in a lifetime, and he did it all in his teenage years and early 20s. He is a few years younger than me, and at times I found him inspiring, at other times inscrutable, but always I kept an eye out for his latest work. Sometimes I wondered — if I had done things differently, been more passionate, just a bit smarter — if I could have been like Aaron. Now, learning about his demons, I’m just sad for him, and for us, who no longer have him around.

iPhone lover “confesses” to switching to the Nexus 4

Ralf Rottmann lays out his reasons for switching from an iPhone to the newest Google Android device. I too have been interestingly eyeing a Nexus 4, for many of the same reasons he expresses. The iPhone has consistently won out with both hardware and software design, quality of user experience, and availability of compelling applications. But Android has finally improved to the point where those advantages are greatly diminished. Meanwhile Android’s advantages with regard to intra-app communication, more seamless sharing, and Google service integration have not been matched on the iPhone side, no matter what the misleading Siri advertisements claim.

My concern continues to be what it has been from the beginning — in addition to wanting a device that works reliably and well, I also want one that belongs to me. I am extremely wary of loading my entire life into Google’s digital vaults, from which it can never be fully extricated, is subject to massive amounts of collation and analysis, and is sold to their advertisers and kept for their future uses.

I do use Google’s Gmail service for my email, but I pay for it, and I wish that along with that payment would come a much more significant promise to segregate my data and not use it for evil. Until such a promise is more clearly expressed, I don’t trust Google’s Android ecosystem any more than the rest of the services they provide — the Chrome sync that I disable, the Web History that I purge, the mandatory public Google Plus profile that I strip of all useful information, etc. It’s not that I’m afraid to share information online, simply that I want to control my own information and make my own choices, and have the right to change my mind in the future. I don’t think that is too much to ask.

Gun violence triptych

We don’t have to go far. Following the tragic school shooting yesterday in Newtown, Connecticut, the New Yorker is out in force with a set of devastating reactions, all worth reading:

I’m seeing the same sentiments all over the media I read and among the people I follow on Facebook and Twitter. I hope this is a trend, and that we may finally, finally have the momentum to take meaningful action in America on firearm legal reform. Remarkable — yet not — that it took the mass murder of nearly two dozen kindergarden children to get us here. If we don’t take action now, I’m not sure we ever will.

A Portrait of the Systems Administrator as a Young Man (Part 1)

When I started at the Instructional Computing Group, I aimed to be subservient. I was coming from a small fast-paced research center with a lot of strong personalities, complicated politics, and limited technical resources. I had charted my own path there, and I thought it was a good one, but now I was ready to learn how the “professionals” do things. Plus I was looking forward to working normal, non-crazy hours.

Through reorganizations and office moves, I stuck with the philosophy that I was here to do a job, and it would not behoove me to be involved in the politics of the place. Boy was I naive!

I came into the group tasked primarily with maintaining and improving three servers, and secondarily with building some new stuff. Anyone who knows me can guess I was much more interested in the latter than the former.

This was a transitional time in IT, or at least in Harvard IT. At my last job I had stood up virtualization, letting a few powerful servers run many smaller virtual systems. This was a pretty new thing, but it was awesome, and I just assumed everyone was going to be doing it soon. I had no idea at the time how conservative IT organizations often are, or how fragmented. Yes indeed there were virtualization initiatives — four of them that I was eventually became aware of, some well-funded and some small, some for Windows and some for Linux and some for both, none out of the pilot phase.

To do cool new stuff for instructional computing, I needed hardware, and if we were speccing out a few powerful (and expensive) servers, and there was no current virtualization solution available and supported, it just made sense to me that we should install VMWare or Xen on them (this was before KVM) and spit out a bunch of little VMs we could use for experimentation and student projects.

This was when I (and my boss) discovered that things were changing, control was being centralized, and being an “Instructional Systems Administrator” meant pretty much squat when it came to making decisions about infrastructure. Our request was not outright denied, it was just delayed and eventually pocket-vetoed.

Given the new realities, and the lack of enough work on the existing systems to fill my days, we came to the conclusion (my boss and I) that I should be embedded in the systems group part time, giving them a hand, learning about their technologies, and advocating for/formulating plans for ICG’s technical future.

And here’s where things went awry again — the UNIX/Linux team didn’t want me, I got to go sit with the Windows folks. Which is fine and all, but not at all relevant to what I was hired to do. Nor did they seem to much understand what to do with me.

Four months into the new job, I had accomplished very little, and things were not going very well. So I took a vacation to Barcelona. Sitting in a hot hostel common room after a long day I was finally able to cajole my Linux netbook onto the wifi. The first email that came in was from my boss, telling me he wasn’t my boss anymore, and I had been reorganized.

Coming in part two of our gripping tale: becoming a technical architect!

Two days back with Fitbit and I’m already annoyed

Yesterday morning I picked up my old Fitbit sensor and plugged it back in for the first time in about two years. If I’m going to do this thing, I want to do it right — record all food eaten, record weight and body fat each morning, etc. And ostensibly Fitbit supports this. But it is stunningly opaque. It flashes up calculated data such as calories burned that fluctuate wildly throughout the day. So I’m eating based on its estimate of 2300 calories, and at the end of the day when I sync it up I’ve walked 11,000 steps and 5 miles, and the little flower tells me I’m being active, but my “activity” score is 15 (out of 1000?) and my calories burned are down closer to 1800, way lower than it was predicting.

And “whoops,” it tells me, “you’re over your daily calorie goal.” Well, I wouldn’t have had that cookie after dinner if you had just given me some consistent data! I think this is why I gave up on Fitbit pretty quickly last time — I don’t want your faked up data, I don’t want your opaque “scores,” I don’t want your meaningless graphs that say I’ve been sedentary all day and seem to ignore the walks I took and stairs I climbed. The hard truth is that these pedometer-based fitness tracking systems are all hopelessly inaccurate and seem to verge on pseudoscience. If all you care about is footsteps, go nuts. For anything better, you’re going to need something much more sophisticated.

Funny, you would think the amazing tiny pocket computers studded with sensors that we all carry around nowadays and refer to as “phones” could help with this problem!

Travel

Scandinavian Adventures: Copenhagen and Zealand

Copenhagen, the guidebooks say, is packed with museums and historic buildings. Brochures show pictures of charming storefronts and cafes fronting a canal. Tourism literature uses words like “eclectic” and “funky” to describe parts of the city. And in the middle sits an old-timey amusement park, something like Coney Island in its heyday.

All these things are true, after a fashion. But there is a lot left unsaid.

Copenhagen is a real, vibrant city, not a fairy-tale land of history, architecture, and clean streets. It is packed with cars, construction, dirty back alleys, and even (in spite of a massive social safety net) street beggars. Which is not to say I hated it — I loved it. But I had to wildly shift my expectations.

After the whirlwind museum tour of Frankfurt, we were content to just spend some time wandering in Copenhagen without a clear itinerary. Given the sheer density of old buildings, squares, fountains, churches, and other interesting architecture, not to mention canals, this worked out rather well. We also worried a bit less about “authentic” local cuisine — our first meal was shawarma, our last meal with steak, and in between we did try the local street hot dogs (not very good).

In terms of Copenhagen things done:

  • Canal tour – So-so, it mostly served as a chance to doze.
  • Tivoli Gardens – one of the oldest amusement parks in the world, Tivoli lived up to its billing and was actually quite an enjoyable time. Things to see, churros to eat, roller coasters, bumper cars, swing dance (we did not participate) and interesting typography abounded.
  • Christiana – This artist commune (or something?) had a strict “yes pot, no photos” policy, was full of a lot of scruffy-looking people, stray dogs, guitars, and beer. I expected something less grungy and more hippy; we didn’t stay long.
  • Sandcastles – Oh gosh yes. This summer’s theme for the exhibition was the history of man, and the sand artists of various nationalities were certainly creative in their interpretation.

All this in photo form using the link at the bottom.

One of the guides I read said that tipping of wait staff in Copenhagen is unnecessary, and once you experience the service you won’t want to anyway. That advice was spot-on. Never before have I had a quick stop in a cafe for some tea and pastry taken almost an hour and a half.

We spent as much time outside of Copenhagen as in it, although we confined our adventures to the Zealand region (sort of the “greater Copenhagen area) to maximize our limited time. This included a train trip north to Kronborg Castle in Helsingør, the setting for Shakespeare’s Hamlet and for a time Denmark’s most important fortification. The castle is situated to control the Sound and allowed Denmark to tax all ships trading there, including those bound for Sweden. As you can guess, this didn’t go over well for the Swedes, but then the two countries were arch-enemies for centuries.

The castle was an impressive sight and a very interesting experience. Co-located there is a naval museum containing far more model ships than anyone really needs to see, as well as a tower with a great view of the surrounding city and Sweden across the water.

We also stopped in at the Louisiana, a well-regarded modern art museum located along the coast in an otherwise sleepy town. An interesting piece of architecture with low-slung buildings and beautiful gardens, the Louisiana also contains a museum shop filled with wonders and was featuring an exhibit on “new Nordic architecture and identity”.

We learned the difference between “Scandinavian” (incorporating Denmark, Sweden, and Norway), and “Nordic” (which also includes Greenland, Iceland, Finland, and various smaller islands and territories). All of these countries share deep cultural and social links, but also a long history of mutual aggression and occasional conquest. The exhibit explored how art and architecture both highlight cultural differences and serve to bring people together. It also asked difficult questions about how globalization and homogenization clashes with identity rooted in a sense of place.

On the whole I found the modern Nordic architecture quite compelling, and an excellent counterpoint to much of the design one sees in the US today. I wish we had more time to explore this museum, but despite the sun being out past 10pm, most things don’t stay open much later than 6.

Our second excursion was to the Viking Ship Museum in Roskilde. Kevin and I both really wanted to try and swing this, and while it meant giving up more time in Copenhagen, it was well worth the travel. Sadly we did not arrive early enough to reserve a spot sailing (and rowing) on a reconstructed Viking ship, but we did have plenty of time to explore the museum and grounds. Half a dozen reconstructed ships in various styles sit in the bay, one of which we were allowed to climb on and explore (mind the pine tar, it stains clothing!). There were also stations with information and demonstrations on various aspects of Viking ship construction, including rope making, sail weaving, wood carving, and smithing.

The sheer quantity of human effort that went into building these imposing vessels is astounding. While the Vikings had iron, looms, and a heck of a lot of expertise, they lacked such implements as saws and drills. Which meant that each tree had to be felled by axe, laboriously chopped, carved, and chiseled. Every nail had to be hand-forged, one at a time, after mining and purifying the iron ore, of course. And every strand of rope had to be harvested, peeled, washed, dried, beaten, woven…

The museum proper was built to house the discovered remains of five ships scuttled in the Roskilde fjord in the 11th century to block a channel against approaching invaders. The ships were raised 50 years ago and the fragments reconstructed. It is amazing to view ships over a thousand years old and read about the (believed) customs and traditions of the Vikings of that era.

Following our Viking adventures we visited a nearby restaurant on a mission to get in one good meal of thoroughly Danish fare. In the spirit of Scandinavian adventure, I ordered the “Five Ships Platter,” which consisted of herring, chicken salad, smoked salmon, spelt salad, cheese, and a small glass of mead.

My lunch

Kevin will confirm that I tried every item. As to which I finished, well…

Our brief Denmark trip an overwhelming success, we are off to our next destination, across the water: Sweden!

View photo slideshow

Travel

Scandinavian Adventures: Frankfurt (Day 2)

All that was promised for day two was accomplished! The Communications Museum was great, and the architectural museum was awesome, especially since every descriptive tag had a German and English side. When we got to the top floor, the museum itself got a massive descriptive tag — all very meta. Sadly no photos of either, owing to silly museum policies.

Just try to imagine a herd of sheep created from old-fashioned rotary telephone handsets and coil wire. And, far to the side, a black sheep similarly composed. And a wall of mailboxes spanning the last 200 years. And an exhibit about the internet! (I hear it’s going to be big.) And a mail sorting train car you can go inside! And fax machines!

The architectural museum was all about architectural models, dioramas, and other design artifacts. There was an exhibit about utopian architecture that was sort of scary, and a nice history of human architecture tour, with a few digs at the end against American over-consumption. My favorite designs were amazing tensile structures by Frei Otto.

We also toured the entire Städel art museum in less than an hour; I most enjoyed their new underground contemporary wing, but by this point we were getting a bit museumed out.

We stumbled upon a delicious Turkish place for lunch and got our best service experience so far (the German restaurants we’ve tried have had terrible service), and then walked 2 km to the botanical gardens. We hung out there for a while, seeing some wonderful sights, and got to the tram stop just in time to beat the oncoming rain. I’ve updated the Facebook photo album, linked below, with pictures from day 2. Sorry non-Facebook friends, but 500px doesn’t support mobile uploads, and this trip is exclusively iPad.

Tomorrow we are off to Copenhagen!

View photo slideshow

Travel

Scandinavian Adventures: Frankfurt (Day 1)

We are calling this trip “Scandinavian adventures” although the actual itinerary consists of brief hops in Frankfurt (Germany) and Copenhagen (Denmark) followed by several days in Sweden, divided between Stockholm in the north and Gothenburg in the west.

This is my third European jaunt with Kevin, and this time we departed late Wednesday night from Boston on a direct overnight flight to Frankfurt. We both slept much of the journey and were able to hit the ground running (well, walking — shuffling really) when we arrived.

Lodging in Frankfurt is a cozy (read: tiny) apartment we booked via Airbnb. We are situated on a tram line near the center of Frankfurt and “steps away” from the Main river and museum row. After navigating trains and subways from the airport and getting situated, today consisted of some exploration on foot.

Lunch at a well-known local haunt consisted of Frankfurt delights, specifically a somewhat tasty beef brisket and potatoes with “green sauce” and some yucky apple wine. We followed this with a “romantic stroll” along the riverfront (although Kevin doesn’t make a very good Meghan-substitute, no offense) and some time exploring churches, sampling gelato, and being utterly confused by the museum of modern art.

After some downtime (read: napping) as we strive to adjust to the time change, we wandered off for a German-Italian dinner, where we were frequently visited at our table by unruly dogs while the staff and their dog-owning friends stood around drinking wine.

Tomorrow: more museums! Botanical gardens! Perhaps some German-Turkish fare (or at least a frankfurter)! Trying to figure out how to call family members who are 9 time zones away!

View photo slideshow

Surprises in the garden

This is my second year of serious “square foot garden” experimentation, and so far it has not been going nearly as well as last year. My lettuce was infested with little bugs, which apparently is par for the course. After copious washing I got a nice (if somewhat bitter) salad out of it, but was constantly worried about finding more bugs in my dinner. I didn’t end up using the other two heads, and eventually threw them out.

I kept seeing maturing strawberries appear, but by the time I got to them they had disappeared — apparently eaten by birds. Now I’m putting up bird netting to try to protect my other berries, none of which have come in yet. My bell pepper seedlings refuse to grow, for no clear reason. My broccoli bloomed and was ruined because I wasn’t paying attention. And when I just pulled my single head of cauliflower (1 per square foot), it was infested with both earwigs and little green worms. Yuck!

I’m tempted to call the whole thing off and go back to just getting everything from the supermarket. Or, I guess, learning about pesticides. Bah.