Why Web Development Is Complex

I keep hearing things like “programming is easy” and “everyone should code”! These are both interesting, kind of misleading statements. The actual core part of programming — actually writing code, is perhaps not that hard. Writing simple code is, relatively, simple. Many more people can, and should, probably learn to do that. Actually being a good developer is vastly different (and massively more complex).

It’s a worthy goal for people to get into development, and I love that more and more folks are, but you should also set your expectations realistically if you’ve never coded anything before. You can learn some basics pretty quickly, but you have a lot to learn before you will be fully proficient in a real development environment.

If you’re building something big-ish on the web, here’s a non-exhaustive selection of the things that are (or potentially are) relevant to that process:

  • HTTP
  • File paths
  • DNS
  • Servers and Clients
  • Databases
  • HTML
  • CSS
  • JavaScript
  • Browser Compatibility issues (affects all HTML, CSS and JavaScript!)
  • Web Server configuration and tuning
  • RTL languages and their effects on your designs/layouts
  • Internationalization, including keeping your designs flexible enough to handle variable-length text
  • Search Engine Optimization, normally based on dynamic, user-input data
  • Semantic markup
  • Accessibility
  • Design
  • Color theory
  • Visual hierarchy
  • Graphic design/image manipulation
  • Image optimization
  • Web fonts/Icon fonts
  • Image sprites
  • FTP
  • SSH
  • Scalability
  • Memory management
  • Performance bottlenecks
  • Mobile web development (infinite screen sizes)
  • Responsive Design
  • Touch vs Click interaction models and considerations
  • Offline considerations/handling (for “live” web apps)
  • Version/Source Control (Git/SVN)
  • Visual/Text Editors
  • The command line (and everything that comes with it)
  • Retina/HiDPI screens vs “normal” screens

Depending on the specific technology stack you end up using, there will be entire swaths of other technologies, libraries and tools involved as well.

Guess what I didn’t even actually mention in that list? A server side programming language (unless you’re going to use Node.js and write JavaScript on your server perhaps). So all of that other stuff up there doesn’t even actually get you writing the code on your server (PHP/Python/whatever) to do the heavy lifting that you’re here to do.

Also, just so that you’re aware; in the 15 years I’ve been doing this, the list never gets shorter, it only gets longer.

Personal Location Tracking

I’ve been pretty fascinated with the idea of recording my own location for a while now. I started using Foursquare at SXSW in 2009 and have mostly continued to do so since then (I have over 3,700 check-ins). You can see my check-ins being syndicated back to this website (using Keyring Social Importers), and if you scroll back through the history of the main page, you’ll get maps aggregating a few check-ins at a time.

TripIt helps me keep track of all (most) of my travels, and provides back some data (via API), which I also import into this site. Here are all of my trips since March, 2008.

In February last year, I started using Moves, and quickly came to love its simplicity. It’s a background app that runs on your phone and keeps track of your location. Using server-side data processing, they crunch the raw location information to figure out when you were walking, running, riding, or on some form of transit, then give you back a timeline and a map showing what you’ve been up to. It’s a really nice “set and forget” way of keeping up with how many steps (roughly) you’re doing each day, plus your other forms of exercise. The app has continued to make small improvements, and then on April 24, Facebook bought them. I can’t say I’m stoked about the acquisition, but regardless, it’s a cool app, and it collects some fantastic data.

Since it’s all data, and there’s a growing sphere of location/movement-related data services out there, shuffling your data around is just a matter of a little programming. As I mentioned, I’m importing my Foursquare data into my blog already. I also have a Moves importer that’s currently creating a text-only summary of my information. I’ll probably add simple maps to it at some point. Moves-Export is a pretty neat service that will automatically import your Moves data and can give you a better breakdown of things, plus auto-post to Runkeeper and Foursquare (if you like) when activities are over certain thresholds (e.g. riding for more than 15 minutes). Pretty awesome.

Today, Chris Messina tipped me off to Move-o-scope, an awesome web app that will slurp in your Moves data, and give you back a rich visualization of it all. It lets you toggle things on and off, pan around the globe and see what you’ve been up to. It’s fascinating. Here are some places I’ve been since last February!

It’s fun to turn on the “Transit” layer (orange/brown, seen in the last picture above and the first one in the post), and follow the lines around the globe to see where you’ve been, then turn it off and zoom in to get a feel for what ground you covered while you were there.

A Whole New Dented Reality

Back in April, this blog celebrated 10 years of existence, and it’s been almost five years since the theme on this site changed. Yesterday I decided to just go ahead and flip the switch on something I’ve been working on here and there since late last year. It’s a complete new, very experimental theme that I call “Homeroom“.

There are some specific things driving what I was aiming for with Homeroom:

  1. First and foremost, a lot of the decisions are based around the intention that it would use Keyring and the Social Importers to pull in my content from all over the web. With that much data being collected and displayed here, I realized I couldn’t go exactly with a traditional blog layout, and had to get a bit creative with some types of data.
  2. It’s intentionally heavily integrated with Jetpack (although it works without it). Jetpack powers the comments, infinite scroll, sharing buttons and more. I’ve taken care to try to make that integration feel as native as possible (although I know there’s more to be done there).
  3. Homeroom started out as an _s-based theme, although it’s been pretty radically modified from there.
  4. I’m using a technique that I’ll call “post lookahead” when going through the loop to check the next post, and do some things like collapse sequential posts if they’re the sam type of thing.
  5. I wanted something with a bit of a timeline feel, since it’s now collecting some much data, and it’s all sequential; I wanted to show the relationship of different things along that sequence of time.

It’s not particularly beautiful because, well, I’m not a designer :) In the near future I’ll be talking to some friends who are though, so hopefully I can get some advice on improving things there. I’ve been mainly focused on getting it working the way that I wanted it to. Here are some other bits that might be interesting:

  • Allows custom Header and Backgrounds via wp-admin
  • Options (currently defined in a file, because I haven’t built a UI) to control some preferences around stuff like hiding Foursquare check-ins until a set amount of time after they happened (to help avoid the creepers!).
  • Heavy use of Post Formats
  • Ability to hide Twitter replies, which you probably don’t want showing up on your site.
  • There’s the beginnings of a front-end post box (partially inspired by o2), although it doesn’t actually work yet
  • Handles Foursquare checkins differently. Rather than showing them all as individual posts, it “collects” them and shows a single, multi-point map at the end of each day.
  • Shows a map automatically on any post using the WordPress-recommended postmeta fields.
  • Using the new taxonomy introduced in Keyring Social Importers 1.4, allows you to easily filter your display based on where posts were imported from.
  • Includes a custom icon font from IcoMoon to display social icons indicating where things were imported from.
  • Search results use a Masonry-based layout so that you can quickly scan the results. Unfortunately something is broken with the search mechanism on this site right now, so that’s not working :(
  • Automatically lists out child-pages when you view a Page that has them, for example my Projects page.
  • Dynamic heading re-writes: format your posts for individual viewing, and the H1 etc tags are automatically “stepped down” on listing pages to maintain hierarchy
  • Has some fun mapping stuff for TripIt in particular, which draws out a “flight path” between airports. Check it it out in my TripIt section. Here’s a fun one.
  • Uses Photon to apply some effects to images in places
  • Borrows liberally from the styling of sites like Instapaper and Readability.

There’s still a lot of work to go, both on the theme itself and the importers that power a lot of the content. I wanted to get this online because I knew that’d motivate me to spend more time on it. I’m also hoping that other folks might be interested and/or have some ideas on ways to improve the theme. I haven’t got all of my content imported yet (that takes a while :) ), but you’ll see more and more things fill in over the coming week hopefully.

If you’ve got any ideas for improvements, I’d love to hear them down in the comments!

o2 at WordCamp San Francisco

I just realized that I never posted anything about speaking at WordCamp San Francisco. This was my 8th WordCamp SF (I’ve been to every one since the first, in 2006), and the second one at which I have spoken. Matt invited me to give a short introduction to the work we’re doing with o2, which is the next generation of P2. It’s not available for external (non-Automattic) use yet, so I had to settle for a relatively surface introduction, and couldn’t give people a link to download it or anything which was a bit of a pity, but it still got a good reception.

o2 is a pretty different approach to building on top of WordPress, and has meant a steep learning curve for my team and me. We’ve been digging deep into the world of front-end development, and ramping up quickly on Backbone.js, Underscore.js and a bunch of new development approaches and workflows. It’s been really fun. I’ll let the presentation do the talking though (you can also watch it on WordPress.tv):

And here are the slides I used, which you can also see on Slideshare.net.

We’re really excited to get o2 out into other people’s hands, but we’ve got a lot to build still before other people can experience it in a similar way to how we do at Automattic. The future is bright.

I Want to Love My Pebble

It’s true, I really want to love the Pebble that I got from being a Kickstarter backer. I want to, but right now I can’t. I can only like it. And I can like it a lot — there’s a lot to like!

  • I have a computer on my arm!
  • I can read text messages without taking my phone out
  • I can control music that’s playing on my phone
  • WordPress notifications on my wrist? Yep.
  • Calendar alerts? Got ‘em.
  • The form-factor is slick: it’s slim, super lightweight and IMHO, looks pretty darned cool.
  • Nice backlight, which I can activate by shaking my wrist or tapping the watch
  • It’s waterproof! (although I’m too nervous to actively put that to the test)

So why can’t I love it? Let me count the ways (biggest reasons first): (more…)

Import DokuWiki pages into WordPress

I used to maintain a wiki full of personal ideas using DokuWiki, but at some point just gave up with wiki syntax entirely. A few months ago when I was moving all of my sites and content over to a new server and trying to consolidate things as much as possible, I decided to import all of that old content into a WordPress install (which was actually a single site within the same Multi-site install that runs Dented Reality). I ended up writing the following super-rough script to just scrape the contents of the pages and throw them into WordPress. Scraping the pages meant that I could get the actual output of all plugins etc, and also get full links between pages.

(more…)

TimesOpen Hack Day/Readtrack

On Saturday, I attended the 2012 New Times/TimesOpen Hack Day. It was a long day, but I had a lot of fun. I sat in on an intro session to Arduino which was pretty cool, and also a session on the EchoNest API, which I ended up using in my project. You can find out all about my project on the Readtrack project page.

It’s a bookmarklet-powered little app that analyzes the page you’re looking at (using the AlchemyAPI) and then tries to find related music (using the EchoNest API) which it then plays back to you in your browser (using rdio). I got a “runner up”/honorable mention prize :)

One of the most visually-polished projects was “Story Arc”, which showed a visual representation of the frequency of mentions of keywords over the NYT archives. Probably the most fun one was a set of drivers for a DDR pad, hooked up to commands for things like deploying code!

Jetpack 2.0, Packed With Magic

Last night, I preemptively tweeted about the upcoming release of Jetpack 2.0:

(more…)

Where is Your Digital Hub/Home?

I’ve been using WordPress to power my own website for a while now, and working with it in some way or another for even longer. Over the years, I’ve developed the belief that it’s a pretty perfect platform for people to build their own “digital home on the web”, considering the range of plugins and themes available, the flexibility of the publishing options it offers, and the fact that it’s completely open source, so you can do whatever you want with it.

That last bit is important in more ways than you might immediately think. Apart from just being able to write my own plugins or tweak my themes, this also means that I own my own data. I think in this MySpace/Facebook generation, people are all too loose with the data trails they create — giving up ownership of their digital self at the drop of a hat. In case you didn’t realize, when you use something like Facebook, it is not the product, you and your data are the product.

(more…)

Moving Jetpack Sharing Buttons

I’m working on a new WordPress theme (for this site, and it’ll be released for download once complete). The theme is deeply integrated with Jetpack, and one of the things I wanted to do was have the Jetpack Sharing buttons appear in a location other than the very end of the content. Normally they are applied as a filter on the_content, so they just appear right at the end. I wanted to relocate them into a different location, and it turns out that’s really easy to do with the power of jQuery.

jQuery( document ).ready( function( $ ) {
	// Relocate Jetpack sharing buttons down into the comments form
	jQuery( '#sharing' ).html( jQuery( '.sharedaddy' ).detach() );
} );

The #sharing selector is just the DOM location where I want to move the buttons to, and the .sharedaddy one is the container that Jetpack places its buttons in normally. We just detach it from the normal position and then dump it into the new location exactly as it was.

Welcome to the new server

I’ve finally gotten around to moving this site (along with all my other random sites) onto a single server. It’s all now hosted at Media Temple, on a 512MB (dv). We’ll see how that goes. I’ve got a bunch of things to write up once the move is complete, but if you’re seeing this then it looks like we’re most of the way there.

There’s definitely some more tuning and tweaking to happen, but at least I have my memory usage below 100% :)

My Development Setup/Flow

Developers seem to love to hear about how other developers work, so I thought I’d try to capture my entire environment, from end to end, in a single post. This will change (has changed) over time and depending on the project/company/whatever, but this is how things are for me right now. A couple of points up front:

  • I work for Automattic, so a lot of this is influenced by our internal policies/security/workflow.
  • I don’t always use all components of this “system”. I’ll try to detail when I do/don’t use certain parts of it as I go.

OK, here goes.

Note: This turned into a little bit of a summary of how we work internally at Automattic as well. Oh well, maybe it’ll provide some inspiration, I think we do some pretty cool things.

(more…)

Stroll in the Park

It is now 12:04am, and I just got back from a walk.

There are a few things about this walk to make it notable:

  1. I got up from my desk at 11:35pm and headed out for a brisk walk, mainly so that I could try to get over 6,000 steps today.
  2. I measured my steps using a Fitbit, and had the specific target because it’s part of an internal fitness challenge Automattic is hosting through Keas.
  3. I took the chance to compare Fitbit to RunKeeper for measuring walking (mainly looking at distance accuracy, but also at calorie burn).
  4. A guy (probably high) most definitely lined me up to attempt to mug me, he even tried to walk with me/talk/engage to distract me and get me to stop, but I out-walked him and he kind of gave up.

So; fun stuff all around. Here’s the data, from RunKeeper:

and for the same time period from Fitbit:

What’s interesting here?

  • FB reports that I burned 163 over RK’s 126; that’s a 23% difference.
  • FB reports that I covered 1.22 miles, while RK reports only 0.76 miles. Almost 40% difference.
  • Pace is barely even worth comparing when distance is so differently recorded. Time even shows as 40 seconds different, although I’ll give FB the benefit of the doubt and assume it’s rounding.
  • You can actually see the part where the guy approached me, because I sped up. If you look at the map from RK, it was on Larkin St, between Broadway and Pacific. He tried to “walk with me” so that he could get me to stop, but instead I just walked faster. I topped out right there at 14.4 minutes per mile according to RK (just over 4 miles per hour).
  • If I hadn’t been trying to meet a daily fitness goal, I wouldn’t have been walking around at midnight, making myself a target for getting mugged :)

Maybe I should run instead of walking, since high/homeless/whatever people are even less likely to give chase?

Closing a Window in JavaScript

If you’re having trouble closing a window using JavaScript, this might help. I’ve had this problem sometimes in Chrome in particular when I open a window, then follow through a bunch of redirects or something. It seems to lose track of the fact that I “control” this window object, so it doesn’t allow me to close it. This fixes that:

window.open('', '_self', '');
window.close();

Basically it reopens the window on itself, then immediately closes itself. Neat. Haven’t had any problems with it not working in other browsers, but I also haven’t super-widely tested it.

WordPress Authentication Framework: Keyring

Keyring: An authentication framework for your plugins

Quite a while ago (like, in at least 2009), I started thinking about regaining control of all the content I was producing online. I was posting photos to Flickr, saving bookmarks to Delicious. I started Tweeting. I was checking in. All fun and games, and all of those services offer great tools for interacting with them (let’s face it, tools that are much better than WordPress’, because they are focussed on one thing). So I figured, why not write importers for these services and pull my content back over to my WordPress. And keep doing it periodically, so that I could keep using those tools. I want WordPress to be my “home on the web”, my digital hub, but I want to use these neat tools with their fancy apps and what-have-you.

Very quickly, I realized that if I was going to do anything useful on most web services, I’d need to be able to authenticate with them. No biggie, right? I know my username and password… Oh. Right. OAuth. Turns out that most web services use OAuth (or something similar) to authenticate, and it turns out that that’s actually a bit of a bear to implement, when all you want to do is write a simple little Twitter importer. And then again for a Foursquare importer. And a Flickr importer.

What I needed was a shared, generic authentication framework that would do all the heavy lifting for me. I would tell it I wanted a connection to specific service, and if it didn’t have one, it’d walk the user through the process of getting one. It’d give me a standardized format of authentication credentials and abstract out all the complexity of making authenticated requests against those services. Then it would make me a coffee*. What I needed, was Keyring.
(more…)

A Random Idea Around Small-Group Collaboration

I use CloudApp a lot, and recently saw mention of Dropmark. That got me to thinking of a relatively simple, WordPress-powered application for small-group-collaboration which I thought I’d throw out here in case anyone wanted to build it.

  • A WordPress installation, with a group of people who all have accounts
  • A small Mac/Windows app that lived on your desktop/menu-bar/somewhere easy to get at. Let’s call it… Guppy.
  • All users download a copy of Guppy and configure it to point to the same WordPress install.
  • Guppy now sits on their computer, waiting for them to interact with it.
  • When they drop files on this copy of Guppy, it does some magic
    • Images will be uploaded and inserted into a Post, or set as a gallery (if there’s more than one),
    • Text files will be uploaded and rendered as pre-formatted text,
    • Other will be uploaded and attached to a Post,
  • Double-clicking/clicking Guppy would open a small window where they could type in some text (and perhaps also drop files into that window and have them appear as “objects” within their text)
  • Right-clicking Guppy would allow you to change options, jump to the configured website (like you didn’t already have it open in a browser tab!), create a new instance of Guppy, maybe some other stuff.
  • All of this content is published as Posts on the connected WordPress install, and is automatically formatted beautifully (perhaps via an optional theme that’s geared heavily towards collaboration?). Comments are enabled, so you can discuss things on that site, right there while you’re looking at it.

In essence this is just a streamlined, simplified, beautiful WordPress client. There is nothing stopping someone from creating this today. There you go internet, 1 more free idea.