Friday was my last day building Stanford's open-source online education platform. What started as a lark turned out to be one of the most fun and rewarding times in my career. So I'll use this opportunity to reflect back on the project and what we accomplished.

First a Little History

"MOOCs are just information technology happening to higher education." -- George Siemens

I joined Stanford in 2012, coming off of my self-imposed sabbatical. My goal then had been to get hands-on again, to sharpen the technical tools. I did a few online classes and hobby projects, but my friend Jane suggested I could do some good and have more fun working in a group back on the Stanford campus (I'm an alum, after all). So I joined her and a small group of engineers in a conference room in the fourth floor of Gates.

Recall what was happening in education in 2012. Stanford's first three big Massive Open Online Courses (MOOCs) had just made big splashes. Later that year the New York Times would famously declare 2012 to be the Year of the MOOC. What could consumer-grade web tech could do for higher education? Or even do to higher ed? The low cost and ease of cloud computing removed many of the barriers to trying new things out. You approach experimentation completely differently when things get 100x or 1000x cheaper. Profs were literally getting out their credit cards and buying Wordpress blogs or throw up virtual machines for automated grading. Wild stuff.

So this was the environment when I joined Stanford. That first summer we built Class2Go to host free online courses. We went from empty buffers to a live Python/Django site for hundreds of thousands of enrolled students in eleven weeks. Man, that was a fun ride.

We built Class2Go for a few reasons. First, we feel strongly that Stanford needs to control its destiny. There is no "file format" for online ed content, so every course development online is a bet on a platform. And we didn't want to be beholden to one platform vendor (still don't). In 2012 the technology that had powered the early MOOCs were becoming "platforms" and moving off-campus. While we were happy with the success of Udacity, Coursera, and (a year later later) NovoEd, we were also a bit wary. They were going to have to repay their investors at some point. The last thing we wanted was for online education to look like textbook publishing or academic journals. We know what happened there.

Second, we wanted to have a broad impact. We were lucky enough to have an engineering team at Stanford, but it makes no sense for every school to develop their own platforms. This kind of project is tailor made for open source development, and I've advocated strongly for that since the beginning. Not only would this mean many others could benefit form the software, Stanford would benefit from many more developers and see many more use cases (and we have).

And the third reason is the need to modify the platform. There are many reasons why you need to get hands dirty in the code. It could be just developing a point feature, like tracking changes in peer evaluation. Or it could enable a whole new use case, like authenticating on-campus students. Our teachers and researchers want to do interesting work online, not just put up courses. They have great ideas about using the things that make MOOCs unique (different cultures, scale, data-gathering, etc.) as powerful tools, not obstacles to overcome. (To hear more about interesting online ed projects, follow the Stanford VPOL Signal Blog).

Scaling Up With EdX

Come early 2013 we started seeing some interest from others to use Class2Go outside Stanford. While exciting, we were concerned about the quality of the code. There were giant missing features that were going to be hard to write (e.g. peer evaluation). And the code was quick and dirty, with nearly zero tests and other things a "real project" needs. We could have backfilled all of that, but it sounded like a lot of work.

It's around that time that we started talking with edX. Our academic ties to MIT, Harvard, and other edX consortium members are strong. We liked their philosophy and approach. And our technologies were similar: both Python/Django stacks running in Amazon, etc. We decided to work together. Rob Rubin and I got the deal done quickly, mostly between sessions at PyCon. In April 2013 we announced that we'd shutter Class2Go and adopt the Open edX platform.

Stanford didn't join the XConsortium. Rooted in our belief in the power of open source, we made open-sourcing their platform a condition of us working together. EdX didn't need convincing. But Stanford was of a forcing function to do it then, and it did take some work. They pushed the button on June 1 2013.

For the past year and a half the Stanford team has operated as a virtual member of the extended edX engineering team. They've been a fun group to work with, generous with their time and good collaborators. We've been running the platform successfully now for over a year, supporting dozens of courses for Stanford students, MOOCs, online high school students, and many other uses. We've contributed back many features, like theming, course email, instructor analytics to name a few.

I've spent a lot of my own time helping to make sure the Open edX project a healthy open source project. It's not enough to just open up the code, to have a thriving community you have to conduct your development out in the open. Beyond helping other institutions get up and running I've worked to drive the open-source agenda overall. With my friend Nate Aune last May we published recommended changes to technology, governance, and community. And with Paul-Olivier Dehaye this past June we hosted the first Open edX workshop, in Zürich. I think those efforts have made a difference.

Moving On

I've heard the siren's song of the startup. Last Friday August 29th was my last day at Stanford. I'll post about my next gig when the time is right.

But I do feel good about moving on. The Stanford engineering team is solid and productive. The engineers work well with each other and have a good breadth of skills. The course operations team is dedicated and strong. Jane Manning will do great running the team in addition to her day job as of product management -- she's done this before and knows the product inside and out. And Jason Bau, the Stanford Open edX tech lead, will continue to anchor eng and ops.

And what about the open source community? I do feel like the flywheel is starting to spin up. EdX is fostering this in several ways, I'll mention two. First, they are in the process of opening up their bug tracking database and sprint planning (Jira). That's a key step to doing true open development. And second, I'm really excited to see the first full-on Open edX conference happening this upcoming November. I expect that to be well attended.

EdX has asked me to continue on as a member of the Technical Advisory Board, which I am happy and honored to do. I look forward to staying plugged into the project and working with my friends on the board (Armando, Ike, Jim, Phil, Ross...)

Thanks to my many friends on the project: the Open edX engineering team, especially Jason and Jane; the team in the Office of the Vice Provost for Online Learning, especially Professor John Mitchell who made this possible; and my many friends at edX. There's a lot to be proud of over the past two years, and a lot of good work ahead.

Today was launch day. It went really well.  I wanted to capture what a good launch feels like and contrast that with a more exciting launch, just five months ago.

Today we turned on our first class on Stanford's instance the open-source edX platform, what we're calling OpenEdX. The class is Statistics in Medicine, taught by Kristin Sainani of the Stanford School of Medicine. With over thirteen thousand students signed up it's a medium-sized MOOC (Massive Open Online Course).

We have launched MOOC's for Stanford before: two in Fall Quarter, and one in Winter. Even though the classes were huge success, but the launch days weren't so smooth. We had written that platform, Class2Go, from the ground up with a small team in a dozen weeks in Fall; in the weeks before the Winter launch we ripped out the whole evaluation system, about one-third of the code, and replaced it with a whole new engine. In both cases most of our code was fresh off the presses.

Those launches were rocky. I'll tell the story of the DB class launch in January. The first thing we do is a "soft launch," where you open the front door and some people find their way in. Those first visits give you a sense of how things will go.  Surprisingly, the servers were a bit busy.  But we wanted to keep going, so we scaled up capacity and moved on.

The thing that drives real traffic is the announcement email. That gets people to the site. The announcements started going out, students started coming in, and the site lit up. We were in hot water. Servers were overloaded, and most surprising, the database was getting hammered. This was scary and unexpected. We control-C'ed the mail job and quickly hacked additional caching into the site.  We had to trickle out announcements over the next twelve hours.  We made it, but it was a long, stressful day.

And then the days/weeks post-launch were spent watching graphs, triaging 500 errors (user-visible "we're sorry" pages), and installing daily hotfixes. But we got through it. The classes were a success and the team was proud.

So, contrast that to today's launch.  Totally different.

Everyone came in early as usual. I bought bagels. We turned on the class (soft-launch) and the servers hardly noticed. We sent the announcement mails, people came and took their pre-course survey and watched the intro video. Hardly any load. This chart shows the average CPU on our four appservers from 8:00 AM PDT / 15:00 UTC until 10:45 AM or so.

Those are happy servers. Other charts we watched (db connections, load, etc.) told the same story. The most impressive thing was not a single user visible error, no 500's!

Those folks at edX made some solid software.  We're happy to be working with a strong group of engineers and a quality product.  We've had our hands on it only since April, and it was released open-source to the world on June 1.  I fully expect a lot of other universities and organizations are going to have a great time running classes on OpenEdX too.

I just turned off half the appservers since we're fine on capacity. Now off to bed with a good feeling.

I am of two minds about Peter Reinhardt post today calling out online education's poor retention rates.

I agree that retention rates are low. At Stanford we we just completed the Winter run of Jennifer Widom's Database Class, here are the course stats:

• Registered: 64,127
• Turned in some work: 20,836
• Took the final exam: 4,771
• Received a Statement of Accomplishment: 4,854 (1,927 with distinction)

How do I feel that 23% of students who started the class finished? Or that only 7.6% who enrolled finished? Mostly I'm fine with that. I think there's a lot of value in someone kicking the tires. Some were experimenting with on-line education, some didn't know before hand if the class would be right for them. I share Curt Bonk's view that this shouldn't be seen as a black mark but as another form of outreach.

I consider this over forty thousand people who got some exposure to the topic, the platform, and what this MOOC thing is all about.

But I totally agree that increasing retention is a worthwhile goal and a valuable metric.   Online education platforms (mine included) should do what we can to keep students engaged.  This article has many good suggestions.  And there's a lot of room for experimentation.

(disclaimer: I'm the engineering manager on Class2Go, the open-source MOOC platform we've built at Stanford)

I've joined a research project at Stanford University (my alma mater).  I am working with a small team to build a platform for on-line education. This post explains what we're building, my part in it, and why this is an interesting area right now.

Class2Go

We are building Class2Go, an application to put Stanford classes on line.  Envision a video-driven web site with exercises and tests.  It will run much like a class today (professors, TA's, lectures, homework, tests, schedules) but with everything happening on-line.  For some kinds of classes this could improve the classroom experience for enrolled students; the excitement comes when you can bring in the a much larger set of non-enrolled students.

The professors who have expressed interest so far are those who want to flip their classrooms or host a Massive Open On-Line Course (MOOC). Others have ideas on experimentation with the learning process itself. For example, do students learn better when they see slides, a talking head, or both? A large enough student population makes meaningful studies possible and cheap.

There are a bunch of features needed over and above basic basic video.  First, can we devise a good system for students to grade each others work? This is an absolute must to do anything in the humanities at large scale.  We have some ideas of how to do peer evaluation, but it's an area that needs much more experimentation. A second important feature is enabling off-line use from laptops and tablets.  When we think of disconnected students we typically think of commuters, but we've heard it's just as important (or maybe more so) for students in developing countries where bandwidth is precious.

While there are commercial offerings to do this, and even things from other established universities, Class2Go will be different in three important ways:

• Research will be first and foremost. Hey prof, you want to do a wacky experiment? We can help.
• Value produced by the course is retained and controlled by the professor and the university.  That value comes from the assets (video, homework) but even more so the community and technology.
• The professors will have quick and ready access to their data. No waiting in lines for reports.

Stanford Professor Rob Reich made a very coherent argument for this in a recent blog post. (Getting to work with smart people like him is a major job perq, by the way.)

Part of what makes this doable is that there are so many great building blocks available.  We plan to use YouTube for video and the Khan Academy framework for exercises.   Piazza has great discussions and forums to start with. Because of all this good stuff, we feel good about getting something strong out by the Fall for a couple of courses, and then expanding from there.

And me?  I'm the line manager. It's fun to work on a project of this size where I can keep my hands dirty. For example, this week I'll be working on authentication (oauth).

Why Education, Why Now?

Two things brought me to education.

Firstly, education is uniquely important. I challenge you to name another human pursuit that is as important, consumes so many resources (for good reason), and can be so transformative when done well.  The other two that come to mind are health and agriculture. But I'd say education is right up there.

And second, education is changing right now. Of the meetings I've had while on sabbatical  some of the most interesting have been about education. Why can't information technology have as big an impact on how people learn as it has had on everything else? Technology could mean making it available to to many more students, in different places and of different means. Educators could spend less time lecturing (dry stuff, to be sure) and more time working with students. Better measurement and analysis and could give educators near-real-time feedback and students tailor-made homework. Those are just a few of the examples that I heard about that got me excited.

I look forward to doing something that will make even a little difference here. I'll blog about my experiences along the way.

PS - Peter Norvig's TED Talk captured the reasons and potential nicely.  That seven minutes is worth your time.