This week I used Claude as my pipeline engineer and data analyst. I had a ton of fun and figured I'd write about it.

I've been taking a Data Visualization class at Stanford Continuing Studies (TECH 26, Winter 2026). I needed a final project. There's this data set I've always wanted to play with, TweetsKB. Researchers took the Twitter Firehose data from 2013-23, back when it was more available, and ran it through entity extraction and sentiment analysis. They made the dataset available for others to work with. It's well documented and reasonable size, about 500GB. I figured there'd be something interesting in there.

The cool part didn't turn out to be the analysis itself. I did find and graph some trends, like Wordle hitting the scene in Feb 2022 and then falling off; contrast that with K-Pop, which we all know has been much more durable. You can see those in the presentation I did for the class, and I included some charts below.

But the main thing is this was just plain fun.

What I (We) Did

What I spent the week doing was chatting with Claude interspersed with running pipelines, bringing up and down jobs, etc. What it felt like though was having my own junior engineer working hard for me, doing whatever I asked, and doing it well and without complaint.

This section is has more of the details of how it went. Consider skipping if you don't want to get in the weeds.

We started with ETL pipelines. File format conversion, multiprocessing, scaling to fit RAM and processor limitations, progress bars and interruption/restart logic.

Yikes, this is full of offensive stuff! I asked for redaction with stable terms and it found and used a nice off-the-shelf library.

When I didn't trust what I was seeing, I asked for end-to-end data quality tests. Claude wrote them and then found and fixed a double counting bug. Nice!

When I wanted to wander around in the data I asked for an EDA dashboard and got one. EDA is the term of art I learned in class, "exploratory data analytics". The idea is something where you can click around, slice and scrub, looking for interesting things like correlations and trends.

You can see this in the screenshot at the top of this post. Essentially I got the "Overview" and "Slice by Entity" tabs in one shot. How you do EDA with it? For example, the default view are two base ball teams. See the peaks when the Red Sox won championships in 2013 and 2018? But the Astros peak was higher in 2017, presumably because of the scandal. Fun, right?

I then iterated on that dashboard quite a bit. Instead of a normal notebook or colab-style workflow I was used to, instead I iterated around the dashboard. I'd ask Claude add or update charts in the dashboard ("annualize the last data point"), it'd reload, test and repeat. I did fall back to python or a notebook when I had to look at the raw data, but just in throwaway mode. I found this to be a really fast way to work.

But I still had a hard time finding someting interesting in the dataset. So I just asked an open-ended question, basically "find interesting stuff" and it produced five analysis scripts. Most of what's in the "Analysis" came from this one query, except for "Democrats vs. Republicans", which I did on my own and didn't prove too insightful.

And yes, when I had ops issues Claude fixed those for me too, see screenshot below. Sure, I could tune workers vs. threads and manage PID files, but who wants to? And yes, when it came time to write the presentation, Claude wrote that for me too. Well, most of it.

You can play with the dashboard yourself. It's hosted on an underpowered server at my home, so if it doesn't work, let me know or try again later. All the code is in GitHub and there are known issues.

At some point I felt it was helpful to define "personas" for Claude to better describe the goals of each prompt. Then I started a prompt with "hey dashboard engineer, I'd like to...". Tough to tell how much that helped

Some Thoughts

Let's not give AI too much credit. Pulling entity mentions out of tweets is something that data scientists and journalists have been doing for years. And this particular dataset probably has been around for a while and might very well have been used and written about, by undergrads. The story here might very well not be "gee whiz, looks at how smart AI is" and more "look at how AI has scooped up and repackaged years of everyone else's work."

The speed of accomplishing these dev/debug tasks was remarkable. I felt like I had a coworker. And at remarkably low cost, too. This was all using the Claude $20/month plan. This past week was the time I poked above the token cap. I happily paid $5 more to keep on going.

Is this programming? I think so, at least it felt that way when I was doing it. I was very much using the same parts of my brain as when I'm writing all the code myself. Indeed for years most of us have been "progamming" mostly via Google and Stack Overflow. It's like that but fast.

Charts

Here are three charts that fell out. Again, don't focus so much on the insights but how these came from a high-level, open ended prompt.

Some Screenshots

Maybe some people would like to see what this looked like when I was doing it. Here's a screenshot during pipeline development. On the left is Claude adding some logging; the top right is tail -f on a log file, and the bottom right shows the progress bars churning through the data.

My modest Mac has 8 performance cores, hence 8 workers.

And this screenshot is kind of fun. Look at how Claude explained a worker crash issue that I asked it to debug: "You already flagged it, the UI literally says..." 😀

Claude made the adjustment and saved its suggestions in a Github Issue to look if I want to look at it again later.

It's been nice having AI tools at the ready for cleanup work. You know, those tasks that require a little program or script. Sure you can write it yourself, but it won't be fun, and it'll take some time, so instead you just don't bother and it doesn't get done.

Instead of just knocking this out I decided this would be a fun one to try with the two tools I'm using the most these days, Gemini and Claude. I can't really call this a proper bakeoff, since it's just those two and it's not an especially hard task.

And the result? Both did well but Gemini did a little better. Gemini got it right on the first try; Claude had two bugs that were easy to find and fix. Claude was a little nicer to work with and produced a nicer description of the solution. Aside from that, the end product was identical.

The Task

I wanted to clean up the some of the old files backing this blog. I migrated the site from Wordpress to Nikola in 2013. Even back then I complained about the crufty file format, but couldn't be bothered to fix it then.

The prompt describes in a fair bit of detail what I wanted done. Probably too much detail. It's down at the bottom of this post.

The best way to see what they did is to just look at the resulting commits: Gemini and Claude. They're pretty similar. I asked both to save their work in a migration_scripts subdirectory, including a it's own summary of the work and full log.

For both tools I pay the $20/month that gives me access to reasonable token limits and good production models -- Gemini 3 and Sonnet 4.6.

Thoughts on Claude

I didn't describe the problem as "Wordpress-style naming" but Claude sussed that out. Honestly I'd forgotten that was the source of the problem. Nice!

I like that Claude inserted comments in the configuration file without being asked to.

Despite being told to test, there were two bugs that I had to find.

• Some of redirects broken. I pointed the problem out and it did more thorough testing using curl and found the problem. The new format was what Gemini came up with in the first pass (luck? smarts?).

• A missing newline between the metadata and the body caused the hero images to be dropped.

Thoughts on Gemini

I haven't used the Gemini CLI as much by this point. It seems to have borrowed much of it's UI and flow from Claude code (slash commands, asking questions as it goes) so it's familiar. It's really nice to use, just a smidgen less mature and polished than Gemini.

I preferred Claude's writeup a little so I went with that one. But Gemini wins the prize for nailing the task on the first try!

Usage

I really liked how Claude shows how much of it's context window is used via the /context command. I had a harder time getting this kind of thing out of Gemini, and even when I did it was harder to grok.

I couldn't tell with either if I'm getting close to any global usage limits. I should hope not, this wasn't a very big job.

Claude /context Output

Gemini /stats model Output

Prompt

Both started with this:

First read DEV.md for context about this web site.

In the posts subdirectory are many files that follow an old 
naming convention.  Please convert them all to the new file
naming convention.

The old convention has files of the format YYYYMM<slug>.SUFFIX,
where YYYY is the year, MM is the month, and SUFFIX is html or
meta. <slug> is the short form of the the title, and should match
the "slug" field in the metadata

To convert to the new convention

1. Drop the YYYYMM file prefix

2. Combine the "html" and "meta" files into one markdown file
   with a "md" suffix. The contents of "meta" are at the top of
   the file as an HTML comment

3. Add redirects so the old names can still be served. Add tuples
   in the REDIRECTIONS of conf.py

Use the `nikola build` and `nikola serve` commands to test your
work. Make sure all old URL's still function. Make sure pages
look the same. Make sure tags still work.

Do not commit changes. Do not push to production.

I also checked in a DEV.md file that described the purpose, directory layout, and some constraints. Again, maybe overkill, but I figured this isn't the last time I'll be asking one of these tools to help me futz with this site.

This is a story about my recent experience Vibe Coding. The work itself isn't impressive and this writeup isn't different than the many gee-whiz posts you see these days. I didn't do three apps in a day. But I wanted to write up my experience, mostly to give me an excuse to tell an old tyme programming story from the 1900's.

Today, 2026

While I'm on a break I'm taking a Vibe Coding class. It's a fun excuse to play with some new toys, and it's well taught, and I like doing things with my friend Jane.

One interesting tidbit: the first day of class, February 3, 2026, was one year and one day after "vibe coding" itself was coined via tweet. That name seems to have stuck, for the time being at least. On last week's ATP they said that by this time next year this will probably just be called "coding" and I bet they're right.

Anyway our week one assignment was to code up a game. In about two hours and $10 I had a something up and running. I spent another couple of hours futzing with version control, documentation, and hosting. But that's it!

It's pretty basic, and not all that much fun, but you can play it here. It's hosted on Github pages, just like this blog. The code and construction notes are in checked in.

This was my first experience with Replit. It's impressive and fun. This was what was recommended for the class and the good folks at Replit were nice enough to give us all $30 in credits, which I had plenty to spare. Most of my comrades presented apps that were fancier than mind with 3d graphics, sound, and more interactive game play. some also said though that they ended up spending much more than I did, so YMMV on costs.

One interesting part was dealing with integration. To get their code deployed onto my personal website site required wiring up a GitHub workflow, which I'd never done before. No problem, Replit took care of that too. Then I asked Gemini to get local hosting running. When I hit a permissions problem and a crash I had to resist the urge to copy paste the error messages into a search boxes and Stack Overflow, like I've done for years. Instead I asked Gemini to debug and sort this out for itself and it did straight away. Pretty great.

Forty Years Ago, 1985

Why'd I pick this weird old chase game? Well, that's the more fun and nostalgic part of the story.

When I was fourteen years old, I spent the summer hand-coding a video game. I'd gone to a family gathering and my older cousin Erik brought his Mac from college. It was the first I'd seen a Mac and fell in love. I thought the Daleks game he had running on it was so cool. The screenshot on the right is from that Classic Mac website of that game that I found online, and is exactly how I remember it looked.

Upon return to Fresno I got to work. I spent most of the rest of that summer writing a clone of Daleks on my Apple //e. All hand-coded 6502 opcodes and twos complement math for branch offsets by hand (I didn't have an assembler), in pencil on graph paper. The hardest part was getting smooth animation working, since the Apple //e "hi res" graphics system was super quirky.

It took about two months to get it working. It's the first time I can remember being in a flow state and I loved it. It was my first "real program" and began my lifelong love of computers.