Today, a more technical note on the evaluation of agentic harnesses, drawing from the process of development for the Work Ledger, my harness-neutral agentic context management file system.

You ask a coding agent to update an API endpoint. The repository already contains a decision about backward compatibility: keep the existing response fields. You want the implementation, a contract test and a short migration note. Leave the commit to you.

The final diff looks good. The test passes. The migration note is in the right place.

But you also pointed the agent to the current decision after it started from an obsolete one, rejected a breaking field rename, approved the test command and reminded it to save the migration note.

This is an invented example, but it contains a real evaluation problem. If you score only the final diff, several different things disappear into “success”: repairs to delegated work, a permission decision, and a reminder to save the result.

You need to distinguish them before you can decide what to fix—or what to delegate next.

The method I use is to keep the evidence about the agent's actions separate from the evidence about the work it left behind, with my own contribution visible alongside both.

Agent-evaluation guidance already distinguishes a transcript from an outcome in the environment; Anthropic's guide explains that distinction well. Here I want to work through a particular complication: a person keeps changing the conditions and repairing the work (or, more generally, intervening) while the agent runs. What should the resulting record let us conclude? This is an especially important question if we are to avoid agent hand-holding that silently diminishes the productivity gains they are supposed to bring.

(Relevant context on how the resultant system, the Work Ledger, achieves the productivity gains in prior posts here, here, and here.)

What did the Person Have to Do?

An experienced engineer can make a shaky workflow look quite good. You catch the old decision, recognize the breaking change and ask the question that gets the agent unstuck. Because those interventions come naturally to you, you may barely count them as work.

Counting interruptions is a start, but it doesn't explain the engineering problem. An approval you deliberately reserved for a person is different from a correction to something the agent was supposed to handle.

I find these categories useful when reviewing a run. They are labels for diagnosis, not a universal scoring system:

A single intervention can serve more than one role. Record what happened before choosing the labels. The point is to avoid treating every human action as an autonomy failure, or treating every rescue as ordinary review.

Follow One Run All the Way Through

Here's a more explicit version of the endpoint example. The files, commands and events below are synthetic. I haven't run this example or measured a model with it.

The task is to add an optional display_name field to a profile response while retaining name. Before editing, the agent must consult docs/adr/014-profile-response.md, which records that compatibility decision. It must update the contract test, save docs/profile-migration.md, and leave Git commits to the engineer.

Before starting, the engineer identifies three kinds of evidence: the activity record for the required sequence, the permission history for the test approval, and the final files and test output for the deliverable. The engineer also expects to approve the first test invocation in this environment.

Suppose the visible record contains these events:

01 agent: read docs/archive/profile-response-v0.md
02 agent: edit src/profile.py; replace name with display_name
03 human: point to ADR 014; instruct agent to keep name
04 agent: read docs/adr/014-profile-response.md
05 agent: edit src/profile.py; retain name and add display_name
06 agent: update tests/test_profile_contract.py
07 app:   request permission for pytest tests/test_profile_contract.py
08 human: approve that command once, as planned
09 tool:  test command exits 0
10 agent: describe migration note in chat; report task complete
11 human: request the missing docs/profile-migration.md file
12 agent: write docs/profile-migration.md

For this illustration, the engineer also preserves the diff before intervening at step 3, checks that the note is absent at step 10, and inspects the final files after step 12. The first attempt ends at the first corrective intervention; the rest is a recorded continuation. The final result can be perfectly usable. The record still supports several different judgments:

The first two rows may share a cause. This trace doesn't establish whether the agent missed an instruction, retrieved the wrong context, or understood the decision and still edited incorrectly. It gives you a smaller place to investigate.

Nor do failed requirements, interventions and causes have to produce the same count. Here one correction addresses both the source and the field rename, and another addresses the missing note. Keep the actual record so those distinctions remain recoverable.

The useful conclusion is that the task was completed with corrective help and one planned permission approval. This run does not show that the workflow can satisfy those delegated requirements without that help. It also doesn't show that the permission approval was a defect.

Three Evidence Sources, Three Different Jobs

The starting materials tell you which task you tested. Keep the request, relevant files and starting revision. Git may omit local settings, so save the configuration that matters to the question as well. If you change the prompt, configuration or supplied context during recovery, record the change. The repaired run answers a different question from the unassisted first attempt.

The activity record tells you what you can observe about execution. Tool calls, visible approvals, errors and human messages can establish particular events and their order. They don't give you an all-seeing account of the runtime. In the synthetic trace, the late recorded read is visible. To claim that the agent had never received ADR 014 through any other route, you'd need to account for the starting context and other ways the app supplies material.

An absent read event is especially easy to overinterpret. The file might already have been in context, the event record might be incomplete, or the read might really have been skipped. If you can't distinguish those possibilities, retain the uncertainty.

The final files tell you what survived. Inspect the diff, deliverables and any saved revision. Equal before-and-after bytes don't prove that nothing happened between the snapshots. A file could have been changed and restored. Conversely, an agent can describe a correct result without writing it anywhere the next task will find it.

This matters for review, too. If a reviewer approves a particular diff and the agent then changes it, the earlier approval applies to the earlier candidate. Record a revision or a hash of the reviewed files so you can tell whether you're still looking at the same work. The hash identifies bytes; it doesn't certify their quality or explain how they were produced.

A Record You Can Use

For a consequential check, I would start with something this small (n.b. the fields are an example template; tweak as needed):

question: "Can this workflow add display_name while preserving name?"
starting_revision: "<revision>"
request_and_context: "<saved request, instructions and supplied decisions>"
relevant_local_settings: "<saved files or hashes; unknowns listed>"
environment: "<app, engine/model, mode; unknown where unavailable>"
requirements:
  - "consult the current ADR before editing"
  - "preserve name; test compatibility; save the migration note; no commit"
planned_human_steps: ["approve first test invocation", "review final diff"]
delegated_requirements: ["choose current ADR", "preserve name", "save note"]
first_attempt_boundary: "first completion, stop, or corrective intervention"
event_record: "<tool events, human messages, approvals; coverage gaps>"
interventions: "<event, purpose, expected or corrective, time if useful>"
final_candidate: "<diff/files and revision or content hashes>"
verdicts: "<procedure; permissions; saved result; remaining unknowns>"

Choose both the boundary and the division of responsibility before the run. Planned approvals can belong to that attempt; help repairing a predeclared delegated requirement starts a continuation. Classify the intervention against that earlier allocation. Otherwise you can quietly redefine a rescue as expected assistance. Preserve the first state when practical, then record the continuation separately. If you must intervene immediately to prevent harm, do so and record the stop and any missing snapshot.

For a small task, file copies and a short event note may be enough. If you are comparing repeated runs, preserve a fuller input inventory and use the same boundary each time. A time measure is useful only if you say what it includes: waiting for the tool, reading a permission dialog, reviewing the diff or repairing the result.

Before treating a failed requirement as an agent failure, check the test itself. Was the ADR actually supplied or retrievable? Does the contract test exercise the old response field, or does it merely pass because the new implementation and new test share the same mistaken assumption? A precise record of an invalid test is still an invalid test.

One run can reveal a failure path worth repairing. It doesn't estimate a reliability rate. And if you revise the workflow repeatedly, the eventual success shows what the revised arrangement achieved; it doesn't isolate which change caused the improvement.

Permission Prompts’ Opacity

I encountered a related problem while investigating Claude's Windows desktop app. In a September 4 report, I recorded a plain PowerShell command running silently under an allow rule, while selected expressions still prompted despite their own explicit rules.

The observation justified asking how to diagnose the prompt. It didn't establish which internal check caused it. A rule mismatch, an additional permission check and an ineffective configuration source would call for different explanations. The visible dialog alone couldn't choose among them.

Controls help narrow that uncertainty. Compare the same plain command with and without its allow rule; silent execution alone could have another explanation. An ask or deny control checks another expected outcome. Returning to the initial configuration tests whether a change reverses. These controls narrow possibilities without proving the internal mechanism. Changing both the command's syntax and the kind of expression it evaluates changes more than one possible explanation at once.

This is why I would record the exact command and permission outcome before summarizing the behavior as “allow rules don't work.” It is also why broadening permissions until the interruptions disappear would answer a different question.

How this Shaped the Ledger

Work Ledger gives the next task a route to the current API decision, its source, the record of the attempted change and the resulting files. Those roles correspond to current/, sources/, dossiers/ and artifacts/. Git records saved revisions. For selected consequential conclusions, TRACER records the support, limits and reasons to revisit them.

I developed the Ledger through work on my own projects, beta feedback, and revisions to setup, source handling, saving and recovery. Turning my established workspace into an empty starter exposed assumptions that my own accumulated context had concealed. The starter asks users how they want to work, separates preserving a source from accepting an interpretation of it, and gives current decisions a shorter route than searching the full history. Recovery starts by inspecting saved and unfinished work.

PROTOCOL.md describes the procedure, with AGENTS.md and CLAUDE.md as entry points for the supported assistants. The app still controls its permissions. V1 uses plain-text instructions and configuration with no required Ledger executable.

The starter's setup guide and protocol explain the implementation. You can use the record above independently. The important test is whether it helps you distinguish a workflow you can repeat from a result you managed to rescue.

Graham

Let's transform learning together. Feel free to connect on LinkedIn! (And press reply to contact me: all replies go directly to my inbox.)

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I close this three-part series by handing you the Work Ledger starter and walking you through installing it, using it, and judging it.

Hi—

My August 28 post argued that the software around an AI model — the harness — matters as much as the model (Agentic Harnesses Need a Proper Workspace). My August 31 post described my answer to this problem (the “Work Ledger”) and the lab-notebook discipline behind it (Teach Your AI to Keep a Lab Notebook). I also described some alternatives and their pros and cons. I ended that second post with a promise: a starter version of the Ledger you could put on a real project of your own, plus a guide to get it running.

Here they both are. I'm not going to re-argue either post today. What follows is the practical part: whether this is worth your time, and if it is, exactly what to do.

📦 What You Get Today

Work Ledger v1 is a free starter: a folder of plain text files and instructions that you open as a new project in an AI app you already use. You can download it from the release page, with the self-contained ZIP named work-ledger-v1.zip. The maintained step-by-step guide is START-HERE.md, but below I’ll walk you through the setup.

It’s been tested extensively in two agentic harnesses: Anthropic’s Claude Code and OpenAI’s Codex. You can use the former through Claude Desktop's Code tab in Local mode and the latter through the ChatGPT desktop app in Local mode. ("Local" just means the app works directly with files on your own computer.) It also works in the CLI versions of those apps, if that’s your style.

In terms of operating systems, it works on either Windows or Mac, and it is designed to be portable across those surfaces, with parallel functionality in any permutation of them.

I am not finished making it compatible with Claude Cowork. That comes later this year, along with confirmed compatibility with a range of open-source harnesses (I believe several are already compatible but further testing is needed).

It also isn’t compatible with the browser version of either assistant. If that is how you use AI day to day, this release is not for you.

I’d recommend keeping a backup of the ledger folder, the way you would for anything you would be annoyed to lose. (Git also keeps a history of changes inside that folder, which I’ll explain below. You can push your Git updates to a private online storage location as a form of backup.)

🧭 Is This Worth Trying?

This is for people with projects that keep going across many sessions: a course redesign that spans a semester, a grant revised over months, an accreditation cycle, a committee rewriting a policy, a research paper. Work where resuming from the wrong assumption costs you something — time, a bad draft, a meeting spent defending a decision that was already reversed.

If you mostly ask an assistant to tighten an email or suggest five discussion questions for tomorrow, you do not need any of this.

I’ve used it extensively for months, along with some beta users. It has performed extremely well for me across ~30 billion tokens and 5 heavy workstreams (including two with a lot of research and coding). My productivity is 5x what it was before I ended in roughly the mature Ledger state I’m sharing with you. Beta users report similarly transformative outcomes for heavy and long-running workloads.

But this is the first public release, and I expect to keep improving it as people put it to work on their own projects. So if something looks different from what I describe below, that is useful information for me, not proof that you did something wrong.

One more honest note. The first hour is more technical than every hour after it. You will open a terminal, confirm that a program called Git is installed, paste a few commands, and send one exact first message. After that, the assistant handles the mechanics and you go back to asking for things in ordinary language. Allow some uninterrupted time for setup, especially if you still need to install Git.

🧰 Before You Open Anything

Windows: install Git for Windows, which includes Git Bash. (Feel free to ask your AI assistant what Git is, or read this, if you aren’t familiar with it.) If you use Codex on Windows, you also need PowerShell 7.5 or later, which Microsoft distributes free; the PowerShell that ships with Windows is older and will not do.

Mac: macOS 14 or later. Open Terminal and type git --version. If Git is not there, macOS may offer to install Apple's command-line developer tools; let it, then try again.

After installing a prerequisite, restart your AI app before continuing.

🪜 The Walkthrough

Follow these with this newsletter open beside the app. Where Windows and Mac, or Claude and Codex, differ, the step says so; otherwise it is the same for everyone. If you get stumped, share this newsletter with your favorite AI assistant and ask for help about the relevant step, or just email me (respond to this newsletter in your inbox).

1. Download and extract. 

On the release page, under "Assets," download work-ledger-v1.zip. Skip the "Source code" links GitHub adds automatically; they are not the starter. Extract the ZIP into a new folder and keep everything inside it together. The folder you will open in your app is the one that directly contains a file called START-HERE.md — not the folder around it. After extracting you may see one folder sitting inside another; that is normal. Keep opening folders until you find the one with START-HERE.md in it.

2. Open a terminal in that folder. 

Windows: right-click an empty area inside the folder, choose "Open in Terminal," and make sure the tab says PowerShell.

Mac: open Terminal, type cd followed by a space, drag the folder in from Finder, and press Return.

On either system, then run:

git --version

Continue when it prints a version number. If it does not, stop there and email me rather than pushing forward.

3. Claude only: activate the folder's permission profile, once, with Claude closed. 

This copies one settings file into place. It tells Claude which routine file and Git commands are already allowed in this folder, so ordinary work does not stop to ask you every few minutes. Codex does not need this step; skip to step 4.

Windows (PowerShell), one line:

if (Test-Path -LiteralPath ".claude/settings.local.json") { throw "Stop: .claude/settings.local.json already exists; do not overwrite it." } else { Copy-Item -LiteralPath ".claude/policies/desktop-windows-standard.settings-local.example.json" -Destination ".claude/settings.local.json" }

Mac (Terminal), the whole block:

if [ -e "./.claude/settings.local.json" ] || [ -L "./.claude/settings.local.json" ]; then
  printf '%s\n' 'Stop: the local settings file already exists; leave it unchanged and ask for help.' >&2
else
  /bin/cp -n "./.claude/policies/desktop-macos-standard.settings-local.example.json" "./.claude/settings.local.json"
fi

Success normally prints no message. Never run the Windows version on a Mac or the Mac version on Windows.

4. Both apps: put your name on your local saves. 

In the same terminal, run these two lines, typing your real name and email inside the quotation marks in place of the examples:

git config --local user.name "Your Name"
git config --local user.email "you@example.com"

This applies only to this folder. It labels the saves the Ledger makes on your computer; it is not your AI account, and nothing is uploaded.

5. Open the folder in your app. 

Choose the exact folder from step 1 as a Local workspace — the folder itself, not its parent — and work in it directly, with worktree isolation off (a setting that keeps you working in this one folder rather than a hidden copy of it). Claude: select "Accept edits" (not Auto, not bypass). Codex: use normal editing inside this workspace. You may see one prompt asking whether you trust this new folder. That one is expected. What is not expected is a stream of permission dialogs after it. If a permission request appears that you do not understand, choose Deny and email me; do not click "Always allow" to make it go away.

6. Send the exact first message. 

Copy the message for your app, and where it says [Windows or macOS], type the one word that is true for you, without the brackets.

Claude:

Codex:

The first thing you see back should be a card with five choices. The "card" is just the assistant's reply in the chat, not a separate screen. It should arrive before the assistant does anything; if it starts acting before you have answered, stop and email me.

7. Answer the card. 

In plain terms, the card asks five questions, each with two or three choices shown right on it, and "Recommended" takes the first choice on each:

• What kinds of files will you add? (Recommended: work files already approved for this AI workspace. The alternative is a mix of work, personal, and client files, which it asks about group by group.)

• What should become reusable understanding? (Recommended: stable goals, decisions, constraints, and open questions.)

• How detailed should the source notes be? (Recommended: balanced.)

• How should it talk to you? (Recommended: brief, handling the mechanics for you.)

• How should it save? (Recommended: automatically, as ordinary local Git saves in this folder that upload nothing. The alternatives are to be asked each time, or to manage Git yourself.)

To take the recommended choices, reply:

using a short, non-secret label you recognize for your account or organization, and the same name and email from step 4 — never a password or key. Correct any line the card shows that is wrong, and fill in any line marked "not yet supplied." Once setup is complete, the assistant confirms its first local save.

8. Add your first files. 

Put them directly into the folder called add-files-here — for a first trial, something small and not confidential, with your originals kept elsewhere. Keep filenames simple — ordinary unaccented letters and numbers, spaces, periods, underscores, parentheses, and hyphens; start with a letter or number; no double periods and nothing ending in a period or a space; no longer than 120 characters. If the assistant flags a reserved or unsupported name, rename the file and try again. Then say:

The assistant stores a copy of each eligible file with a note of where it came from, shows you a short review of the understanding it extracted, and asks once what should become reusable. The review offers three choices: A saves the understanding as shown; B saves it with corrections you provide in the same reply; C keeps the files as sources only and saves no reusable understanding. It saves and ends with a readiness summary.

9. Work normally, one conversation at a time. 

Ask for real tasks in your own words. At the start of a conversation, list everything you already know the task needs; add related work to that same conversation while it is unfinished; start the next conversation after the work is completed and saved. Keep one active writing conversation per ledger folder. Two conversations changing the same folder at once can leave it half-written, and that is a bad way to learn what "half-written" means. You can always reopen an old conversation to read it.

Saving finished everyday tasks has its own one-time opt-in, separate from the card. When the assistant offers it, reply SAVE COMPLETED TASK FILES AUTOMATICALLY if you want that; otherwise tell it to leave the work unsaved as a checkpoint. The draft files remain in the folder, but they are not recorded in the Ledger's saved history.

10. Three phrases for later. 

"Update this workspace from the files I added."

"Show what this workspace currently understands."

And "Run the garden," which is designed to trigger a review of whether your Ledger contains anything stale, unresolved, or contradictory — it shows you what it has recorded first and asks before changing anything. I run it twice a week.

Once you are set up, though, I’d recommend asking your harness directly if you have any questions about what you can do. It will review the Ledger and answer accurately. For instance, if you want to setup a connector to your email account, just ask it how and it will explain how it could handle data in such a case, allowing you to specify your preferences.

🧪 The Test That Matters

Do not judge the Ledger by opening the folder and admiring the structure. The folders are arranged with care, but so is my spice rack, and neither fact tells you whether anything gets cooked.

Give it one real task on a real project. Ingest the relevant context in the form of old files. Say you are six weeks into a course redesign: the faculty group adopted a shared rubric, the oral-assessment question is still open, and you ask it to revise the assessment instructions to match the rubric without settling the open question. Then leave the project alone for several days. Come back in a fresh conversation and hand it a related second task — the syllabus-facing summary of the new plan, say — without re-pasting the background.

Then ask honestly. Did it resume from where things actually stood, or from a plan someone later dropped? Did it keep the open question open instead of quietly resolving it for you? Could it tell you why the rubric looks the way it does, and point to the record, rather than just asserting it? How much did you have to re-explain? And on balance, did it save more effort than the setup cost?

Let me know your answers!

🏁 Closing the Series

Three posts, in order. Friday: the environment around the model matters as much as the model. Monday: serious projects need something closer to a lab notebook than a chat transcript. Today: a notebook you can open and use.

This was a deliberately finite series, and it ends here. I'm not promising the return of a weekly cadence — I said at the start that I wouldn't, and I'm keeping to it. I may write other things later. What I want from you is simpler: try the Ledger once on something that matters to you, and tell me whether it earned its keep. I think it will, but you be the judge.

For installation or everyday-use help, respond to this email and it will come to my inbox. For a reproducible bug, open a GitHub issue and include the release filename, your operating system and app, the step, and what happened, with private material removed. The guide at https://github.com/automatedteach/work-ledger-starter will stay current.

EDIT: The technical companion, where I discuss how I evaluated the Ledger’s impact on agents’ behavior/processes, is now available here.

P.S. I have created a new tool, the Schedule Workbench, for the unglamorous problem of assigning faculty to courses, which most departments still do through spreadsheets, email, and one person's memory. It organizes the agreed inputs, proposes assignments a human scheduler can edit, lock, use, or leave alone, and keeps a history of what changed. You can watch a video walkthrough in ~6 minutes here: scheduleworkbench.com. And if your department schedules some other way, I would like to hear how, as I am expanding its functionality every week.

Graham

Let's transform learning together. Feel free to connect on LinkedIn! (And press reply to contact me: all replies go directly to my inbox.)

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I follow up on Friday’s post to explain the architecture of the memory solution I’ve developed for agentic harnesses and compare it with some alternatives available to you. I hope this piece helps you navigate your options.

Hi—

Friday’s newsletter argued that an AI model matters enormously, but that the harness around it—the software environment that decides what context it gets, what it’s allowed to touch, and what happens to its work when the session ends—matters too. I closed by naming the sharpest version of that problem: better memory isn’t the same thing as a trustworthy record of a project. If you missed it: “Agentic Harnesses Need a Proper Workspace.”

That shift is already visible at enterprise scale. Visa, for example, has open-sourced a multi-model vulnerability harness that maps its codebase, routes agents through a structured workflow, supports remediation and validation, and keeps human judgment at key gates. At Kognity, when I left, we had been building and testing a range of custom harnesses for content production.

I’ve written before about what a reasoning model can offer as a thought partner for research. The question I left hanging in that piece was how you give a thought partner like that a trustworthy, current brief without re-explaining everything each time you sit back down with it. This post explains my answer: the Work Ledger, which I’ve now used on my own consequential projects for months.

🧵 A Chat has an Ending. A Project Doesn’t.

Return to my example from Friday about a six-month course redesign. It starts with student feedback, moves through a new set of learning objectives, picks up an assessment plan, loses that plan when a curriculum committee objects to it, and settles on something else for reasons that mattered enough to change course. Somewhere along the way, someone finds a source worth keeping, someone else raises a constraint nobody had considered, and one promising idea sits unresolved because the evidence for it isn’t in yet.

Swap in a grant proposal, a research paper, an accreditation cycle, or a departmental policy revision, and the shape repeats. A real project accumulates decisions that currently govern, options that were tried and dropped, sources that carry different weight, and questions that remain open. None of that survives well as governing project state in a chat window.

A full chat transcript that you return to repeatedly keeps everything, which makes it a useful archive and a terrible briefing: the plan you abandoned, the mistake you corrected, and the eleven messages you spent realizing you and the AI meant different things by the same word sit at the same weight as the decision that actually matters. Reading it back to reconstruct what you settled on, and then injecting new context that happened outside of the chat (with relationships to prior states explained), is often more work than starting again from scratch.

Your AI’s own memory solves a related but different problem. It’s good at not making you repeat yourself. It is not designed to serve as the canonical record of whether what it remembers still governs the project. An AI that perfectly recalls a plan your committee killed three weeks ago hasn’t remembered your project. It’s confidently wrong in a new way.

Anthropic’s own documentation for its experimental Dreams system names duplicates, contradictions, and stale entries as predictable consequences of memory accumulated across sessions. Its answer is a separate consolidation pass. That is useful. But it is still a different job from governing what may become current before later work relies on it.

The obvious low-lift fix is one big context document per project you maintain and paste in at the start of every session. For some work, that’s enough, especially if it evolves slowly, is repetitive in some regard, and the parts of the document are easy to assemble. If that sounds good to you or fits some of your tasks, I’d rather leave you with it than talk you into more machinery than you need.

But someone has to decide, every time the document grows, what belongs in it and what’s gone stale, and that job doesn’t get any easier just because you keep doing it by feel. It takes a lot of manual effort, especially as projects gain complexity or you have many workstreams running at once. This is the across-time version of what I once called integration effort: supplying the context once is not the same as keeping it current.

🔬 Borrow a Discipline that Already Exists

I trained and taught as a philosopher before working on AI, and the habit that survived the career change wasn’t loyalty to any particular argument (though I am still working on my Humean system on back burner 6). It was a way of handling a claim: state it precisely, say what it rests on, go looking for the strongest reason it might be wrong, and keep straight what you’ve actually established versus what you’re still assuming.

Philosophers carry out this process in a few specialized ways, but really it’s a general pattern. Researchers in a range of fields use versions of the same discipline, often with more paperwork attached. In preregistered confirmatory work, researchers state hypotheses and methods before analyzing results, so planned tests remain distinguishable from later exploration. Lab notebooks record what was done and observed, in order, and corrections go forward instead of erasing what happened. Review does not make a finding true, but it gives someone who did not produce it a chance to challenge the reasoning, evidence, and record. Together, those practices make a conclusion easier to inspect, test, and correct months or years later (cf. software development).

An AI working across dozens of sessions on a project that matters is the kind of long, error-prone process that benefits from that discipline.

So the Work Ledger adapts five of those habits, and each one earns its place by fixing a specific way things go wrong without it.

A briefing memo, not a transcript

Every substantial session begins by your harness being forced to read a small, current set of files: what’s decided, what’s still open, what the project actually looks like right now. That bounded set comes first; older material is retrieved when the task needs it. If your curriculum committee rejected one assessment plan and adopted another, the current file says so, plainly, in one place. Nobody has to reconstruct the governing decision from fourteen old conversations and the tone of a meeting.

Primary sources, kept separate from what you cite from them

Any historian, lawyer, or journalist already knows why you don’t let your notes about a document start standing in for the document itself. The Ledger first makes the project’s source boundaries explicit, then keeps the original sources inside those boundaries—the actual policy, research paper, feedback, or dataset—separate from the smaller, extracted claims a later task uses. The harness is instructed how to make summaries in systematic ways. And the workflow does not let a summary silently outrank the material it summarized, so a task that needs three sentences does not have to load eighty pages by default while a task that turns on the details in the eighty pages can leverage them. Finally, the provenance remains inspectable: you can see what an extracted claim came from without pretending every source plays the same role.

A prospective task record

Before the AI starts a substantial task, it writes down what it’s about to do: the objective, exactly which sources and decisions it’s relying on, what’s still uncertain, and whether it thinks it has enough to go on. Then it does the work.

Writing that down before the task, not after, is the whole point. A record composed afterward can always make a wandering process look tidier than it was. Writing it down up front means that if the AI later asserts something its own stated sources don’t support, the conflict is visible to the owner or reviewer; the task can’t borrow authority from its own confident prose.

A lab notebook for claims that matter

Most day-to-day assumptions, propositions, or conclusions don’t merit being treated as notable claims in their own right. But a few are load-bearing: governing principles, settled conclusions, or other claims later decisions will be built on. For those, the Ledger keeps a dated, append-only assessment history. Entries do not disappear when they turn out to be wrong; material corrections supersede them, with the old entry and the reason still there to be checked.

Before something enters that record, the AI runs it through a structured sequence I call TRACER:

• define the Target claim;

• Reconstruct what it means and what it rests on;

• Assess the evidence and each source’s role;

• Challenge it with the strongest reason it may be wrong;

• Establish the warranted stance and confidence; then

• Record what should change, what later work depends on it, and when to revisit it.

The rigor scales with the stakes—nobody needs a six-part defense to reschedule office hours—but for anything the project will lean on later, the AI has to show its work before it gets to keep the conclusion.

This is the most Graham-opinionated part of the Ledger’s functionality, I would say, because it basically operationalizes how I as a philosopher think about how one should be in a “justificatory stance” about serious claims. (I taught a similar process in my courses, applied to specific philosophical domains.)

A fresh review, and a paper trail that doesn’t get rewritten

Before a change to one of those load-bearing claims becomes official, it is frozen and checked in a fresh pass against the exact candidate being proposed. Change the wording, evidence, dependencies, or affected files, and the old review no longer counts. Only after acceptance does it become part of the project’s durable history.

The durable history itself lives in ordinary Git version control: each accepted checkpoint is dated, attributable, inspectable, and recoverable. (A local Git commit does not itself publish the repository to the internet; pushing it to a remote is a separate action, and needn’t be involved at all if there are reasons to keep the records for a Ledger entirely local.)

The distinctive part is how those pieces fit together. Under this workflow, sources do not become claims by accident. Confidence alone does not make a claim current. Superseded claims remain traceable, and dependent work can be found when a premise changes. The Ledger does not decide the project for you; it makes the current decision, its authority, its evidence, and its consequences inspectable.

This is also the operational answer to an argument I’ve made twice before (AI Integration is Like Hiring and Will Students Need Management Skills as AI Develops?): if AI is something we increasingly manage, its authority, evidence, handoffs, and review cannot live only in the manager’s head.

⚙️ How It Works, Literally

Those are the disciplines. Mechanically, the Ledger is much less exotic: one local folder of ordinary text files, with Git keeping the history. Inside current/, a handful of Markdown files divide up the briefing work: context-pack.md says what the project looks like now; decisions.md says what governs; open-loops.md preserves what has not been resolved; technical-preferences.md keeps stable working preferences; and context-map.md tells the AI where to look next. Original material lives separately under sources/; substantial tasks get dated Markdown records under dossiers/; deliverables live under artifacts/; and the relatively few claims that merit formal treatment go into a small, append-only current/claims.jsonl record. The default version of the Ledger needs no database, background service, or Ledger-supplied program running behind the scenes, and the AI retrieves older material only when the task calls for it instead of loading the whole history every time.

Initialization is a one-time setup inside that folder, not a training run or an import of every conversation you have ever had. You open it as a project in Claude Code or Codex and answer five plain-language questions: what kinds of files will enter the workspace, how much understanding should become reusable, how detailed the source notes should be, how the AI should communicate, and whether it should save approved changes locally for you. Those answers bind a unique Ledger identity to an operating profile; they do not quietly turn on web access, connectors, external actions, public sharing, or remote synchronization. When you add the first sources, the AI preserves eligible originals and their provenance before proposing what, if anything, should become reusable project understanding for you to correct or approve.

After that, you use the AI normally. At the start of a substantial task, the harness first checks that the Ledger’s identity and claim record still agree and that it is not stepping over a half-finished checkpoint. It reads the bounded current brief, then opens one dated task dossier before doing the work: the objective, the sources and decisions being relied on, the remaining uncertainties, and the intended output. Most conclusions never enter the formal claim record. If the work changes something load-bearing, TRACER exposes the evidence, strongest challenge, confidence, consequences, and conditions for reconsideration; a fresh pass checks the exact proposed change; and only an accepted version becomes a local Git checkpoint. Later corrections supersede the old assessment rather than deleting it.

Claude Code and Codex enter through different project instruction files—CLAUDE.md and AGENTS.md—that point to the same shared protocol and the same project-owned state. The claim is not that their internals are identical: their configuration syntax, permission systems, and review mechanisms differ (this is just something we have to navigate, given how Anthropic and OpenAI have approached their harness development), and the Ledger keeps those differences explicit.

The invariant is narrower and more useful: given the same role and authority, both get the same editable paths, source and action boundaries, review and recovery obligations, and saved outcome. Because the authority lives in the folder rather than either product’s private memory, I can close one harness, open the same project in the other, and keep going from the same record—one active writer per shared worktree, so their edits do not silently collide.

🛠️ What this Looks like in Practice

Come back to that course redesign after six weeks away. You open a fresh session. The AI reads the current state first: the assessment redesign is active, the faculty group adopted a shared rubric, and the oral-assessment question is still open. It doesn’t start by rereading every chat and every draft since January.

You ask it to revise the assessment instructions to match the adopted rubric, without touching the still-open oral-assessment question. It writes down, before doing anything, exactly which rubric, which committee decision, and which feedback it’s using, and flags the open question as something to leave alone rather than something to helpfully resolve for you. Then it does the work and checks the result against what it said it was trying to do.

Maybe that’s the whole story: the instructions got better, and nothing about the project’s settled understanding changed. Maybe the revision also reveals that one current decision was incomplete. If so, that gets written up in proportion to how much it matters, checked, and folded into the record. Three weeks later, the next session starts from the corrected understanding, with the source and the reason still there to check.

What this guards against is a specific, recurring list of failures: an abandoned idea resurfacing as a decision, a summary outranking the thing it summarized, a stray file becoming part of the project’s understanding just because it happened to be lying around, or a later session keeping a conclusion while losing the reason for it.

✅ Has it Earned its Keep?

For my own work: yes, clearly. I’ve used this across at least five major active workstreams for months, moving between Claude Code and Codex as the work demands. It has repeatedly done the thing I built it to do: let me pick a consequential project back up and trust that what it says is current, decided, and backed by something I can check.

In one case, the Ledger caught an analysis that was about to reopen work I had already finished. In another, its reviewer rejected a status update because the evidence showed that a beta installation was working, but not the stronger claim the draft was making. The wording changed before it became project memory. That’s not a hypothetical benefit. It is the failure mode I built the Ledger to stop.

I’m not the only one it’s worked for. One beta tester had been using Claude Cowork more than four hours a day for more than six months. After a day or two with the Ledger, she told me, unprompted, that it was already significantly better for her work, even without having the benefit of as much context as her Cowork. She told me that it repeatedly would one-shot the documents she needed from it (she didn’t need to edit them in any significant way afterwards, and she didn’t need to iterate with Code to get them right).

That is the early signal I wanted: the Ledger can help inside somebody else’s day-to-day AI workflow, not just the system I built around my own work. The third post in this series will put it in front of you, so you can see whether it earns its keep in yours.

🗺️ Where it Fits in the Optionspace

Some of the key building blocks here—Markdown files, Git history, project instructions, and a record of sources—aren’t new, and I’d be overselling the Ledger if I told you nobody else had assembled them. TRACER, on the other hand, is entirely novel as a harness instruction set, as far as I know. But, again, several tools are in the vicinity, trying to solve these problems. Here’s the comparison:

One development worth watching: Google’s Open Knowledge Format v0.2 gives portable Markdown knowledge files optional fields for provenance, verification, freshness, lifecycle, and attestation. That points in the same direction: portable context needs explicit trust signals. It still adds a vocabulary, not the Ledger’s governing workflow for what becomes current, what a task may rely on, or how a load-bearing claim change earns acceptance.

But even if Google’s efforts gain traction and maturity, I’m not convinced we’ll end up in a world where massive harness providers like OpenAI and Anthropic converge on a shared file architecture that’s powerful and harness-neutral. It isn’t in their interests from a lock-in perspective or from a model training perspective. Their current products point toward provider-specific model–harness integration instead: Google already offers Gemini CLI and Jules, while Meta says Muse Spark 1.2 was co-trained with Muse Code around goals, compaction, and subagents.

Regardless, I have work to do. So, for now and probably for quite some time, I’ll be working on the Work Ledger: a harness-neutral system for governed work continuity with an evidence trail, for the specific case where being wrong about the current state costs you something real.

The final post in this mini-series lands Tuesday, September 8. I’ll give you a starter version of the Work Ledger that you can put on some real projects of your own and test against everything I’ve just argued, plus a step-by-step guide to get it running.

P.S. I have also built Schedule Workbench for a departmental problem I saw firsthand: assigning faculty to courses through spreadsheets, email, and a great deal of tacit knowledge. The tool I built imports the agreed information, pauses for review, and then makes proposals that a human scheduler can edit, lock, use, or leave alone. It also preserves checkpoints, history, and exports. You can see a walkthrough at scheduleworkbench.com. If struggles with the same process, I would be interested to hear how it works and if you think I could help.

Graham

Let's transform learning together. Feel free to connect on LinkedIn! (And press reply to contact me: all replies go directly to my inbox.)

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I share some updates on where I’ve been, I explain agentic harnesses like Claude Cowork and OpenAI Codex, and I discuss how I’ve accelerated my productivity with them.

Hi—

Before I tell you where I have been, a request: please hit reply, (re)introduce yourself, and tell me how you are doing and what you are working on.

I mean it! It has been quiet here for a while, and I know that a group as large and diverse as this one (roughly 5,500 professors, learning experience designers, and edtech product people) contains an equal number of rather different stories: courses redesigned, jobs changed, research stalled or finished, AI adopted enthusiastically, AI avoided with equal enthusiasm, …

I should also be clear about what this email does and does not announce. This is the first installment of a deliberately finite three-part series. The second will arrive Monday, August 31, and the third on Tuesday, September 8. I may write other things later, but I am making no promise that the old weekly cadence is returning. In fact, I can promise it won’t. It was tough to sustain!

What I can promise is that these three pieces concern the most important practical change in how I think about AI since I last wrote to you regularly. And, to boot, I’ll give you a tool to leverage it!

👋 “Where Have You Been?”

For readers who joined long ago, or who understandably no longer keep my biography near the top of their working memory, I am Dr. Graham Clay. I trained as a philosopher, spent close to a decade teaching and researching at universities, and eventually moved most of my work toward AI and education. Through AutomatED, I wrote about AI pedagogy, helped professors and institutions think through its use, built AI workflows and tools, and tried to clearly and fairly explain the technology in its many manifestations.

Then, after consulting for a while, I stepped away from the regular AutomatED rhythm to join Kognity, an interactive digital curriculum and learning platform focused on the IB DP curriculum trusted by 2,000+ schools across 130+ countries, as its full-time AI Strategist.

The title I had is somehow both too vague and quite accurate. I led company-wide AI content-generation strategy across evaluation, workflow architecture, agentic integrations, people, and tooling. I helped build systems for writing curriculum, assessments, and teacher materials. At peak I managed quite a few people, directed several content production teams, and I helped a content organization of roughly 20 people move from predominantly manual production toward designing and operating AI workflows.

That last sentence is easy to read too quickly.

A content organization that adopts AI does not simply give everyone a better autocomplete box. Roles change (a lot). The people who once wrote or edited each individual item begin designing the process by which thousands of items are created and checked. Learning experts who once applied their judgment one artifact at a time begin writing rubrics, constructing evaluation datasets (norming scorers around the rubrics, identifying outliers, and ensuring statistical benchmarks are met), calibrating AI judges, and deciding which disagreements matter. Subject-matter experts and learning experience designers remain responsible for “final” review, while their expertise also moves earlier into the architecture of the system.

The AI systems I helped build at Kognity produced 35–90% lower costs and 50–90% faster delivery than manual baselines. One effort shipped 480 teacher slide decks (20-50 slides each, with diagrams, teacher supports, and more), which reached about 10,000+ downloads. But the numbers belie the messy process that led to the results. Many execs seem to think that we can sprinkle AI over a process and deduct 90% from the cost, but it’s far from the truth.

The deeper lesson I’d share concerned where human expertise belongs.

We often imagine “human in the loop” as a person stationed at the end of a production line, inspecting whatever the AI has made. Sometimes that is necessary. Yet it is only one place for human judgment, and often not the most important one.

Experts also need to design the workflow, determine what good work consists in, encode that standard in rubrics and stated process, assemble the datasets against which judgments are tested, validate the AI systems that apply those judgments, define review gates, and decide how organizational roles should change. A learning designer’s judgment becomes more powerful when it is made explicit, tested, versioned, and applied across many outputs. It also becomes more demanding, because the organization must be able to say what standard it is using, who established it, and where the system should stop.

That work changed the question I ask about AI. For years, the natural question was: What can this model do? (For those of you who are long-time readers, you saw it in my many contests to see if AI could complete a given academic task.)

Now, the fact is that AI can complete most work when positioned properly in our working environments. So, the more pertinent question is: What kind of working environment should we build around AI to position it most effectively?

🤖 Agents? Harnesses?

Nowadays, the environment around the most powerful deployments of AI models is often called an agentic harness, a term apparently designed to make an important idea sound like outdoor equipment.

The model is the part that takes in a prompt and generates a response, while reasoning along the way (read: generates intermediate responses that guide its production of subsequent ones by becoming parts of their prompts). The harness is the software environment around it. Depending on the product and the permissions a user grants, a harness may let the model inspect project context, search files, use tools, edit documents or code, browse the web, connect to other services, complete a sequence of steps, recover after an interruption, and coordinate several agents working on different parts of a task.

A blank chatbot answers a prompt. A harness can participate in situated work.

Longtime readers will recognize the argument underneath this. I have argued in this newsletter before that increasingly capable AI is better understood as something like a new colleague than as ordinary software (see AI Integration is Like Hiring), and that what usually stands between a capable model and a useful result is what I once called integration effort: the work of giving it the right context, connecting it to the right tools, and checking whether the output is actually good.

A harness is, by design, intended to reduce integration effort, and thus if you are a user of AI, you should dedicate your own integration effort to understanding and improving how your chosen harness works instead of supplying AI the relevant inputs manually or semi-manually every time.

Suppose a teaching center is revising its AI policy after a difficult semester. The work involves the existing policy, faculty feedback, institutional constraints, examples of assignments, minutes from several meetings, a new decision about student data, and an unresolved question about assessment. In a blank chat, somebody has to assemble that context, explain which version controls, request a draft, and manually compare the answer with the documents.

Inside a capable harness, the system may be authorized to inspect the project, find the relevant committee record, apply the project’s instructions, draft the revision in the correct file, run checks, show the changes, and retain enough task state to continue later. The quality of the model still matters enormously. The useful work also depends on what surrounds it.

I think this interaction form will persist even though the present products, interfaces, and leading models certainly will not remain fixed.

One reason is that a harness connects model reasoning to the actual circumstances of work. Organizations do not operate through isolated questions and answers. They operate through documents, databases, permissions, recurring processes, decisions, and people with different authority.

A second is that a harness makes iterative verification possible. An AI can inspect, act, test, compare the result with the requirement, and revise. For a professor, that might mean checking an assessment against learning objectives and a rubric. For a teaching center, it might mean comparing a proposed guide with institutional policy and the cited sources. Work becomes a loop rather than a single impressive-looking response.

And the harness gives an organization somewhere to encode how work should happen: which sources govern, which tools may be used, which files may be changed, what requires approval, and how a result should be reviewed. Those controls will sometimes be weak or badly designed. Still, the possibility is substantially more important than asking every employee to become a virtuoso prompt writer in a blank text box or, even less productively, writing manually.

💡 Four Harnesses, Four Ideas of Work

The differences among current harnesses help explain why the surrounding environment deserves more attention.

The first one to the market to make a big splash was Claude Code. It is centered on the terminal and the software repository. It can follow project instructions stored with the work, load rules when particular paths are involved, use local repository-scoped memory, call tools, and place parallel work in isolated Git worktrees. Its design assumes that the repository is a durable working environment and that continuity should serve developers operating inside it.

Cowork presents another model, also from Claude but not for software developers. It is a desktop knowledge-work harness rather than a developer-first terminal tool. Projects can bundle files, links, instructions, chats, connectors, and project-scoped memory. That makes the interaction more coherent for the many intelligent knowledge workers who have no desire to learn Git or operate inside a terminal. Continuity is correspondingly tied more closely to the product’s own project and memory system.

OpenAI’s Codex is also highly capable at local repository work, but it can sit inside the broader ChatGPT Work surface (OpenAI’s analog to Cowork). That broader environment can combine projects, chats, files, connected sources, tools, and longer-running tasks. It is important, however, to distinguish checked-in project instructions and files from native personal or project memories. Those memories may be helpful recall, but they are separate stores with their own behavior. For consequential work, I would no more make a product’s synthesized memory the canonical record than I would make my recollection of a committee meeting the official minutes.

Then there is Muse Code, Meta’s new terminal coding harness. It has a local event log to which model calls, tool runs, approvals, and edits are appended before action; crash-and-restart recovery; persistent background agents; isolated worktrees; and a model co-trained with the harness itself. Muse’s event log is intended to answer: What did the agent do, and where should it resume?

A project ledger, like the tooling I’ve been developing (more below and on Monday) answers another question: What is currently true, what has been decided, what supports it, and what remains unresolved?

Perfectly replaying an agent’s actions would not necessarily tell a new collaborator that a decision was later superseded, a source turned out to be weak, or an apparently promising proposal was rejected. Conversely, a concise project record will not reconstruct the precise seventeenth tool call in a crashed 20-hour coding session. Execution recovery and project continuity live at different layers.

The comparison points to something larger. Models increasingly arrive inside opinionated runtimes, developed by companies that want you to stay in their product space (let’s be honest). Their memory stores, permissions, context-loading rules, recovery behavior, and relationships to files all differ. Some are optimized for code, some for desktop knowledge work, and some for a product’s own cloud project space. Model–harness co-training may widen the differences further.

🧠 My (WIP) Answer: the Work Ledger

These products make it inaccurate to say that agents simply forget everything. They can carry instructions, memories, files, summaries, task state, and repository context across sessions. In many ordinary cases, native continuity may be entirely sufficient.

A system can remember far more than a blank chat, however, and still leave a project with state that is fragmented across products, stale relative to the controlling source, opaque to the user, or difficult to move elsewhere.

Consider a course redesign that lasts six months. One chat remembers an early assessment plan. A project contains the revised rubric. A local coding agent has learned a formatting preference. A committee later rejects the original assessment plan, but that decision lives in meeting notes none of the other systems automatically treats as authoritative.

Nothing here requires an AI to have forgotten. The danger is that it remembers the wrong thing, or that it cannot distinguish recollection from a governing decision.

The same problem appears in research projects, accreditation work, curriculum revisions, grant development, faculty-development programs, and departmental policy. Useful continuity requires selection: the current objective, accepted decisions and their reasons, relevant sources and constraints, open questions, and the authority boundaries around the next task. More context does not automatically produce better context.

Over the past several months, I have been developing a system for this problem in my own cross-project work.

The Work Ledger is a local, Git-backed, human-readable repository designed to work across AI harnesses. (Yes, read that again: it is harness neutral, in that it enables you to move your work from one harness to another, so you are neither locked in nor stuck when you run out of usage with one.) It is for projects with continuing workstreams, sources, decisions, constraints, preferences, and unresolved questions. One way to put its core idea succinctly is: governed work continuity with an evidence trail.

In practice, a few things happen. Before an AI touches your project, it reads a short, current summary of where things actually stand, not the entire back-catalog of everything anyone has ever said about it. It also keeps your original sources (the actual policy, the actual feedback, the actual data) separate from whatever gets pulled out of them later, so a rough paraphrase never quietly stands in for the real thing. And before it starts a real task, it writes down exactly what it's about to rely on and why, before doing the work rather than after, so you can catch it in advance if it's about to lean on the wrong thing.

For the handful of conclusions a project will actually keep building on, the bar goes up further. The AI has to say what it's claiming, what's actually backing it up, and the strongest reason it might be wrong, before that gets to count as settled (I call that check TRACER; the full mechanics are explained in Monday's post). And nothing becomes official until a second, independent pass checks it and logs it in a permanent, dated record you can always look back through. Put simply: the AI has to earn the right to call something true, and either way, you get the receipts.

That is a great many mechanisms. Their practical effect is simpler.

When I open a fresh session, including in a different harness (I often run out of usage with one harness and need to keep working; 1-2B tokens a day burns through even the heaviest Pro plan with Codex 🫣), I want the AI to find the current project state without treating every old conversation as equally authoritative. I want it to know which source supports a claim, which decision replaced an older one, what remains uncertain, and what it is allowed to change. When a task ends, I want a reviewable account of what changed before the result becomes part of the project’s durable memory.

The system has established its value for my own work. It has helped me resume consequential projects, recover source relationships, and move between AI surfaces without reconstructing the whole argument from memory. One beta user I’ve shared it with says it is significantly better than native Cowork, even after a day or two of use. And this is coming from someone who used Cowork 4+ hours a day for more than 6 months!

But it won’t be a fit for everyone. The people for whom this may matter most are those with continuing workstreams and real consequences if an AI resumes from the wrong premise. If you occasionally ask ChatGPT to improve an email or suggest discussion questions for tomorrow, you do not need this level of structure. Please enjoy your lighter life.

But if you have ever returned to a project after three weeks and thought, “I know we settled this, but I no longer trust either my memory or the chat history,” then you know the shape of the problem.

On Monday, I will examine the Ledger itself and compare it with Obsidian, native AI memory, Memcrate, LLM Wiki-style systems, and related alternatives. Some of them solve portions of the problem with much less friction, which is precisely why the comparison is worth making.

In essence, different types of harnesses are different types of teams of people that have a thin management layer built-in, with the memory substrate you choose, like the Ledger, adding a more opinionated operational process on top of that base layer. As I argued back in April 2024 (“Will Students Need Management Skills as AI Develops?”), AI is increasingly something you manage and direct, so your theory of how to do so should reflect your broader views on management, operations, and process.

On Tuesday, September 8, I intend to share a version of the ledger for you to use.

For now, hit reply. Tell me who you are these days, what you are working on, and whether continuity across AI-assisted work is a real problem for you or merely an elaborate solution looking for one.

Graham

Let's transform learning together. Feel free to connect on LinkedIn!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I take on another big topic: how to conceptualize the process of adding AI to your workflow, department/team, or institution/organization.

I also remind you about my webinar this coming Friday, which will concern Zapier.

📝 This Friday: 
Creating AI Automations with Zapier

This Friday, May 2, from 12 until 1:30pm Eastern, I’m going to cover how to use Zapier (and similar apps) to improve your AI-powered workflows and processes.

If you don’t know, Zapier is a no-code/low-code automation tool that enables you to connect any 2 or more apps together, from ChatGPT’s o3 model to Gmail to Microsoft Excel (and 5000+ more), in a workflow that is automated based on triggers, time periods, etc.

In short, it’s an easy way to get data and software, including AI, to talk to each other in a way that saves you time and headaches.

I would argue that Zapier — or another tool like it, such as Make or n8n — is one of the most important tools you learn to use in the AI era, on a par with LLMs like ChatGPT themselves. I save tons of time with it every week, and it was key to running AutomatED while teaching full-time.

As part of this webinar, I will convince you that you should use it (trust me, this won’t take long), before pivoting to demonstrating 3 simple ways to use it that can change your life as an educator:

1. Responding to students’ queries and needs in a dynamic and timely way

2. Meeting management — before, during, and after

3. Grading and feedback — never do rote work again, and instead focus on what makes you special as an evaluator

These 3 demonstrations will show you general strategies for using Zapier that you can apply to whatever interests you in automating.

All registrants will receive the recording afterwards, even if they don’t attend live.

Webinar Details:

• Cost: $25 ($0 for ✨Premium subscribers, like all webinars this year; discount code here, if you’re logged in)

• Format: ~70% presentation, ~30% discussion (come ready with questions and comments)

👉👉👉

📢 Quick Hits:
AI News and Links

1. Anthropic launched Claude for Education, their education-focused AI offering (similar to Google's LearnLM) featuring "Learning mode" that uses Socratic questioning rather than providing answers directly. They've established partnerships with Northeastern, LSE, and Champlain College for campus-wide access, and are collaborating with Instructure to integrate with Canvas LMS. The initiative includes a Campus Ambassadors program and API credits for student projects.

2. Relatedly, Anthropic released a study on how university students use Claude, analyzing one million conversations from higher education emails. Key findings: STEM students, especially Computer Science (36.8% of conversations despite only 5.4% of U.S. degrees), are early adopters, while Business, Health, and Humanities show lower adoption rates. Most concerning for educators: Claude primarily completes higher-order cognitive functions like Creating (39.8%) and Analyzing (30.2%) from Bloom's Taxonomy — suggesting students may be outsourcing exactly the cognitive skills we most want them to develop.

3. OpenAI released o3 and o4-mini, their "smartest and most capable models to date," which can now use all ChatGPT tools (web search, file analysis with Python, image understanding, and image generation) while thinking through problems. Unlike standard models, these "reasoning models" are trained to think longer before responding, resulting in significant performance improvements in coding, math, science, and visual tasks. I have been especially impressed by o3 in my testing so far (it’s the model that has been powering Deep Research; previously, this was the only way to use it).

4. Anthropic launched "Research", a new capability that transforms how Claude finds and processes information. Unlike typical search, Claude conducts multiple searches that build on each other, determining what to investigate next while systematically exploring different angles. This complements their new Google Workspace integration, which connects Claude to your Gmail, Calendar, and Google Docs — enabling it to search emails, review documents, and consider calendar commitments when answering questions. For educators and students, this means Claude can now analyze learning materials, scan notes from past courses, search the latest academic research, and create personalized study plans. Available in early beta for Max, Team, and Enterprise plans in the US, Japan, and Brazil.

5. A fascinating new field experiment from Harvard with 776 Procter & Gamble professionals found that AI fundamentally transforms teamwork dynamics. The study revealed three key insights: (1) individuals with AI matched the performance of human teams without AI; (2) AI eliminated functional silos between R&D and Commercial professionals, helping both groups produce balanced solutions regardless of background expertise; and (3) AI's interface prompted positive emotional responses, partially fulfilling the social role traditionally provided by human teammates.

💡 Idea of the Week:
Reconceptualize AI’s Role

The Friday before last, I found myself in St. Petersburg, giving two presentations/workshops at the University of South Florida. The first session included local businesspeople and community leaders. What always strikes me about this sort of interdisciplinary session is what people — very different from me in expertise and background — find interesting or useful.

There was one topic that several businesspeople discussed with me afterwards: namely, my suggestion that we need to stop treating AI tools like software purchases and start approaching them as new team members we're bringing onboard.

Having spent the last year helping institutions, professors, instructional designers, and companies navigate their AI adoption journeys — some gracefully, others with considerable struggle — I've become convinced this mental shift is critical.

Let me walk you through why this matters and how it might transform your approach to AI integration, whether as an individual or as a member of a bigger organization.

Traditional Approach: AI as “Software”

Most of us have been part of a software acquisition decision-making process at some point. We create feature comparison spreadsheets, check for LMS compatibility, verify FERPA compliance, and wade through licensing terms. Or, as individuals, we Google options, skim user reviews, and seek expert-evaluated comparisons. (I think I — “cutting-edge AI guy” — just dated myself a little bit. Oops!)

When ChatGPT burst onto the scene, many institutions and individuals defaulted to these same evaluation processes.

This approach isn't entirely wrong — Claude, ChatGPT, and Gemini are, after all, software in some sense. And it made a lot of sense early in AI’s development, like up until 2023, perhaps.

But watching higher educators of many stripes struggle with implementation, especially as AI has become more intelligent and more tool-connected over these past two years (see above for two examples from Claude and ChatGPT), makes it clear that something crucial is missing from this framework.

Better Approach: AI as “New Team Member”

Here's the perspective that resonated with those USF faculty:

Today's AI systems — whether Deep Research, Operator, or more advanced setups like Google’s Co-Scientist — aren't just executing pre-programmed routines like traditional software. They're dynamically adapting to your specific needs/data, learning from feedback, and operating with increasing autonomy, much like a new colleague would.

Consider how differently we approach these scenarios:

Software acquisition: "Does this product have all the features we need right now? What's the annual cost? Is it bundled with something we are already buying? Who handles maintenance?"

Team member acquisition: "How quickly can they learn our processes? What information do they need to succeed? Will they work well with our existing team? How much supervision will they need? Do their working methods align with our department's approach?"

This second set of questions gets much closer to what actually determines AI implementation success in higher ed settings, here at mid-2025 and going forward.

It’s a version of the approach I first outlined from a different angle a little over a year ago in my newsletter titled “Will Students Need Management Skills as AI Develops?”

What This Means for Different Roles in Higher Ed

For Department Chairs: When faculty propose developing a department-wide AI tool for assignment feedback, don't just ask about cost and security. Ask: "Who will 'mentor' this AI on our department's assessment approach? How will we ensure its work aligns with our program learning outcomes? In what ways should our process be modeled on how we train teaching assistants and adjuncts?"

Just as you shouldn't hire an adjunct or task a graduate student with grading 100 papers and immediately turn them loose without orientation, training, or some sort of verification process, you shouldn't deploy AI without similar onboarding, either via native functionality, tool integration, or prompt engineering.

For Instructional Designers: You're often bridging faculty needs with technological capabilities. Instead of just showing professors how to use AI features, help them develop "training protocols" — collections of exemplars, rubrics, and style guides that help the AI understand their specific teaching approach — or use the latest AI tools that do this for them.

For Individual Faculty: Think about the "working relationship" you want with AI tools. Do you want ChatGPT to be your research assistant, teaching assistant, or content developer? Each role requires different "training" approaches, tool needs, and supervision levels. What would you tell a generalist human — smart but not knowledgeable about your domain or tasks — to get them up to speed? How would you check behind them? When would you decide to let them operate without oversight, or with less?

Upskilling Ourselves as "AI Managers"

This new paradigm requires us to develop skills more akin to personnel management than technical administration, as I predicted in my piece a year ago:

• Effective onboarding: Creating clear documentation and examples that teach AI systems about your specific context and expectations, or integrating them with a team or environment that trains them on-the-go (this will become more and more common as AI has native access to our data).

• Appropriate delegation: Identifying which tasks benefit from AI augmentation and which require human touch.

• Ongoing supervision: Establishing review procedures that ensure AI outputs maintain quality and alignment with goals and values.

• Performance evaluation: Developing metrics beyond technical functionality — Is the AI actually helping students learn? Is it reducing faculty workload or just shifting it?

Some of these skills aren't typically covered in standard faculty development workshops but are becoming essential as AI becomes more deeply integrated into our work.

I believe this perspective shift from "software" to "team member" represents more than just an implementation strategy — it's a fundamental reorientation that will determine which individuals, departments, and institutions successfully navigate the AI transition.

In the coming years, we'll see increasing autonomy in AI systems. Those treating AI as merely another software purchase will find themselves with powerful tools that nonetheless fail to integrate effectively into their teaching and research ecosystems. Meanwhile, institutions that have developed robust "AI onboarding" and "training" protocols will leverage these systems to enhance student support, accelerate research, and create more personalized learning environments.

Put simply, the most forward-thinking universities and organizations aren't asking "Which AI tools should we buy?" They're asking "How do we build effective working relationships with increasingly capable AI systems?" and “Who — not what — would I want in my work ecosystem, if I could have them on demand?”

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I try to reframe your thinking around AI’s role in general and in higher education. When thinking about AI, it’s most productive to think not about what knowledge work AI can do, but rather about how much it would cost to get it to do what needs to be done.

I also share the sign-up link for Friday’s $25 webinar on AI grading and feedback (a popular topic I present on to institutions around the world).

📝 Save the Date: Webinar This Friday

This Friday (April 11th, 12-1:30pm ET), I'm hosting a Zoom webinar about all the different ways you can save time and improve your grading/feedback with AI.

Here’s what I'll cover:

• Options that fit your views on the ethics of using AI for these tasks, whatever your views are

• AI-driven feedback methods that don’t share student work with AI

• Ways of grading student work with AI that overcome AI weaknesses and don’t violate students’ data privacy rights

• How I help specific, real professors save time — and avoid headaches — with AI feedback/grading workflows

Webinar Details:

• Cost: $25 ($0 for ✨Premium subscribers, like all webinars this year; discount code here, if you’re logged in)

• Format: ~70% presentation, ~30% discussion (come ready with questions and comments)

👉👉👉

💡 Idea of the Week: Integration Effort

One of the most frequent misconceptions about AI in higher education is that AI models are incapable of truly advanced, specialized tasks. But in my experience — developing AI tutoring solutions for institutions, custom AI graders for individual professors, and sophisticated content with subject matter experts — there’s no knowledge work AI can’t at least assist with significantly.

The real sticking point is what I call “integration effort.”

Integration effort is essentially the work (or cost, in the broadest sense) required to prompt AI, give it the right context or data, connect it to the right software, and manage quality control so it can actually complete the task you have in mind.

For a simple filler assignment or routine email text, integration effort might be near-zero. But when you’re building a sophisticated AI tutor that needs specialized domain knowledge, a consistent “voice” over a whole course, and resistance to student jail-breaking, the integration effort can be substantial.

Below, I want to argue that while integration effort is currently a significant obstacle for some tasks, it’s decreasing rapidly and unlocking more and more tasks for AI each semester. And, crucially, I’m seeing that those who don’t invest in understanding, analyzing, and minimizing integration effort in their domains are lagging far behind.

AI Can Do (Virtually) Anything — If We Let It

Over the last year, I’ve worked with clients who were convinced that aspects of their work “just can’t be done by AI.”

One instructional designer I worked with believed that AI could never produce the nuance needed for an advanced philosophy of education module; then I showed them a multi-step prompting workflow using two models in succession (one for initial drafting, another for in-depth editing) and a third process for content validation. The result, produced with $5 of API calls and taking ~15 minutes, rivaled what human designers typically produce in weeks.

In my experience at this point in time in 2025, the barrier is rarely AI’s inability. Instead, the biggest challenge is the integration effort required to set everything up:

• designing custom prompts

• weaving in relevant references/information

• developing a multi-step “flow” that chains the prompts together and handles quality assurance/errors/etc.

• and training the humans (faculty, staff, students) to create, use, and modify these processes effectively.

Practically any knowledge task can be done, but prompting and process design is a skill in itself. When folks assume AI simply “can’t do that,” they often haven’t either (i) used the latest models or (ii) explored the flow-creation techniques that unlock the real power of these tools.

This leads me to my next point…

Determining Integration Effort is Key

Because integration effort can be the difference between massive success and total frustration, it’s critical to figure out — as early as possible — how much integration effort is needed for any valuable task.

This is especially true in higher ed, where time is a premium and academic units often move at a glacial pace, as we’ve all seen.

If you’re a professor who wants to grade 200 midterm essays in a few hours with the help of AI, you need to know how much time you’ll need to spend on prompt calibration, flow creation, etc. to handle edge cases and detect certain nuances. Is it worth it to do this, rather than simply committing to an AI-free method to grade them all over a week or two (like it’s 2009)?

It’s easy to miss these hidden costs. Yet, your approach to integration effort can determine how quickly you see real gains in efficiency and quality. If you ignore it altogether, you’re making decisions in the dark about which tasks AI can (or cannot) take off your plate.

Some Tools Make Integration Easier

AI development companies are trying to reduce friction.

NotebookLM from Google, for instance, can dramatically cut the time it takes to onboard your documents and have an AI agent refer to them accurately. That means you can set it up to read your syllabus, lecture notes, or student questions, and produce relevant responses without too much engineering.

Similarly, specialized tools like OpenAI’s “Operator” and “Deep Research” are being designed to reduce the headache of context management and tool integration, so you can minimize how much prompt engineering, tool connections, etc. you need to do.

Yet, from my vantage point, so many use cases in higher ed still require tailor-made solutions. Maybe you need advanced data analysis features connected to your LMS or your scheduling apparatus, or a system that can handle your institution’s unique compliance requirements.

At that point, the built-in functionalities of a prepackaged AI product can’t cover it — bespoke solutions (and their associated integration effort) might pay off. But you won’t know until you begin scoping out the task itself, per the above.

Many Are Behind on Integration Effort (and Skills)

In my consulting work, I’m consistently surprised by how many teams (from faculty to staff and admin) are months or years behind the current capabilities and best practices of AI.

Picture a department that’s still debating whether AI can write good multiple-choice questions for basic 100-level content (it can do very easily, regardless of subject), while, meanwhile, entire college systems are developing customizable AI tutors, structuring their unstructured advising data with AI, or using AI to turn out entire modules of HR training content in a day.

The main difference is that the latter group has recognized that success with AI depends on devoting time and resources to bridging that integration effort.

Don’t let your institution stay in a secure bubble of “We’re just not sure AI is up to it yet.” With the advent of the reasoning models and more powerful tools, the question is never “Can AI do the work?” so much as “What do we need in order to integrate AI effectively?” and “Is that worth it?”

These are the new baseline questions in the age of advanced AI.

Over Time, AI Itself Lowers Integration Effort

One of the most remarkable developments (even in the last six months) is how prompt adaptability and context management have become core features of new AI releases.

Just as we see with Deep Research, AI can begin to handle tasks like context retrieval and data pipeline set-up on your behalf, gradually collapsing its own integration effort. We’re seeing the earliest versions of AI that can chain together tasks, or fetch third-party data and tools on demand with minimal human guidance.

Eventually, many of the workflows we currently find “too involved” will become automated. That’s the horizon some call “frictionless integration,” where AI can do all of the background data wrangling, validation, and creative rearranging we currently manage ourselves.

At that point, the role of humans in the workplace will shift even more radically than it already is, and higher education won't be an exception. A system that can produce, cross-verify, and deliver tailored content in real time to thousands of students simultaneously will reorganize the entire educational infrastructure — how we staff, how we design curricula, and how we conceive of knowledge transmission itself.

It may feel futuristic, but it’s not as far away as you might think. And, even if you think I’m wrong about this, proving it would require you to join me in trying to figure out the relevant level of integration effort. That’s my point.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Yes, I am still alive! For those keeping track, I didn’t send a newsletter the last two weeks. (In short, I was on vacation and then I was sick.) But I am back in the groove now…

Today, I share 5 notable AI developments from recent weeks, plus my commentary on their relevance for higher educators.

I also announce two webinars for April.

❓ 5 Big Recent Developments in AI

1. OpenAI’s New Image Generator

What Happened: OpenAI integrated a much more powerful image generator directly into GPT-4o, making it the default image creator in ChatGPT. Unlike previous image models, this one excels at accurately rendering text in images, precise visualization of diagrams/charts, and multi-turn image refinement through conversation.

Why It’s Big: For educators, this represents a significant advancement in creating educational visuals, infographics, diagrams, and other instructional materials with unprecedented accuracy and control. It’s not perfect, but you can now quickly generate custom illustrations that accurately display mathematical equations, chemical formulas, or process workflows — previously a significant hurdle in digital content creation — without requiring graphic design expertise or expensive software. This capability dramatically reduces the time between conceptualizing a visual aid and implementing it in course materials. And, perhaps most crucially, the improved text rendering means created materials are more accessible to students, as text within images is now clear and legible rather than distorted or illegible as with previous generators. You don’t need to add labels afterwards, in many cases, and instead can direct for them to be added by ChatGPT.

How to Try It: Free, Plus, Pro, and Team tiers can use it now, with Enterprise and Edu coming soon. To use it, select 4o with the model selector in the upper left corner of your chat screen, then click the “…” button right below the chat input box, then click “Create image.” See below…

2. Google Releases More Educator-Focused AI Courses

What Happened: Google continues to put money into AI education initiatives, including two new courses specifically for K12 (click here) and higher education instructors (click here) on using Google AI effectively in educational settings. These complement their existing resources like the Generative AI for Educators course (created with MIT RAISE), the updated Guardian's Guide to AI, and the Experience AI program.

Why It’s Big: These free, specialized resources arrive just as many institutions are scrambling to develop faculty AI competencies without dedicated training budgets. (Trust me, I know, as funding is often the main obstacle for institutions trying to hire me for PD, trainings, consultations, etc.) For teaching centers and faculty development offices, Google's education-specific courses provide ready-made professional development that addresses pedagogical applications rather than just technical skills. Department chairs can now direct curious but overwhelmed faculty to structured learning paths rather than ad-hoc exploration, potentially accelerating AI adoption across disciplines. Sure, it’s all Google-focused, but it’s better for major tech companies and AI developers to invest in high-quality education-specific AI training than not.

How to Try It: It’s all free, so just click the link above that interests you and go from there! The higher ed one starts with free Google AI functionality and goes to the Premium stuff afterwards. And, at worst, the Premium functionality referenced requires a Gemini Advanced account, but you probably have one for free through your institution or can get one for $10-20 a month (mind you, almost everything can be done in Google AI Studio for free or using the free level of NotebookLM; more from me on the Plus version here).

3. Claude Search

What Happened: Anthropic added web search capabilities to Claude, enabling it to access real-time information from the internet and provide cited sources in responses.

Why It’s Big: For academics and researchers, this means Claude can now help gather current literature, identify research gaps, and assist with building stronger grant proposals, literature reviews, etc. Claude is still behind OpenAI Deep Research and Google Deep Research, but this is an important step in the right direction. Given the frontier capabilities of Claude’s top model (Sonnet 3.7) — I prefer it to o1, o3-mini-high, and Gemini 2.0 Flash Thinking for many tasks — it’s good to see Anthropic begin to expand its functionality with tools like this.

How to Try It: The feature is currently available in "feature preview" to all paid Claude users in the United States (with free tier and international access coming soon). Turn it on with the slider at the bottom of your chat window:

4. Gemini 2.5 Pro

What Happened: Google DeepMind released Gemini 2.5 Pro Experimental, which they describe as their "most intelligent AI model" designed for complex problems. It's a reasoning model that’s currently #1 on the LMArena leaderboard (which measures human preferences), and claims state-of-the-art performance across reasoning, math, and coding benchmarks. For instance, it achieved 18.8% on "Humanity's Last Exam" and 63.8% on SWE-Bench Verified (for coding).

Click here for the full table from Google.

Why It’s Big: Unlike OpenAI's o3-mini-high or Claude 3.7 Sonnet with their 200k token context windows, Gemini 2.5's million-token context window combines with its reasoning power to enable us to complete complex and deep tasks. For instance, you could analyze entire semesters of course materials simultaneously — not just individual lectures or assignments — revealing semester-long progression issues that shorter-context models miss. Grant-writing professors can feed Gemini 2.5 the funding opportunity, their preliminary data, multiple papers establishing the research gap, institutional facilities descriptions, and previous reviewer comments all at once, receiving integrated feedback that shorter-context models simply cannot provide when documents must be chunked or summarized. (I covered this use case with Gemini 1.5 Pro already, and all of my prior discussion still applies with this new, smarter version.)

How to Try It: It’s available now for free in Google AI Studio (select it in the model selector on the right side once you are in the prompt view) and to Gemini Advanced subscribers, with a 1 million token context window (expanding to 2 million "soon").

5. Microsoft’s New Agents

What Happened: Microsoft introduced two AI "reasoning agents" for Microsoft 365 Copilot: Researcher and Analyst. Researcher combines OpenAI's Deep Research model with Copilot's orchestration to help with complex, multi-step research tasks by analyzing work data alongside web information. Analyst, built on OpenAI's o3-mini reasoning model, functions like "a skilled data scientist" to transform raw data into insights using Python (with viewable code for verification).

Why It’s Big: It’s big that Microsoft is taking Deep Research, which is already a beast on its own, and combining it with their pre-existing data organization strengths. To pick one domain for examples, these specialized agents could revolutionize administrative work across higher education, dramatically reducing the time senior leaders spend gathering information for institutional decision-making…

• Program directors might use Analyst to transform years of enrollment and outcome data into compelling visualizations that strengthen program review documentation or accreditation materials without requiring dedicated data science staff.

• Research administrators could expedite grant applications by having Researcher quickly assemble relevant institutional performance metrics and compliance information alongside funding trends.

• For deans and provosts, these tools offer new capabilities to conduct just-in-time analyses of budget allocations, faculty productivity, or student success metrics — turning previously month-long projects into afternoon tasks while maintaining the transparency needed for shared governance through the viewable code.

How to Try It: Both agents will roll out to Microsoft 365 Copilot license holders starting in April through a new "Frontier" program for early access to Copilot innovations.

📝 Two Webinars in April

Due to some scheduling obstacles, I couldn’t host my March webinar. But I’ve got you covered with 2 webinars on the books for April!

1. Friday, April 11th: Using AI for Grading and Feedback

Next Friday, from 12 until 1:30pm Eastern, I’m going to cover all the different ways you can save time and improve your grading/feedback workflows with AI.

As part of this webinar, I will provide a range of options that fit your views on the ethics of using AI for these tasks; regardless of what you think about, say, sharing student work with AI, I will have an AI solution for you to help you do your job better and faster.

Interested? If so, just answer “Yes” in the below poll and I will send you an email with more information and the sign-up link. (Note: the cost is $0 for Premium subscribers and $25 for all others.)

All registrants will receive the recording afterwards, even if they don’t attend live.

2. Friday, April 25th: Why You Should Be Using Zapier (And How)

Then, on Friday, April 25, from 12 until 1:30pm Eastern, I’m going to cover how to use Zapier (and similar apps) to improve your AI-powered workflows and processes.

If you don’t know, Zapier is a no-code/low-code automation tool that enables you to connect any 2 or more apps together, from ChatGPT’s o1 model to Gmail to Microsoft Excel (and 5000+ more), in a workflow that is automated based on triggers, time periods, etc.

In short, it’s an easy way to get data and software, including AI, to talk to each other in a way that saves you time and headaches.

I would argue that Zapier — or another tool like it, such as Make or n8n — is one of the most important tools you learn to use in the AI era, on a par with LLMs like ChatGPT themselves. I save tons of time with it every week, and it’s key to running AutomatED while teaching.

As part of this webinar, I will convince you that you should use it (trust me, this won’t take long), before pivoting to demonstrating 3 simple ways to use it that can change your life as an educator:

1. Responding to students’ queries and needs in a dynamic and timely way

2. Meeting management — before, during, and after

3. Grading and feedback — never do rote work again, and instead focus on what makes you special as an evaluator

These 3 demonstrations will show you general strategies for using Zapier that you can apply to whatever interests you in automating.

Interested? If so, just answer “Yes” in the below poll and I will send you an email with more information and the sign-up link. (Note: the cost is $0 for Premium subscribers and $25 for all others.)

All registrants will receive the recording afterwards, even if they don’t attend live.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I share results from an experiment where OpenAI's o1-pro AI model provided remarkably thorough and helpful feedback on an economics research paper, revealing that advanced AI can serve as a valuable thought partner for us and offering promising applications for research development and student feedback.

As I note, if you want to participate in this experiment with your research, respond to this email to let me know!

The future of presentations, powered by AI

Gamma is a modern alternative to slides, powered by AI. Create beautiful and engaging presentations in minutes. Try it free today.

💡 Idea of the Week:
How Good is AI as a Thought Partner?

Can any current AI serve as a genuinely useful thought partner for academic researchers operating at the frontiers of their fields?

To explore this question and provide evidence to you all about the answer, I am conducting a simple experiment with OpenAI's o1-pro, their most advanced reasoning AI model (which requires a $200/month subscription via ChatGPT Pro; its newer, faster, but slightly dumber sibling, o3-mini, is available to all in a low-compute version).

A few months ago, when o1-pro came out, I invited professors to volunteer drafts of their academic papers, which I would then share with o1-pro, asking it to provide thoughtful feedback as a colleague might. The AI would evaluate the paper and offer suggestions for improvement, after which I'd gather the professors' reactions to assess the value of this AI-assisted thought partnership.

Today, I'm sharing results from one particularly illuminating case: a paper about economics education titled “Relevance, Belonging, and Growth Mindsets in Economics: Differences across Institution Types.” The paper was co-authored by Caroline Krafft, Kristine West, Allen Bellas (my correspondent), Ming Chien Lo, Adriana Cortes Mendosa, Gabrielle Agbenyiga, J Dombroski, Nayomi Her, and Joy Moua. Allen Bellas and Ming Chien Lo are at Metropolitan State University, St. Paul, MN, while the rest of the co-authors are at St. Catherine University in the same city.

Crucially, I did not turn on Deep Research, which would have connected o1-pro with the internet (notably, Deep Research is now available to all paying ChatGPT users; more on it here, here, and here). Instead, I tested o1-pro’s ability to engage with the paper based on its training data alone.

In short, the authors received extensive, structured feedback from o1-pro that impressed them in multiple ways, as researchers and as teachers of research methods.

What's most striking is how little guidance the AI needed, all due to the power of o1-pro — like other reasoning models — to prompt itself.

Let's look at what happened.

The Paper

The paper I shared with o1-pro examines why economics as a field has struggled with diversity and inclusion compared to STEM disciplines, focusing on psychological aspects (relevance, belonging, and growth mindset — collectively "RBG") across different institutional settings, including minority-serving institutions (MSIs) and women's colleges.

This was a substantive, data-driven paper with a robust methodology section, analysis of survey data from 24 colleges and universities, and thoughtful discussion of implications for economics education. At 15 pages, single-spaced (excluding the 4 pages of references and 15 tables), it represented the kind of complex academic work that typically requires careful review from knowledgeable colleagues.

My Prompt

The prompt I used relied on no field-specific expertise, as I have none (I majored in economics a long time ago but let’s be honest here). Lengthier prompts with more background information, criticisms from knowledgeable colleagues of the paper, etc. would likely get even better results (more on this below). My prompt just conveys the broad context of my request, without any special insight into how to proceed.

Here is my prompt:

The AI Feedback

Given this limited and generic prompt, o1-pro developed its own comprehensive feedback plan with eight distinct parts:

1. High-Level Framing and Organization

2. Literature Review and Theoretical Foundations

3. Data and Sampling Strategy

4. Measurement of Key Constructs (RBG) and Empirical Methods

5. Presentation and Interpretation of Results

6. Discussion, Contributions, and Limitations

7. Writing Style and Readability

8. Future Directions and Broader Implications

For each section, the AI provided detailed, thoughtful analysis that balanced acknowledgment of strengths with suggestions for improvement. The feedback totaled over 10,000 words — longer than many published papers themselves!

Here's a sample from the "High-Level Framing and Organization" section to give you a flavor of the feedback:

The feedback wasn't just superficial; it engaged deeply with the paper's methodology. For instance, on sampling strategy:

o1-pro also went beyond critiquing the paper to offering thoughtful extensions and implications for future research:

The AI consistently balanced affirmation with constructive critique, maintaining a collegial tone throughout. It identified genuine strengths while suggesting concrete improvements — much like an ideal peer reviewer.

The Professors' Response

The professors were genuinely impressed by the quality and thoroughness of the AI feedback. As Bellas wrote:

While they noted that "the length of the feedback was a bit daunting," they appreciated that "the excellent structure made it easier to read through and digest the feedback and to identify specific things to work on."

Bellas reported that their colleagues immediately recognized the educational potential:

Takeaways and Implications

What does this experiment tell us about AI as a thought partner for academic work? Here are a few of my thoughts:

1. Minimal prompting of the reasoning models like o1-pro can yield high-quality results

The most surprising aspect of this experiment was how little guidance the AI needed. I didn't provide detailed instructions about what constitutes "good feedback" or specify a particular structure. As noted, I don’t know anything about the subject matter, so this prompt is about as good as it’s going to get! Instead, I simply described the basic task in general terms and let the AI determine how to approach it. This may mean that for certain professional tasks, sometimes the best prompt is the simplest one for reasoning models like o1-pro — allowing the AI's capabilities to shine through without constraining them with overly specific instructions. (This matches with the advice of the AI developers, mind you.) But it also leaves room for the possibility that more information in the prompt, which could be provided by a subject-matter expert, would get even better results. I’ve found that the latter can be true in some cases where I do have expertise.

2. AI feedback offers unique advantages over traditional peer review

AI feedback offers several distinctive benefits over human peer review and collaboration:

• Exceptional thoroughness by default

• Consistent structure and organization (the use of bullet points is borderline compulsive from these models, I’ve found, but it is useful for this sort of case)

• No personal biases or academic politics (though there may be biases in the training data, of course)

• Immediate availability (no waiting for busy colleagues)

• A balanced approach that acknowledges strengths while suggesting improvements (reasoning models do this almost to a fault)

3. It's particularly valuable for structure and organization

The AI excelled at analyzing the paper's overall structure, flow, and organization — aspects that even experienced academics sometimes struggle to evaluate objectively in their own work. (I’ve definitely had issues with this…) By offering a fresh perspective on how arguments are sequenced and connected, AI feedback can help researchers strengthen the fundamental architecture of their papers.

4. There are still important limitations

Despite its impressive performance, AI thought partnership has clear boundaries. The AI can't evaluate the factual accuracy of novel research findings (without source material being provided to compare), can't bring disciplinary expertise beyond what's in its training data or what’s shared in the prompt, can't fully understand the nuanced dynamics of academic publishing in specific fields, etc.

5. The educational applications are particularly promising

The authors immediately recognized the potential for using AI feedback in their teaching. For students, receiving detailed, non-threatening feedback from an AI might feel less personal and more constructive than instructor comments. The AI can provide a first round of comprehensive feedback, edited by the instructor if needed, allowing instructors to focus on higher-level guidance or areas the AI might have missed. Furthermore, providing this level of feedback from one professor for all 20, 40, 60, or more students would be extremely time consuming.

✨Pedagogical Suggestions

Here are some practical approaches for integrating this technology into your teaching, including some $0 AI models that are comparable to o1-pro (and how to access them):

✨ What Premium Members Get

As I improve ✨Premium subscribers’ benefits, it can be hard to keep track, especially if you’re new to AutomatED. Here’s a summary…

Core Benefits

• $0 monthly webinars on timely AI topics (last one: Feb. 28, on “Using AI to Create Course Content”; next one: on using AI for non-pedagogical productivity, at the end of March), while everyone else pays $25

• Access to all our practical Guides, including topics like AI-aware assessment design and student AI policies, and Tutorials on how to use AI tools, from using ChatGPT for analyzing teaching data to using LearnLM for tutoring

• Discounts on our focused courses (like our new custom GPT course)

• Annual 1-hour consultation to discuss your specific challenges

• Premium-only sections in weekly newsletters, like the above

You can see all the benefits, including all the Guides and Tutorials, here.

Nearly 100 professors and learning specialists already use these resources to save time and enhance their teaching. Want to join them?

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I share all the big news from last week, including the announcement of the new GPT-4.5, Claude 3.7 Sonnet, and the wider availability of OpenAI Deep Research. But first, we have an awesome guest post on several AI-powered voice tools.

📣 By the way, I just spun up a little website to explain the AI tools I’ve developed to help department course schedulers:

PS - There are only 2 more department slots available for this semester.

Your daily AI dose

Mindstream is your one-stop shop for all things AI.

How good are we? Well, we become only the second ever newsletter (after the Hustle) to be acquired by HubSpot. Our small team of writers works hard to put out the most enjoyable and informative newsletter on AI around.

It’s completely free, and you’ll get a bunch of free AI resources when you subscribe.

Click here to subscribe

🎤 What is the Future of Voice in Education?

A Guest Post by Elliot Roe and Duncan Johnson

Voice-based AI tools are becoming more common in education, offering a potential solution to AI misuse. But do they represent a genuine opportunity to improve student outcomes, or are they just another layer in the cat-and-mouse game of academic integrity?

Duncan and I are student researchers at Tufts University and Georgia Tech, respectively, and we've spent the last 4 months researching and designing a voice-based tool for submission to the Learning @ Scale conference. We started exploring voice-based AI learning tools to see if they could help foster student-to-student connections in online classes but have shifted our approach significantly since then based on our exploration. We spoke with online education professionals and tested various tools before building our own voice-based tool and piloting it in classes. Along the way, we've learned a lot — both about how students are actually interacting with the tech and about the realities of integrating voice AI into education.

We'll discuss three existing voice-based AI tools that we found, each using voice AI in a unique way. Then, we’ll provide a quick breakdown of how we approached our own design and share our thoughts on where this technology is heading.

Socratic Mind

Voice-based assessment methods — like oral exams — have become an accepted way of "AI-proofing" assignments. But oral exams tend not to be very scalable, which is exactly what Socratic Mind aims to solve.

Built by researchers at Georgia Tech, Socratic Mind replicates oral exams asynchronously with AI. Here's how it works: Instructors set up assignments by adding questions and writing an "ideal" answer for each. Then, when students take the exam, the AI uses Socratic questioning to dig into their knowledge and guide their thinking.

While it's great to see how effectively Socratic Mind can help students develop their understanding through this back-and-forth, we've heard from students and educators who’ve used the tool that it can balloon the length of assignments.

Sherpa Labs

Where Socratic Mind tackles comprehensive oral exams, Sherpa Labs takes a different approach with quick, 10-minute quizzes on course content. Their main innovation is in content transformation — instructors simply drag and drop a PDF reading and specify a few key areas they want the questions to cover. What makes Sherpa interesting is that it's not trying to replace traditional assessments with oral exams. Instead, they're bringing an extra layer of security to lower-stakes activities.

This seems like it could work well alongside essays or other assignments. But there's a catch: the tool generates a fixed set of questions for each reading, and some educators have noted that students can still attempt to game the system by preparing scripts in advance. It's a trade-off between simplicity and security — you get a streamlined process, but you might not get the full benefits of spontaneous oral assessment.

MirrorTalk by Swivl

MirrorTalk, developed by Swivl, is a voice-AI tool designed to support self-reflection in education. It allows students and teachers to record spoken reflections and receive AI-generated insights.

We include MirrorTalk here because scaffolding reflection with a voice-AI tool is a novel and potentially powerful idea. However, where MirrorTalk falls short is its focus. It’s designed to integrate with Swivl’s co-teacher devices and assist in self-reflection for teachers. While activities can be designed and sent to students as well, it’s clear — from template activities on their platform to their product’s marketing — that student usage is not their first priority. To our knowledge, no independent educators have shared their experiences with the tool beyond Swivl’s promotional materials. So, while it could provide an interesting way for your students to reflect, it doesn’t feel as realistic to do so.

Verse

Learning from these platforms, we saw an opportunity to create something more conversational. We created Verse. While other tools mix text and audio or follow structured question-answer formats, their interactions often feel mechanical and chatbot-like. We wanted to build a purely voice-based experience that feels like a natural discussion, where students can work through course concepts in real-time synthesis.

Assignment Creation View

Student View

This informal approach is intended to let us capture students genuinely grappling with ideas. However, it presents a unique challenge: evaluating free-flowing discussions is more complex than grading structured essays.

To address this, we've incorporated an automated specifications grading system. Instructors define both core discussion questions and specific learning criteria. The system then analyzes each conversation transcript, identifying relevant quotes that demonstrate how students met these criteria. Students can review their evaluation and choose to either refine their ideas through another conversation or submit their transcript directly to their LMS.

Evaluation Output

Product Comparison

What’s Our Take?

Voice-AI in education is still in its early stages. Currently, Socratic Mind shows the most promise with its well-defined use case for oral exams and strong research backing. While Sherpa Labs achieved the highest adoption among these tools, their focus seems to have shifted elsewhere since the tool's creation.

But looking beyond these specific implementations, we see voice-AI's real potential in formative feedback — if we can make it practical for everyday classroom use. This has led us to focus on two key questions:

1. How can we create voice-AI-based activities that build on the best parts of synchronous, teacher-led discussions?

2. How do we provide efficient and equitable feedback to both instructors and students on these voice-based activities?

The answers to these questions will likely determine whether voice-AI becomes a transformative educational tool or just another fancy CAPTCHA in the ever-growing collection of edtech solutions. What do you think?

This section was written by Elliot Roe, who is an Undergraduate Research Assistant at the Georgia Tech Play and Learn Lab, and Duncan Johnson, who is an Undergraduate Researcher with the Tufts University Center for Engineering Education and Outreach.

📢 Quick Hits:
AI News and Links

1. OpenAI announced GPT-4.5, which they describe as their "largest and most knowledgeable model yet." In short, its “innate” strengths are representing empirical facts, avoiding hallucinations, and conversing more naturally, but it is worse at complex reasoning tasks like coding and STEM than their o models (o1, o1-pro, o3-mini, o3-mini-high) and these models’ competitors. Crucially, 4.5 is also worse at representing and finding empirical facts than Deep Research, which is OpenAI’s agentic fusion of their reasoning models with advanced search capabilities (see below for news on it, by the way). So, the sense in which it is strong at representing empirical facts is that, when limited to training alone and not combined with other functions/tools, it does the best. This leaves it in a weird position in our AI toolkit, and it reduces OpenAI to suggesting that its socioemotional intelligence may be the source of its lasting value (and that the benchmarks that they normally tout “don’t always reflect real-world usefulness”):

Many commentators are arguing that the limitations of 4.5 prove the limitations of “unsupervised learning” training methods, although perhaps combining 4.5 with the reasoning models and functions/tools will result in “something far more powerful.” You can access 4.5 via ChatGPT Pro right now or via Plus later this week.

2. In more exciting news, Anthropic released Claude 3.7 Sonnet, their "most intelligent model to date" and the first "hybrid reasoning model" on the market. It can produce either near-instant responses or engage in extended step-by-step thinking that's visible to users. The model shows particular improvements in coding capabilities, with partners like Cursor, Cognition, Vercel, Replit, and Canva confirming its exceptional performance. Many commentators think it is significantly ahead when it comes to real-world development tasks. Anthropic also introduced Claude Code, a "command line tool for agentic coding" in limited research preview, which enables delegating substantial engineering tasks to Claude directly from your terminal.

3. OpenAI expanded access to Deep Research, its web browsing agent that creates comprehensive research reports, to all paying ChatGPT users. Previously only available to $200/month Pro subscribers, the feature is now included with ChatGPT Plus, Team, Enterprise, and Edu plans (limited to 10 queries monthly, while Pro users get 120). This follows Google's similar move last week extending their Deep Research agent to all Gemini Advanced users, highlighting the competition between OpenAI, Google, and Perplexity to deliver value through advanced research capabilities.

4. ICYMI: Google launched “Career Dreamer,” an AI-powered career exploration tool that helps people identify transferable skills and discover new career paths. It uses Gemini to help craft career identity statements, explore job possibilities, and draft application materials. This could be particularly valuable for career services offices and academic advisors working with students on their career pathways. (More info here.)

5. ICYMI: Google also unveiled an "AI co-scientist," a multi-agent system built on Gemini 2.0 that assists researchers in generating novel hypotheses and experimental designs. In real lab experiments, it successfully predicted new drug treatments for leukemia and proposed liver fibrosis targets that showed "significant anti-fibrotic activity." Perhaps most impressively, it independently rediscovered a novel bacterial gene transfer mechanism that had been recently found (but not yet published) by researchers at Imperial College London. Google is opening access to research organizations through a Trusted Tester Program, and you can sign up here.

Google, “Accelerating scientific breakthroughs with an AI co-scientist.”

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I answer 3 reader questions on topics big and small related to tech/AI and pedagogy.

📣 By the way, I just spun up a little website to explain the AI tools I’ve developed to help department course schedulers:

PS - There are only 2 more department slots available for this semester.

There’s a reason 400,000 professionals read this daily.

Join The AI Report, trusted by 400,000+ professionals at Google, Microsoft, and OpenAI. Get daily insights, tools, and strategies to master practical AI skills that drive results.

Sign up now for free and work smarter, not harder.

📢 Quick Hits:
AI News and Links

1. Google launched “Career Dreamer,” an AI-powered career exploration tool that helps people identify transferable skills and discover new career paths. It uses Gemini to help craft career identity statements, explore job possibilities, and draft application materials. This could be particularly valuable for career services offices and academic advisors working with students on their career pathways. (More info here.)

2. Google also unveiled an "AI co-scientist," a multi-agent system built on Gemini 2.0 that assists researchers in generating novel hypotheses and experimental designs. In real lab experiments, it successfully predicted new drug treatments for leukemia and proposed liver fibrosis targets that showed "significant anti-fibrotic activity." Perhaps most impressively, it independently rediscovered a novel bacterial gene transfer mechanism that had been recently found (but not yet published) by researchers at Imperial College London. Google is opening access to research organizations through a Trusted Tester Program, and you can sign up here.

Google, “Accelerating scientific breakthroughs with an AI co-scientist.”

3. In a piece parallel to mine from a few months ago on AI and prestige, Hollis Robbins argues that “it’s later than you think,” in the sense that universities must hurry to radically reimagine their value proposition as AGI emerges: "If universities cannot articulate in detail how their faculty exceeds AGI capabilities, what value are they offering to tuition-paying students?" She predicts universities will retain only faculty who either advance original research beyond AGI capabilities, teach sophisticated physical skills, or work with previously undiscovered materials, and she argues that the other factors that make higher ed valuable, like social skill development, aren’t sufficient to justify the existence of higher ed.

4. OpenAI released a major update to their Model Spec, the document defining how their AI models should behave.

❓ My Answers to 3 Reader Questions

The mailbag returns! Here are a few of the inquiries I’ve received in the last two weeks from subscribers and clients, as well as my answers.

Remember that you can share your questions and comments by responding to any of my emails — responses to any AutomatED email come straight to my inbox — or by submitting a comment on one of my polls.

✨1. “What do you think of using NotebookLM Plus as an AI tutor, rather than a custom GPT?”

The person asking this question noted that NotebookLM seems to offer better control than a custom GPT over the knowledge base that the LLM relies on in responding to user queries.

As I detailed last week, NotebookLM Plus's ability to share notebooks in "chat-only" mode while enforcing "Guide" style interactions is compelling for tutoring use cases. Students can engage with your curated course materials through AI-guided discussion, with the system staying reliably grounded in those sources.

And yes, in my testing, NotebookLM's current technology does a better job staying focused on its knowledge base compared to custom GPTs' deployment of retrieval-augmented generation (RAG); the latter requires significant prompt engineering and testing to get good performance, partly because it seems that OpenAI hasn’t improved custom GPTs in this dimension in quite some time.

However, I would recommend to not write off custom GPTs just yet, whether for tutoring or otherwise. Major updates are coming in the next few months from OpenAI that could change this calculus dramatically.

Let me explain, starting with a key piece of information recently shared by OpenAI.

2. "Did you get Deep Research for free as a demo?"

A reader asked this after my enthusiastic review of OpenAI's new research tool. She was (jokingly?) skeptical that anyone would be this positive about a $200/month service without some... incentives.

Fair enough! That $200 monthly charge is painfully real!

However, as I told her via email, despite writing about AI tools for years, I've never received a free account or demo from any AI company. (OpenAI, Google, if you're reading this... 👀)

But her skepticism highlights something important: I only praise tools I've thoroughly tested with my own money and time. I spend 4-10 hours a day working with AI tools, so it takes a lot to get me pumped. If I am enthusiastic, it’s because I really believe what I am saying.

Deep Research actually impresses me a lot, citation quirks and all. And, by summer, those citation issues I noted will likely be sorted out, with even more features added as OpenAI works to stay ahead of Google.

3. "What's the most common tool that your consultation clients ask for help with?"

I got this question in a recent 30-minute exploratory meeting with a curious subscriber.

My answer won’t come as a surprise.

As I develop more and more custom AI tools for clients (including my department course scheduler), I find that teaching professors benefit the most from tools that help them with their feedback processes.

There are two reasons for this.

1. By and large, professors have always disliked (hated?) grading and feedback tasks, unless they are working one-on-one with a student in real-time. These tasks are time-consuming and draining because they are cognitively demanding and it often feels like you’re sending your feedback out into the abyss.

(Side note: this is one reason why I love doing and advocate for oral exams with live feedback, when feasible.)

2. LLMs are hard to efficiently and ethically use out-of-the-box for grading and feedback.

The solutions I build vary by context, but they often involve creating workflows that let professors focus on what they can uniquely provide — expertise-dependent evaluation —while AI handles everything else about generating their feedback. This avoids data privacy issues as well.

I have one client for whom I’m developing a series of tools to help with two different assignment types. For one assignment type, the professor used to spend 30 minutes per student submission, including time inputting everything into their LMS. They now spend 10 minutes per submission. With 40 students per semester and 5 such assignments, that’s 4,000 minutes saved per semester. I charged them $1250 for this tool, which amounts to ~$18.75 per hour saved. Yet, since they plan to use it for 3+ semesters with the same teaching load, that rate goes to $6.25 per hour at most. As they said to me, their time is worth a lot more than $6.25 per hour (indeed, it’s worth a lot more than $18.75!).

This brings up a longer discussion, but a crucial fact about this client is that they are paying for my services out of pocket. This isn’t uncommon either — most faculty don’t have funds from their universities for stuff like this. However, it’s my view that their institutions should be developing scalable solutions like this for them, not expecting them to either spend the time or the money themselves.

📝 Save the Date: ($0) Webinar This Friday

Want to learn more about on using AI to generate course content?

If so, just answer “Yes” in the below poll and you are thereby registered for my February Zoom webinar.

It will be this Friday, February 28th from noon to 1:30pm Eastern.

All registrants will receive the recording afterwards, even if they don’t attend live.

The rest of my webinars this semester will have an entrance fee, so this is your chance to attend without spending a dime.

If you select “Yes”, that’s enough to register. You’ll get a follow-up email this afternoon with more information, including the Zoom link.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I take a look at NotebookLM — Google’s way of bringing documents and other source materials to life via summaries, audio overviews (podcasts) and chat experiences — and its new premium version (“Plus”).

📣 By the way, feel free to email me if you handle your department’s faculty-to-course or course-to-room scheduling and you’re interested in the AI tools I’ve developed to help improve your processes — saving you time and producing more optimal schedules, without requiring any new inputs from your colleagues. I’ve got only 2 more slots available for this semester.

The future of presentations, powered by AI

Gamma’s AI creates beautiful presentations, websites, and more. No design or coding skills required. Try it free today.

📝 Save the Date: ($0) Webinar Next Friday

Want to learn more about on using AI to generate course content?

If so, just answer “Yes” in the below poll and you are thereby registered for my February Zoom webinar. It will be on Friday, February 28th from noon to 1:30pm Eastern. All registrants will receive the recording afterwards, even if they don’t attend live.

The rest of my webinars this semester will have an entrance fee, so this is your chance to get in on the action without cost.

If you select “Yes”, that’s enough to register. You’ll get a follow-up email on Monday of next week with more information.

🧰 An AI Tool For Your Toolbox: 
NotebookLM

Another month, another AI subscription to consider. This one, though, might be available to you for free.

Google One AI Premium ($20/month after the first two months) now includes NotebookLM Plus, an enhanced version of their document-focused AI assistant.

However, Plus now also comes as a part of Google Workspace and Google Cloud for businesses, schools, universities, and other organizations.

But price and availability aside, is Plus worth adding to our growing stack of AI tools?

The answer depends on how you work with course materials and research documents, and it depends on how you want to facilitate your students doing so, but I think it’s probably “Yes” for many of us.

While the free version of NotebookLM can already analyze documents and generate summaries and podcasts, the Plus tier adds features that might interest higher educators — particularly those who want to create collaborative AI workspaces around course content.

At its core, NotebookLM functions like an AI research assistant that thoroughly reads whatever sources you provide, from lecture slides and academic papers to YouTube videos and web articles. It differs from general-purpose AI chat tools by staying more strictly grounded in your uploaded sources, with inline citations for every claim.

The Plus version adds the ability to customize the AI's interaction style and, perhaps most interestingly, lets you share focused versions of your notebooks where others can interact with your curated sources through AI-powered chat.

How It Works

After creating a notebook in NotebookLM, you start by uploading your sources. The platform accepts a wide range: PDFs, Google Docs, slide decks, web pages, YouTube videos, or copied text.

Each notebook can hold up to 300 sources in Plus — a significant increase from the free version's 50-source limit.

Once your sources are loaded, NotebookLM generates an automatic summary (a “Briefing Doc”) and can quickly produce an Audio Overview of the sources, and you can chat with any subset of the sources in the middle window.

But the real power comes from the "Studio" panel on the right side of your screen. Here you can:

• Create Audio Overviews (i.e. podcasts) that convert your sources into engaging dialogues between AI hosts, and there’s a beta feature to join in with the speakers

• Generate study guides, FAQs, timelines, or briefing documents

• Take AI-assisted notes that link directly back to source material

Beyond enabling more Audio Overviews, notebooks, queries, and sources, the Plus version has three distinctive features:

1. You can share notebooks in two ways:

   1. full access, where collaborators can see all your sources and add their own

   2. chat-only access, where others can interact with your sources without seeing the raw documents (more info here, including sharing limits based on account type)

2. You can customize how the AI engages with your content through new "chat styles", like "Guide” or “Analyst”, and output length options, like “Longer” or “Shorter” (note: the button to changer these settings is easy to miss, but it’s the triple-slider-bar icon, which is in line with the “Chat” and “Studio” headers)

   • This means you can combine the prior feature, where you share a notebook with chat-only access, with the “Guide” setting to build something akin to a tutor for your students or colleagues (more on this below)

3. You can also view data on user access to your shared notebooks, including how many of them have used it and the queries they’ve made

How We Can Use It

For higher educators, NotebookLM Plus opens up several possibilities that go beyond basic document analysis.

First, it's a powerful tool for course preparation.

You could upload all your lecture slides, reading materials, past student feedback, etc. into a single notebook. The AI can help identify themes across materials, suggest ways to better connect topics, and generate review materials that accurately reflect your specific content. Since everything stays grounded in your sources — or, at least, the answers of NotebookLM require a lot less prompt engineering to remain well-grounded, and you can check the “citations” — you're getting insights based on your actual course materials.

Second, and this is where the Plus features become interesting, it can help create guided learning environments, as I foreshadowed above. For example:

• Build a notebook with supplementary readings and share it in "chat-only" mode, letting students explore complex topics through AI-guided discussion without getting overwhelmed by raw source material

• Create topic-specific study guides where the AI acts as a "Guide" (using Plus's chat styles) to help students understand difficult concepts, always citing specific passages from your course materials

• Use the audio overview feature to generate accessible versions of dense readings, which is particularly helpful for students who learn better through listening

Given that I suspect many students will already be using Plus on their own — whether you like it or not — it’s a serious question whether you should try to get ahead of them and provide more structure or just your own flavor to their studying experience.

The analytics feature in Plus also lets you see how students are actually using these resources, including which topics they're asking about most frequently and how deeply they're engaging with the material. This gives you formative information that can guide your in-class activities and topic coverage.

My Take

If you already use Google Workspace through your institution, you should definitely check out NotebookLM Plus since it’s included in your account.

If you don’t, Plus may still be worth exploring if you frequently work with large collections of course documents, have a source-focused course, or can envision using its sharing features. They might justify the cost.

Still, the free version offers solid functionality and you shouldn’t feel pressured to pay.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I discuss the second “Deep Research” AI tool released in the past few months. This time around, the tool comes from OpenAI, competing with Google’s version, and it has many academics impressed and worried. I explain how it works and share one of its outputs below so you can see for yourself what it can do.

📣 By the way, feel free to email me if you handle your department’s faculty-to-course or course-to-room scheduling and you’re interested in the AI tools I’ve developed to help improve your processes — saving you time and producing more optimal schedules, without requiring any new inputs from your colleagues. I’ve only got 2-3 more slots available, as a range of departments and other academic units around the world are going to be using my service this semester.

Keep your meeting data safe

Worried about all the AI bots joining your meetings?

Fellow is the only AI meeting assistant built with security controls and permissions.

Thousands of organizations rely on Fellow to capture their meeting transcripts, summaries, and action items – all in one secure platform.

Stop sharing your meeting data with AI note-takers that lack proper security.

Claim 30 days of unlimited AI notes today.

Get started free

🧰 An AI Tool For Your Toolbox: 
OpenAI's Deep Research

Should you get ChatGPT Pro for $200/month?

A month ago, my answer would have been "probably not."

o1-pro, the reasoning AI model that was the main benefit of Pro, truly is amazing. In fact, I’ll show off some of its powers next week when I discuss its viability as a thought partner to cutting-edge researchers.

But other tools are enough for most use cases, from the freely available Gemini models in Google AI Studio to the custom GPTs you can build with ChatGPT Plus for $20/month.

And this is all the more true now that everyone can freely use o3-mini, which goes toe to toe with o1-pro, via ChatGPT.

However, OpenAI's new Deep Research feature, launched last week, changes the equation and should tempt all of us to find the funds to get access to ChatGPT Pro.

It's the first AI tool I've seen that can immediately assist with academic research — both your own and your students' — without the user having both field-specific knowledge AND prompting skills.

OpenAI’s Deep Research is more than just ChatGPT with web access (that’s ChatGPT Search). It's a specialized research agent that spends 5-30 minutes autonomously exploring academic sources, analyzing findings, and producing detailed reports with proper citations.

And the crucial thing is this: it’s really good at playing this role, out of the box.

Unlike Google's similar tool, which aggregates a wide range of Internet sources quickly, OpenAI's version performs something closer to genuine scholarly synthesis: it identifies patterns across sources, draws meaningful connections, and structures findings based on your specific interests and parameters. (Google’s tool is limited to a narrow range of output formats.)

And you can guide OpenAI’s research process through back-and-forth dialogue, much like working with a research assistant, while Google’s tool is somewhat inflexible in my experience.

One month ago, I declared, after testing Google’s tool, that "the research report is dead and we have killed it” and that Google’s tool was to the research report what ChatGPT was to the essay. I was very impressed, despite the flaws of Google’s Deep Research.

Only one month later, I’m blown away by how much better OpenAI’s tool is, although I’m sure Google will provide good competition throughout this year and beyond.

How It Works

To access it after purchasing ChatGPT Pro, you select "Deep Research" from button below the chat field.

The Deep Research button.

You can pair it with different models via the model selector in the upper left corner of your screen, but its best performance comes with smarter reasoning models like o3-mini-high or o1-pro.

In terms of prompting, no special prompt engineering skills are needed.

It is key to be detailed about what you want — the format, scope, other relevant information, and type of sources to focus on — because it performs best when you provide a ton of context and background. Don’t be afraid to dump more information in!

Deep Research then plans its approach, clarifies what you want (if necessary), systematically explores sources while you watch (if you want), and delivers a comprehensive report with clear citations/links.

Deep Research while conducting research.

You can further refine the output through follow-up questions or by providing additional context.

Some back-and-forth where I clarified my needs (I asked it to help improve my syllabus readings).

How We Can Use It

For professors and learning specialists, Deep Research opens up two key possibilities:

First, it's a powerful tool for your own research and course development.

Whether you're designing a new syllabus, keeping up with developments in your field, or preparing lecture materials, Deep Research can accelerate the initial stages of academic investigation. Unlike standard ChatGPT, it provides verifiable citations and draws from current academic sources, so long as they are available in some form on the Internet.

A task that might have taken hours of preliminary research — like finding alternative readings for a course unit or surveying recent developments in your field — can be completed in minutes, letting you focus on the deeper analysis and synthesis that requires your expertise.

Second, and perhaps more importantly, Deep Research changes how we need to think about teaching research skills.

Just as ChatGPT transformed writing pedagogy, this tool requires us to rethink how we teach students to conduct research. The ability to quickly compile and cite sources is no longer a differentiator — instead, we need to focus on teaching students how to:

• Frame sophisticated research questions

• Evaluate and critique AI-generated research syntheses

• Identify what Deep Research misses or misunderstands

• Use AI as one tool in a broader research workflow

But I will go more into pedagogical implications at a later date. For now, my focus is on showing you what the tool can do.

Let me demonstrate Deep Research's capabilities with a real example from my own course development...

✨ Demonstration & Discussion

To test Deep Research's capabilities in course development, I asked it to analyze Module 2 of my "AI and the Future of Humanity" syllabus. The philosophy course, which I taught several times at UNC-CH, covers the ethical, political, practical, metaphysical, and epistemological challenges and opportunities presented by AI. (The screenshots above are from this request.)

Specifically, I wanted it to find and review nine key readings — from Sven Nyholm on self-driving cars and Regina Rini on deepfakes to Natasha Singer on AI tutors and Philippe Van Parijs on universal basic income — and suggest potential improvements to my syllabus that would better serve the course objectives. I only provided it with a prompt and a copy-paste of my syllabus, with each reading identified only by the author’s last name and a topic under which it fell.

Ideally, I would have provided more information, both about my readings and about ways that I already know my course could be improved, but it still performed amazingly, all things considered.

Deep Research figured out which journal article or think piece I referred to (from context plus its searches) and proposed a really impressive set of alternative readings.

Here’s the prompt I used, followed by a link to the results and my discussion of the upshots of this new tool:

After reviewing, analyzing, and synthesizing a report on 19 sources over 7 minutes, Deep Research produced this report that I share via the “share conversation” link (so you can see the full context):

📢 Quick Hits:
AI News and Links

1. Google has outlined eleven ways educators can use its AI tools in 2025, including:

• NotebookLM for creating lesson plans and study guides with source citations

• Gemini integration into learning management systems like Canvas

• NotebookLM Plus which allows professors to create shared notebooks containing course materials.

2. ICYMI (“in case you missed it”): OpenAI has announced o3-mini, a cost-efficient version of its reasoning model that matches OpenAI o1's performance in STEM fields while being 24% faster. The model is available through both ChatGPT and API, with free users getting limited access for the first time (just select “Reason” in the message composer). Notable features include three "reasoning effort" levels, function calling capabilities, and integration with web search, though it lacks vision capabilities. Their showed users preferred o3-mini's responses to o1-mini 56% of the time, with 39% fewer major errors. I’ve found it is extremely reliable and powerful for a range of my AI tool development use cases.

3. Here’s a deep dive into how LLMs were developed, how to conceptualize them, and how to use them, from Andrej Karpathy (an OpenAI co-founder).

4. Statistician Nate Silver argues that AI will be a major political issue after being largely absent from the 2024 election. He criticizes the political left for dismissively comparing AI to crypto, arguing they risk being "dealt out of the hand" on crucial policy decisions. Silver suggests AI will rank at least "a high 7 or low 8" on his "Technological Richter Scale" of societal disruption, even without achieving superintelligence, and estimates a 5-10% chance of existential risk aligned with expert consensus.

5. Google has made Gemini 2.0 generally available, releasing three versions: Flash (their workhorse model with 1M token context), Pro (their most capable model with 2M token context, especially for coding), and Flash-Lite (a more cost-efficient option). While all versions currently support multimodal input with text output, Google says additional output modalities like image generation and text-to-speech are coming soon. (Try them in their raw forms in Google AI Studio here; Tutorials on using this tool here and here).

6. OpenAI has launched "ChatGPT for Education,” a new newsletter run by their education team. It will start with profiling 30 educators who are effectively using AI in teaching.

📝 Register for February’s Webinar

Want to learn more about on using AI to generate course content?

If so, just answer “Yes” in the below poll and you are thereby registered for my February Zoom webinar. I think it will be on February 28th from noon to 1:30pm Eastern, although this isn’t set in stone…

If you select “Yes.”, that’s enough to register. You’ll get a follow-up email in a few weeks with more information.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I discuss two ways in which AI tools are getting more powerful, as well as what they mean for educators who need to assess students. There are still many options available to instructors but using version history tracking soon won’t be one of them…

Last Friday, I hosted January’s AutomatED webinar on “Building Better AI Tutors.” It went great! Thanks again to all who registered.

February’s webinar won’t have any entrance fee — the only such webinar of this semester — and will cover creating course content with AI. All it takes to sign up: a “Yes” response to the poll at the bottom of today’s newsletter.

Learn AI in 5 minutes a day

This is the easiest way for a busy person wanting to learn AI in as little time as possible:

1. Sign up for The Rundown AI newsletter

2. They send you 5-minute email updates on the latest AI news and how to use it

3. You learn how to become 2x more productive by leveraging AI

Sign up to start learning.

🤖 The Robots are Gaining Power

Last summer, I updated my Premium ✨Guide for preventing and discouraging student misuse of AI. I outlined six broad strategies for professors to prevent and discourage AI misuse in academic settings:

1. motivate students to use AI ethically and for learning through relationship-building and incentive-aligned course design

2. require students to complete work in AI-free zones, like with in-class assessments or with proctoring

3. design AI-immune assignments that students complete with access to AI

   • either increase AI immunity with format requirements (like an essay that is completed with version history tracking turned on in a Google Doc)

   • or increase AI-immunity with content requirements that focus on what AI struggles with (like meeting technical field-specific standards that the professor has determined to be challenging for AI to meet)

4. pair AI-vulnerable assignments with AI-immune verification tasks that check genuine understanding, like an oral exam completed without AI paired with a take-home essay (that they could have used AI to complete)

5. combine these approaches across one’s course (note: this is my recommended solution)

6. do nothing in cases where other pedagogical priorities take precedence or where the instructor simply needs to operate as if the students are working in good faith

Many of these strategies remain live options, but there are two growing challenges…

First, AI’s “intelligence” continues to increase, with a stepwise improvement coming with powerful reasoning models like o1, o1-pro, o3-mini (now available to ChatGPT Plus, Team, and Pro users, with free plan users able to “try OpenAI o3-mini by selecting ‘Reason’ in the message composer or by regenerating a response”), Gemini 2.0 Flash Thinking (available for free in Google’s AI Studio), and DeepSeek’s open-source R1 (also available for free from Microsoft in Azure). These AI tools narrow the range of assignments that AI cannot complete satisfactorily, and their high intelligence is widely available.

Second, AI’s ability to manipulate our environments continues to increase, with OpenAI recently releasing “Operator,” an AI system that can control computers just like a human would, to Pro users in the US. Anthropic previously released Claude’s “computer use” functionality in beta form (video demo here). Both can navigate websites, type in text fields, click buttons, and interact with computer interfaces in ways that are nearly indistinguishable from human behavior. (A deep dive on how Operator works is here.)

To illustrate the challenge to some AI immunity strategies, here’s a video I made of Operator writing a paper with the resultant version history (note: the video is sped up 3x to save you some time):

OpenAI Operator vs Google Docs Version History

A video showing — at 3x speed — how Operator can write essays in your students’ Google Docs.

vimeo.com/1052822032/da926eab76

The resultant version history looks fairly authentic with little effort. Errors and misteps abound, but this is still a sign of things to come.

(LATE EDIT: And then, yesterday, OpenAI also released its own “deep research” tool to Pro users, which is an “agent that uses reasoning to synthesize large amounts of online information and complete multi-step research tasks for you,” just like Google’s Deep Research but focused on tougher tasks. More on it next week...)

So, what do these developments mean for AI immunity for those of us looking for ways to get our students to complete some assignments or tasks without relying on AI in a problematic way?

Below, I explain where things stand.

(Of course, this is not to assume that there are no cases where AI should be used, but it is to recognize that there are reasonable pedagogical views where AI use needs to be curtailed, including for the development of fundamental skills like critical thinking or judgment.)

✨ What Instructors Can Do

One of my recommendations in the aforementioned Guide was to use Google Docs version history or similar tracking tools in Microsoft Word to detect AI misuse. The idea was simple — AI-generated content would typically appear in large blocks of pasted text, while genuine student work would show a more natural writing process with gradual development, revisions, and refinements.

Version history could reveal these telltale patterns, helping professors identify potential AI misuse and enabling them to create rubrics students to use the “correct” AI-independent process. (Sure, a student could type in something they produced separately with AI, pretending to write it themselves, but this would be annoying enough that many would be incentivized to not do it.)

This was part of a more general shift toward what some call "process verification" rather than "output verification." Instead of trying to detect AI use through the final product (which AI can increasingly simulate), we should focus on verifying the student's engagement with the learning process itself.

But tools like Operator fundamentally change this dynamic. They can simulate the natural writing process with remarkable fidelity by typing gradually, making occasional typos and corrections, taking breaks between sections, and creating version histories that look entirely organic. The "fingerprint" of AI use that we once were capable of relying upon is becoming increasingly difficult to detect.

What about the more powerful reasoning from o1 to R1? In short, as noted above, they reduce the domain of AI-immune content — content that AI struggles to produce. While there are some strategies one might want to try to carve out a space where they fail, but other moves are surer bets.

Let me explain.

📢 Quick Hits:
AI News and Links

1. ICYMI (“in case you missed it”): A new study in BJET by Yizhou Fan et al. reveals concerning findings about AI use in education: while students using ChatGPT showed better immediate performance on writing tasks, they exhibited "metacognitive laziness" — reduced self-regulated learning and deeper engagement with the material. The researchers found no difference in motivation between students using ChatGPT versus other tools, but those using AI were more likely to become dependent on it without developing transferable knowledge or skills.

2. The U.S. Copyright Office released Part 2 of its AI report on AI and copyright, concluding that:

• Copyright requires human authorship and cannot protect purely AI-generated content

• AI used as an assistive tool does not affect copyrightability

• Current technology's prompts alone don't provide sufficient creative control for copyright protection

• Copyright can cover human-authored elements in AI outputs and creative modifications/arrangements of AI-generated content

The Office found no need for new legislation or sui generis rights for AI-generated works, noting that providing such protection could actually discourage human creativity by flooding markets with AI content. The report represents the first comprehensive U.S. government guidance on AI and copyright.

3. The Allen Institute for AI announced Tülu 3 405B, an open-source AI model claiming to outperform DeepSeek V3 and GPT-4o on several benchmarks (note: these aren’t the reasoning models mentioned elsewhere in this piece).

4. A Times report reveals OpenAI is investigating whether Chinese startup DeepSeek violated its terms of service by using "distillation" — training new AI systems using data generated by OpenAI's models. The allegations come after DeepSeek's recent breakthroughs challenged assumptions about required computing resources. OpenAI's spokesperson said they're working with the U.S. government on protecting advanced models, while noting Chinese groups are actively working to replicate U.S. AI capabilities.

5. US President Trump announced a $500 billion "Stargate Project" to boost U.S. AI infrastructure, with SoftBank, OpenAI, and Oracle as key partners. Initial funding of $100 billion will expand to $500 billion over four years. As part of the deal, Microsoft's exclusive cloud arrangement with OpenAI will become a "right of first refusal" agreement. Trump also revoked Biden's 2023 AI risk reduction executive order. Fortune reports that Wedbush analysts suggest this could trigger up to $1 trillion in additional U.S. AI investments.

✨ What Premium Members Get

As I improve ✨Premium subscribers’ benefits, it can be hard to keep track, especially if you’re new to AutomatED. Here’s a summary…

Core Benefits

• $0 monthly webinars on timely AI topics (last one: this past Friday, on “Building Better AI Tutors”; next one: on generating course content with AI, at the end of February), while everyone else pays $25

• Access to all our practical Guides, including topics like AI-aware assessment design and student AI policies, and Tutorials on how to use AI tools, from using ChatGPT for analyzing teaching data to using LearnLM for tutoring

• Discounts on our focused courses (like our new custom GPT course)

• Annual 1-hour consultation to discuss your specific challenges

• Premium-only sections in weekly newsletters

You can see all the benefits, including all the Guides and Tutorials, here.

Nearly 100 professors and learning specialists already use these resources to save time and enhance their teaching. Want to join them?

📝 Register for February’s Webinar

Want to learn more about on using AI to generate course content?

If so, just answer “Yes” in the below poll and you are thereby registered for my February Zoom webinar. I think it will be on February 28th from noon to 1:30pm Eastern, although this isn’t set in stone…

If you select “Yes.”, that’s enough to register. You’ll get a follow-up email in a few weeks with more information.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I discuss some striking findings from the Digital Education Council’s just-released Global AI Faculty Survey. If you have any influence over your institution’s AI literacy trainings and resources, you’ll want to pay attention to this…

Remember, too, that my first AutomatED webinar of 2025 is this Friday from 12-1:30pm Eastern on Zoom. I'll dive deep into creating AI tutors using various approaches including custom GPTs, Google Gems, LearnLM, and APIs. You can register here for $25, which includes a discount code for our custom GPT course (free for ✨Premium subscribers; discount code here if logged in). While basic familiarity with prompting and custom GPTs is recommended (and available through the course), this webinar will focus on advanced implementation strategies and common pitfalls in AI tutoring. The session will be 65% presentation and 35% discussion, and the recording will be available for those who cannot attend live.

Save 1 hour every day with Fyxer AI

• Organizes emails so important ones are read first.

• Drafts replies in your tone of voice.

• Takes notes, writes summaries, drafts follow-up emails.

Free 7 Day Trial

💭 What Do Faculty Think About AI?

The Digital Education Council just released their Global AI Faculty Survey of 1,681 faculty members from 52 institutions across 28 countries, and the findings are eye-opening. (Click here if you missed their analogous survey of students.)

While 86% of faculty see themselves using AI in their future teaching [p. 21], only 6% strongly agree that their institutions have provided sufficient resources to develop their AI literacy [p. 35].

This is a concerning gap between the recognized power of AI and institutional support, and it's a clear signal about where higher education needs to focus in 2025.

Speaking with faculty about AI around the world, I've seen this firsthand. But let's dig into the survey’s findings.

The Regional Picture

Before getting into individual issues, I found interesting some of the regional variations on broad stances on AI. For instance, faculty viewing AI as an opportunity vs. challenge varies significantly by region [p. 13]:

• Latin America: 78% opportunity / 22% challenge

• Asia-Pacific: 70% / 30%

• Europe/Middle East/Africa: 65% / 35%

• USA & Canada: 57% / 43%

Likewise, interest in using AI in teaching in the future varied along parallel lines:

Page 21 of the Digital Education Council Global AI Faculty Survey 2025

Faculty Tool Use Outstrips Support

Setting aside regional variations, there’s a noticable gap between faculty using AI tools and using them regularly with expertise:

• 61% of faculty have already used AI in teaching [p. 6]

• 75% of AI-using faculty use it to create teaching materials [p. 7] (see here and here for some of my content on this topic)

• 50% of AI-using faculty use it for teaching students to use and evaluate AI in class [p. 7] (see here for one of my pieces on that topic)

• Yet 88% of these users report only "minimal to moderate" use [p. 8]

• And 40% of faculty identify as complete beginners or have “no understanding” of AI, and only 17% consider themselves "advanced" or "expert" in AI proficiency [p. 16]

Why the gap? Well, one explanation is that faculty lack institutional support.

The survey reveals that…

• 80% of faculty don't find their institutional AI guidelines comprehensive [p. 32]

• 80% say their institutions haven't made clear how AI can be used in teaching [p. 33]

• The top barrier to AI adoption, at 40%? "I don't have time or resources to explore AI" [p. 9]

• The second-highest barrier, at 38%? “I am not sure how to use AI in my teaching” [p. 9]

Faculty’s Concerns About Students

Looking at what keeps faculty up at night when it comes to students using AI…

• 66% agree that incorporating AI is necessary to prepare students for future job markets [p. 22]

• 83% are concerned about students' ability to critically evaluate AI outputs [p. 29]

• 82% worry about students becoming too reliant on AI [p. 30]

• 54% believe current student evaluation methods need "significant changes" [p. 23]

• 50% say they'll need to redesign assignments to be more "AI resistant" [p. 24]

On the latter two topics, see my ✨Premium Guides on how to design assignments and assessments in the age of AI and how professors can discourage and prevent AI misuse.

The Tools & Resources Faculty Actually Want

Here's where it gets particularly interesting. The survey reveals what faculty believe will enable them to integrate AI effectively [p. 36]:

• 65% want better access to AI tools and resources. But it's not just about having the tools — it's about knowing how to use them effectively.

• 64% are calling for training in AI literacy and skills. This matches what I've been hearing in my webinars and faculty workshops — there's a hunger for practical, hands-on guidance.

• 60% want a collection of best practices and use cases.

• 50% need clear guidelines on AI in teaching.

• 31% want environments that encourage innovation and tolerate failure in AI use.

Page 36 of the Digital Education Council Global AI Faculty Survey 2025

Again, while 86% see themselves using AI in future teaching [p. 21], a mere 6% feel their institutions are fully supporting their AI literacy development [p. 35].

It's a gap that needs urgent attention.

What Do You Think?

The survey shows that, in general, faculty worldwide aren't resistant to AI — quite the opposite, especially beyond the USA and Canada. They're eager to embrace it but need proper support.

If you’re here reading AutomatED, you’re probably in the same boat. But this survey makes me wonder: how can I provide you the most valuable resources?

I've created a brief 5-minute survey to understand what you need most, if you have a few minutes to complete it. It’s been a while since I’ve run one of these and would like to get some more insight into what you, my subscribers, want to read or watch.

Click here to share your views.

I will share the responses in aggregate form later on, so stay tuned for that.

📢 Quick Hits:
AI News and Links

1. A new podcast episode from Ed-Technical reveals details about Google's "LearnLM," a family of large language models specifically designed for education. These models are fine-tuned with pedagogical behaviors that can't be achieved through prompt engineering alone. The episode features Google DeepMind researchers explaining their multidisciplinary approach. (Click here for my ✨Premium Tutorial on LearnLM, in case you missed it this past weekend. The first use case is before the paywall.)

2. A New York Times report reveals how Chinese AI startup DeepSeek built a competitive AI model (DeepSeek-V3) using only 2,000 specialized Nvidia chips — far fewer than the 16,000+ typically used by U.S. companies like Google and OpenAI. (Next week, I will discuss reasoning models and the more ed-focused impacts of “agentic” applications of them.) The achievement supposedly cost just $6 million in computing power compared to Meta's hundreds of millions developing Llama and highlights China's growing influence in open-source AI development.

3. A VentureBeat analysis provides deeper context on the performance of the soon-to-be released o3 model (from OpenAI) on the ARC-AGI benchmark, which is intended to test an AI system’s ability to adapt to novel tasks and demonstrate fluid intelligence (hence ‘AGI’). While o3 achieved an unprecedented 87.5% score with high compute (75.7% under standard conditions), experts emphasize this doesn't equate to AGI. The article reveals the high computational cost — $17-20 per puzzle at standard compute, and 172 times more for high-compute performance.

4. Speaking of which, a new 3,000-question test developed by the Center for AI Safety and Scale AI — “Humanity’s Last Exam” — is intended to measure advanced AI capabilities. Questions were sourced from experts who were paid up to $5,000 per accepted submission. Current leading AI models performed poorly — OpenAI's o1 scored highest at just 8.3% — but researchers expect scores to surpass 50% by year's end.

5. A new study in BJET by Yizhou Fan et al. reveals concerning findings about AI use in education: while students using ChatGPT showed better immediate performance on writing tasks, they exhibited "metacognitive laziness" — reduced self-regulated learning and deeper engagement with the material. The researchers found no difference in motivation between students using ChatGPT versus other tools, but those using AI were more likely to become dependent on it without developing transferable knowledge or skills.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

Each week, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

In this ✨Premium edition, I present a Tutorial explaining 4 ways to use Google’s LearnLM (a version of Gemini fine-tuned for learning). I cover how to use LearnLM

• to create sophisticated assessments that promote learning

• to develop clearer and more effective assignment instructions

• to provide more constructive feedback on student work, and

• to support student learning through guided tutoring

🧪👀 Why You Should Try LearnLM

Today's generative AI models often struggle to truly teach. They excel at presenting information but fall short of guiding students through the learning process. Over the past few years, many studies have found that this is the primary problem with students having access to these tools (see, e.g., here, here, or here), as anyone in the classroom already noticed. Effective teaching requires more than just providing answers; it requires understanding how people learn and adapting to individual student needs.

Google's LearnLM, a version of Gemini fine-tuned for learning, offers a promising solution. Unlike general-purpose models, LearnLM is designed to understand and apply pedagogical principles. However, it doesn’t assume that its creator’s theory of learning is correct; it can adapt its teaching approach based on specific instructions, a method Google calls "pedagogical instruction following."

For example, you can instruct it to use Socratic questioning or provide feedback that encourages metacognition. This flexibility allows it to support a wide range of teaching styles and learning contexts.

Early testing from Google and their collaborators suggests LearnLM outperforms other AI models in areas like inspiring active learning, managing cognitive load, adapting to learner needs, and stimulating curiosity. (You can read the lengthy white paper from the LearnLM team here.)

So, LearnLM is worth exploring as a tool to help (higher) educators, especially since it’s free to use (more on access below).

There is one catch: while LearnLM is a version of Gemini — most notable because of its massive context window — LearnLM is limited to 32,000 input tokens, not 2 million like most other versions of Gemini. (See the glossary at the end of this piece for explanations of terms like these.) I expect this to change over time, but for now it is an annoying aspect of LearnLM.

Still, LearnLM is very useful for educators for the above reasons, especially when combined with other LLMs that can condense large swathes of information to give it as inputs, as I explain below.

This Tutorial explores four key ways professors can use LearnLM to enhance their teaching.

I'll show you how to…

• create sophisticated assessments that promote learning

• develop clearer and more effective assignment instructions

• provide more constructive feedback on student work, and

• support student learning through guided tutoring.

These are the four “cases” I cover below. With each case, I provide detailed, practical guidance for working with LearnLM, including how to structure your course materials, craft effective prompts, and refine the model's outputs.

As long-time readers might expect, the net result is a very long block of text with extensive explanations and examples.

Since I know you probably don’t have time to read it all, check out my tip below for a way to quickly apply the Tutorial to your needs…

❓❓ Case 1: Create Quizzes and Tests from Course Content

Creating assessments that both measure and support student learning takes significant time and expertise. While we want our assessments to help students integrate concepts and think critically, it’s often the case that we lack the bandwidth to create ideal assessments. (I’m assuming I’m not alone in this…)

Let's explore how to use LearnLM to create effective assessments, from organizing your materials through refining the final product.

Step 1: Organize and Prepare Your Source Materials

To prepare your materials effectively, follow these six sub-steps, which I'll explain in detail below:

1. Gather all relevant course materials, from teaching content to supporting documents

2. Convert everything to digital, machine-readable formats

3. Structure your documents with clear headers and section markers for LearnLM to reference

4. Break down large materials into topic-focused chunks

5. Use other LLMs to condense lengthy content when needed

6. Remove sensitive information and organize content logically

First, gather your course content comprehensively. This means collecting whatever is relevant to the assessments you are trying to build. For instance: lecture recordings, presentation slides, class handouts, readings, homework assignments, and supporting documents. Include materials that define your educational goals, such as learning objectives, curriculum maps, and assessment criteria.

The goal is to give LearnLM access to both what you teach and how you want students to learn it. However, keep in mind LearnLM's 32,000 token context window — you'll likely need to break this content into smaller, focused chunks organized by topic or learning objective.

Next, make your materials accessible to LearnLM by converting them to easily parsed digital formats and managing their size. LearnLM can work with various file formats, but focused, text-based content chunks are most effective for assessment creation. Here’s a table for your reference:

The third step — structuring your documents — is crucial for both LearnLM's understanding and managing content within its 32,000 token context window. Use clear, consistent headers and section markers throughout your materials. These act as navigation points that help you both refer to specific content when creating questions and divide content into manageable chunks for LearnLM to process.

For example, start main sections with descriptive headers like "Unit 3: Macroeconomic Policies - Fiscal and Monetary Tools" or "Week 5: Quantum Mechanics Foundations - Wave-Particle Duality." Use subsection headers to mark important concepts within each main topic. This hierarchical structure makes it easier to split content into focused segments that fit within LearnLM's context window while maintaining coherence.

Order topics logically, whether chronologically or by complexity. Cross-reference related concepts explicitly in your section headers or summaries, but keep in mind that you may need to feed related sections to LearnLM separately due to its context window limitations. The goal is to create well-structured material chunks that help LearnLM understand both individual topics and their connections to other segments you'll introduce separately.

Fourth, and relatedly, consider enhancing your document structure with brief introductory summaries at key points (read: use brief prompts to get an LLM like ChatGPT to do this). These summaries are particularly valuable when working with LearnLM's context window limitations, as they help maintain coherence across separate chunks of content. They serve multiple purposes: they state learning objectives, highlight essential concepts, and most importantly, make explicit connections between different parts of your course that may need to be processed separately.

If you think the linkages are self-evident within each content chunk, this last sub-step isn't necessary. You could also explain the linkages in your prompts (more on this below).

Step 2: Access and Configure LearnLM

Begin by accessing Google AI Studio and logging in with your Google account. Create a new prompt and select LearnLM from the model dropdown menu (it will be under “Preview” and may be labeled “LearnLM 1.5 Pro Experimental”). The interface shows four main components: a system instruction box (at the top), an upload area (marked by a plus symbol), a prompt input window (“Type something”), and a “Run” button. There are other settings on the right side panel but you can ignore them.

At this stage, upload your prepared course materials using the plus symbol, but don't click "Run" yet — we'll craft an effective system instructions and prompt first in Step 3. The interface will list your uploaded documents in the upload area.

Remember that LearnLM will work with the internal structure you created in Step 1, using your headers and markers to understand relationships between concepts. (Or if you didn’t do enough of that, prepare to flag it explicitly in your prompts.)

Step 3: Craft Structured Instructions

Like Gemini and other LLMs, LearnLM works best with structured instructions that guide it through complex tasks. Your instructions have two components: “system instructions” and “prompts.”

The system instructions provide big-picture context about the role LearnLM is playing, who you are, and what you need. Here’s a schematic template for such a system instruction with filler bracketed parts.

After you insert this system instruction, input a prompt that has several layers, including any relevant background information, assessment question examples, desired output formats (what you want from the LearnLM beyond the questions themselves), and more detailed instructions for how LearnLM should proceed.

Here again is a schematic version of such a prompt, which I explain at greater depth afterwards:

The first layer of the prompt explains the structure and content of your uploaded materials. This helps LearnLM understand how to navigate and use your course content effectively. Describe the organization and relationships between your files, referencing the headers and markers you created in Step 1.

If you can or want, it is an option to paste the documents’ contents into the prompt itself, providing headers along the way (you can use markdown to demarcate structure, as I am in the above examples; #x for H1, ##y for H2, etc). This is more work but may get better results because it guarantees the textual content is in a format that LearnLM can parse.

After establishing the overall context and providing your files as background information, give several concrete examples that demonstrate the types of questions you want. These examples serve as models for LearnLM to follow, showing both the structure and the level of sophistication you expect.

Next, provide a clear format for the output. Ideally this would match the format of the examples above, but it isn’t necessary (this is extra work; I don’t do this here). This structure ensures consistency and makes the assessment easier to review and implement. The format should capture both the question itself and its pedagogical purpose:

Conclude with step-by-step instructions that explain how a generalist pedagogy expert (LearnLM) should proceed through what you provide to achieve your goals.

Step 4: Review and Refine Generated Assessments

Review the assessment both holistically and question by question. Does it create a coherent learning experience? Check how LearnLM has referenced your course materials and explained its pedagogical choices.

When you find areas for improvement, structure your revision requests clearly. For example:

You can request multiple improvements simultaneously:

Given its context window size, LearnLM remembers previous versions and feedback throughout refinement, allowing you to reference specific elements when requesting changes. Through this process, you can develop an assessment that both measures and supports student learning.

(If you go on for a while, or if you have a lot of documents or files, you will reach the end of the context window and may need to start a new conversation to ensure your initial prompt is retained.)

📄✏️ Case 2: Improve Pedagogical Quality of
Assignment Instructions and Rubrics

Clear assignment instructions and well-designed rubrics are essential for student success, yet crafting them requires balancing detail with clarity. Students need to understand both what to do and how their work will be evaluated, while instructors need rules or guidance for consistent grading across many submissions (and many levels of fatigue).

What makes LearnLM particularly valuable for this task is its understanding of pedagogical scaffolding and assessment design. The model can analyze your current materials and suggest improvements that clarify expectations, highlight learning goals, and create better alignment between instructions and evaluation criteria.

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I report back from the field: I share some of the good questions faculty have asked me lately, as well as what I said in response.

You may have some of the same questions, so I hope this is useful to you, but feel free to respond to this email to ask me other questions — I’ll include my answers in a later edition.

Remember, too, about my next public AutomatED webinar, which will be on January 31 on Zoom and will focus on techniques to build better AI tutors.

❓ My Answers to 5 Faculty Questions

This year, I plan to keep you all more aware of what I’m doing and learning while helping faculty engage with and integrate AI.

Two weeks ago, I spoke to faculty at NOVA during the WONDER conference, then flew to Benton Harbor, Michigan to speak to faculty at Lake Michigan College. This past week, I went back to the same neck of the woods to deliver two hands-on sessions on prompting and AI usage to faculty at Southwestern Michigan College. I wrapped up my trip with a brief session with a pedagogy seminar for graduate students at Notre Dame.

(If you’d like to book me for this spring, or discuss options, click here.)

Here are 5 questions I got from the faculty I spoke with. I figure you might find my answers useful…

1. “What if some of my students can afford premium AI tools and others cannot? This is a serious equity issue!”

It’s a legitimate concern and there isn’t a silver bullet. Here are some thoughts.

While there are pricey premium tools like GPT-o1-pro (at $200/month), almost all tools either have comparable free versions or are $20/month (e.g. ChatGPT Plus, Gemini Advanced, Claude Pro).

The bright side is that the free tools are amazing — better than premium tools from ~6 months ago — and aren’t significantly worse than the paid ones for most tasks. 

This means students’ opportunities and access have increased in absolute terms — that is, for goods that they can accrue regardless of how much of them their peers accrue.

The dark side is that there are some key tasks, relevant in the university setting, where the difference is significant. GPT-o1-pro is significantly better than GPT-4o at anything complex, and crucially requires less prompting skill.

This means that students’ opportunities and access have decreased in relative terms, at least if they can’t afford what their peers can.

(Imagine, to make it stark, a course graded on a curve with only perfect-scoring students able to get As, where GPT-o1-pro access guarantees perfect outputs on the assessments but where GPT-4o is imperfect.)

One solution: instructors should try to leverage AI tools in their free versions and create assignments/assessments that don’t reward students who have the means to leverage the premium tools. This requires regular testing of the main tools’ capabilities but also course design that has key moments that are AI free — and those need to be the moments where everyone can shine, including with respect to their grades.

This is the same type of solution we deploy to mitigate other sources of unjust inequalities.

Another option, shared with me by Maria H. Andersen on my LinkedIn post on this topic, is as follows:

I think this is a good idea. It’s just like if you were to “share” access to an expert — access that not everyone has — and discuss/analyze what they told you with your colleagues or peers.

But everything is easier if your institution provides a high-quality baseline AI tool to all students. More on that below… 

2. “What’s the highest impact way to deploy an AI chatbot in my classes? The easiest?”

The best bang for your buck is a custom GPT. 

In essence, custom GPTs are modular or prepackaged AI chatbots that you can share via hyperlink. You do the modularization or packaging; you add meta-prompts or “instructions” and a knowledge base of files to enable them to simulate you (e.g. for tutoring or syllabus knowledge), your teaching assistants (e.g. for grading), or your students (e.g. for peer review or group work).

You spend $20/month for ChatGPT Plus (which is required to build them), you build the custom GPTs, and your students pay nothing to use them. (On the free plan, your students do face usage limits; these can be removed if they pay $20/month.)

You can read more about custom GPTs in my ✨Premium Tutorial dedicated to them.

If you want to learn how to build better custom GPTs, I have created two resources for you beyond the Tutorial:

1. Our new $25 course

   • Early feedback has been very positive, with one learning specialist saying that the price is “bargain basement” for what you get and with a professor describing it as “interesting,” “useful,” and “helpful.”

2. My free-for-you webinar on Friday January 31 at noon Eastern on Zoom 

And you get a discount on the course if you purchase the webinar, for a combo deal.

(For more information on the webinar, see my email from yesterday, if you haven’t registered already.)

3. “Why isn’t it feasible for me to change nothing about my courses, which are ‘old school’ in structure and content (device-free, in-class pencil-only tests, lecture-heavy), despite the power of AI?”

It isn’t feasible and here’s why:

Imagine you are teaching your “old school” course and all of a sudden every student correctly expects to live their life with immediate access to high-IQ tutors, mentors, and consultants who know a whole lot about nearly everything, including your field.

Imagine, too, that within a few years, these “assistants” will be 10x more powerful and integrated with nearly all aspects of your students’ lives, careers, and environments.

That is, outside of a few contrived contexts where they can’t query, direct, and rely on these people, your students will be able to leverage them for a very low cost and will be expected to do so by their managers, family members, friends, etc.

You wouldn’t — and shouldn’t — leave your course unchanged in such a scenario.

You are in such a scenario, with AI.

So, change what you need to change.

(One compact case for thinking you are in such a scenario is as follows. The capabilities we are seeing or expecting from the latest models, whether GPT-o1-pro, GPT-o3 (soon to be released publicly), or Gemini 2.0, are genuinely impressive and go beyond the capabilities of most humans on most knowledge tasks. Personally, I’m finding o1-pro to be incredibly powerful in a range of use cases, and I am not alone — I’ll share other professors’ takes on its abilities in the coming months. Further, many of the past obstacles to AI success are being overcome, from context window limitations to hallucinations. Finally, the AI developers from OpenAI to Google are only gaining speed and resources, and they are repeatedly signaling that we are much closer to AGI and superintelligence than they expected to be at this point.)

4. “Which AI tool is best for <task X> in <my field>?”

You, expert, need to find out yourself by experimenting. I don’t know what high-quality outputs look like in your field, so I can’t tell you.

I can give you some initial guidance on where to look, though.

In descending order (internal to each category)…

For reasoning-heavy fields: 

• GPT-o1-pro

• GPT-o1

• Gemini 2.0

• Claude 3.5 Sonnet

• Gemini 1.5 Pro = GPT-4o

For research-heavy fields:

• Elicit

• Perplexity

• Gemini 2.0

• Google Deep Research

• ChatGPT Search

For writing-heavy fields:

• Claude 3.5 Sonnet

• GPT-4o

• Gemini 2.0

For coding-heavy fields:

• GPT-o1-pro

• GPT-o1

• Claude 3.5 Sonnet

• GPT-4o

With most of these tools, you’ll need to be able to prompt them effectively to get expert-level results. For that, I recommend the various ✨Premium Tutorials I have in the Archive dedicated to the topic (e.g. on prompting Gemini and long context more generally, or on prompting ChatGPT4o for image and data analysis), or wait until my next prompting webinar (later this spring).

(Yes, you really need to dive in and get your hands dirty with the tools, prompting them for — minimally — several dozen hours over the coming semester to better understand what they can do. To be completely frank, if at this point you don’t have significant experience testing the best AI tools for the tasks that play central roles in your field and life, you’re behind and need to catch up.)

✨5. “Which large language model (LLM) should my institution purchase for all students to use?”

📢 Quick Hits:
AI News and Links

1. Want to catch up on AI development over the past few years? Here are 50 papers that explain the latest LLMs, the benchmarks that are used to evaluate them, big moments in prompt engineering, agents, vision, voice, fine-tuning, and more.

2. ICYMI (“in case you missed it”): Google updated NotebookLM with three major changes: a new three-panel interface for better content management, interactive voice conversations with AI hosts during Audio Overviews, and a premium "Plus" subscription for power users that offers 5x higher usage limits. The Plus version will be included in Google One AI Premium in early 2025.

3. A new analysis in Lawfare argues that OpenAI's o3 model, which reportedly scored 87.5% on the ARC-AGI benchmark (above human performance at 85%), represents a fundamental shift in AI development. The authors say this unexpected breakthrough — jumping from 5% to 87.5% in months — proves powerful AGI requires immediate regulatory action. They call for proactive governance frameworks, global coordination, and economic preparation for widespread AI transformation.

4. Google is starting its push of including Gemini as a non-optional feature in all Workspace plans, starting with Business and Enterprise, and increasing their price slightly as a result.

5. Microsoft announced several new AI agents for Microsoft 365 Copilot users, including:

• A SharePoint agent that provides instant answers based on site content

• A "Facilitator" agent for Teams that takes real-time meeting notes and summarizes group chats

• An "Interpreter" agent coming in 2025 that provides real-time speech-to-speech translation in 9 languages

• A "Project Manager" agent in Planner that can create and execute project plans

The agents require a Microsoft 365 Copilot license and are rolling out in preview to customers.

6. Microsoft 365 Copilot is coming to Personal and Family subscribers (in place of Copilot Pro), by default in most markets.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I take a closer look at Google’s new Deep Research tool, which is to the research report what ChatGPT was to the essay.

I also announce the next AutomatED webinar, which will be on January 31 on Zoom and will focus on advanced techniques to build better AI tutors.

Save 1 hour every day with Fyxer AI

Fyxer AI automates daily email and meeting tasks through:

• Email Organization: Fyxer puts your email into folders so you read the important ones first.

• Automated Email Drafting: Drafts replies as if they were written by you; convincing, concise and with perfect spelling in every language.

• Meeting Notes: Stay focused in meetings while Fyxer takes notes, writes summaries and drafts follow-up emails.

Fyxer AI is even adaptable to teams!

Setting up Fyxer AI takes just 30 seconds with Gmail or Outlook.

Free 7 Day Trial

🧰 An AI Tool For Your Toolbox:
 Google’s Deep Research

What is it?

As I reported last week, Google just launched Deep Research, an AI research assistant in Gemini Advanced that autonomously explores complex topics and compiles comprehensive reports with source links. As Google describes it:

This week, I explain how it works in greater detail and give my judgment on its relevance to teaching faculty.

How can one get access to it?

First, go to https://gemini.google.com/advanced and sign up for a Google One AI Premium plan. A one month trial is available for first-time users; the rest pay $9.99 for two months and then $19.99 for month three onward.

Once you’re logged in, click the dropdown menu in the upper left and select 1.5 Pro with Deep Research:

How does it work?

It proceeds through 8 phases:

1. Accepting input (a simple chat/search bar; you input a question or direction):

2. Planning (Gemini 1.5 Pro, the LLM, comes up with the best approach to tackle your question; you wait):

3. Accepting revisions to the plan (a dialogue bar, like chatting with Gemini in other contexts; as you can see in the screenshot above, you can click the “Edit plan” button and then tell it what to change in a conversational way, or not)

4. Researching (Google Search combines with Gemini 1.5 Pro in a way that we cannot observe to search for what the plan specifies; you wait)

5. Analyzing (Gemini 1.5 Pro processes all the text and other media it has gathered; you wait)

6. Creating the report (Gemini 1.5 Pro begins to generate text based in what it analyzed and what the plan concerned; you wait)

7. Delivering the report (Gemini 1.5 Pro delivers a report in preview form, with the ability to open it in its entirety in Docs; you click “Open in Docs” or ask for revisions)

8. Accepting requests for revisions to the output (a dialogue bar, like chatting with Gemini in other contexts; you can tell it what to change in a conversational way, or not)

Finally, since Gemini is in Docs for those with Gemini Advanced, you then have another option for further revisions:

9. Accepting requests for revisions, summaries, analyses of the output (a dialogue bar, like chatting with Gemini in Docs; you can tell it what to change in a conversational way, or not)

✨What do I recommend teaching professors do?

I conducted two preliminary tests of Deep Research’s abilities, each focused on a research question that, for different reasons, I have significant familiarity with:

1. Why has the proportion of adjunct faculty increased at colleges and universities in the US since the 1950s? (some screenshots from this test are included above)

2. What are some of the strongest arguments for and against attributing moral responsibility to AI systems, supposing they qualify as superintelligent (as Sam Altman thinks they soon will) or AGI?

First, I will share what Deep Research did in each case.

After that, I will share my recommendations for what teaching professors should do in light of this new tool.

In short, it is a very big deal for educators who assign research, like ChatGPT was for those who assign essays.

📢 January 31 Webinar:
On Building Better AI Tutors

The results of my subscriber surveys in: the next webinar will be on Zoom on Friday, January 31 from 12pm-1:30pm Eastern time, and it will cover building better AI tutors, with a focus on OpenAI’s custom GPTs, Google’s Gems, and Claude’s Projects, as well as more advanced implementations.

If you want to tutor your students with an AI chatbot — simulating a teaching assistant, a mentor, a peer-reviewer, someone who knows the syllabus really well, or … — this is the webinar for you.

✨Premium subscribers get in for free, while the price is $25 for everyone else.

However, the webinar will assume, as a prerequisite, that you have a solid grip on the basics of building custom GPTs.

If you aren’t confident that you do, I recommend that you take our recently released custom GPT course, now live on LearnWorlds for $25. Those who sign up to the webinar will get 20%-off discount code for the course.

In terms of format, the webinar will be roughly 65% presentation, 35% discussion or Q&A. The presentation part of the webinar will be on more advanced topics like:

• strengths and weaknesses of custom GPTs vs Gems vs Projects

• how to address common tutoring pitfalls with them

• when to pivot to creating more bespoke solutions with LLMs’ APIs, like how to link custom GPTs to other LLMs like LearnLM or how to use AI to regulate itself

In the discussion part of the webinar, you can ask questions, hear what others are doing, and get my take on how to tutor students with these amazing tools.

Sign up here:

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I explain why I think 2025 is a key moment for many educational institutions and educators, at least with respect to the growing AI digital divide.

I also provide some AutomatED updates, including the release of our new custom GPT course and the release of my AI-powered department course scheduler (email me if interested; more information on it at the bottom of this message).

💡 Idea of the Week:
Don’t Let Your Colleagues Wait

This week, I will be speaking about AI at the WONDER conference hosted by NOVA and then at Lake Michigan College.

The following week, I return to Michigan to talk with faculty at Southwestern Michigan College and then go a little south to discuss AI pedagogy with graduate students at Notre Dame.

The commonality across these diverse audiences is that they are all working hard to stay ahead of the AI curve.

This is not just wise — it is essential. AI is yet another source of digital divides, and those institutions, departments, faculty, and staff who aren’t working hard on how it can be useful for them are going to fall behind.

I think 2025 is a turning point; it is the year where the AI digital divide becomes stark.

Consider two recent developmenets that illustrate why:

Google just launched "Deep Research," an AI research assistant in Gemini Advanced that autonomously explores complex topics and compiles comprehensive reports with source links. As Google describes it (here’s also a video that conveys how it works):

Meanwhile, OpenAI just announced o3, their next frontier model (‘o2’ is being skipped because it’s copyrighted). Their internal testing shows it outperforming PhD-level experts on scientific reasoning tasks and beating all of their programmers (except one?) on coding benchmarks. In fact, o3 can solve novel mathematical problems that would "take professional mathematicians hours or even days to solve." Here’s their YouTube video with more:

On the basis of this progress, OpenAI's CEO Sam Altman stated in a blog post released yesterday that the company is now “confident we know how to build AGI” [artificial general intelligence] and predicting that in 2025, “we may see the first AI agents join the workforce and materially change the output of companies.”

Many faculty I talk to about developments like these are already behind. They don’t know basic facts about its functionalities, aren’t working to integrate it with their workflows, data, and field-specific tasks, and aren’t teaching their students how to use it, despite their students’ interest in it (for evidence of the latter, see here, here, here, and here).

Those of you reading this newsletter are likely among the minority of faculty who see what's coming, even if you differ with me and each other about the details. You're in what I observe to be the smallest of four groups:

1. Those who are already there; power users or informed skeptics who have thought deeply about AI's implications

2. Those who wish they knew more and are actively working to learn and adapt

3. Those who wish they knew more but haven't made the time

4. Those who actively resist engagement, often virtue signaling about AI's risks while remaining willfully ignorant of its functionality and its impacts

2025 will be defined by two major trends that make your role as an early adopter crucial for your department and institution:

First, agential AI — AI that can guide itself with more autonomy — will become increasingly capable. Tools like Google's Deep Research aren't just research assistants; they're precursors to AI systems that can actively pursue complex academic tasks with minimal oversight. This isn't just Google's goal — it's an express aim of many major AI developers.

Second, AI will have increasing access to context-sensitive data. For those of us already working to connect AI with such data (like my department course scheduler project), we know its capabilities are remarkable when properly integrated with domain information or field-specific knowledge. From small projects like mine to massive initiatives at Apple and Microsoft, the integration of AI with institutional data and workflows is accelerating.

These trends mean that departments and programs that delay restructuring their curriculum and pedagogy around AI will quickly fall behind those that don't. The gap won't just be in efficiency — it will be in the fundamental quality of education they can provide. Students need to learn not just their field's content, but how to leverage AI within it.

This means you, as an early adopter in your department, have a crucial role to play in 2025. Your institution needs more than just AI policies, vague statements from the provost, and monthly reading groups (which, while well-intentioned, don't move any meaningful needles). It needs concrete plans for instructional integration, program restructuring, and faculty training.

Without them, you’ll end up on the wrong side of the latest digital divide.

Stay tuned as I report back on whether I am right.

Next week, I give my take on how well Deep Research works in its current form. (And if you have any tough problems for me to test o1-pro with, email me.)

🤖 New Custom GPT Course
& Other Updates

2025 brings many changes to AutomatED…

Weekly Newsletters Again but Shorter

I know that my newsletters are often too long and you’re always too busy. In 2025, I’ll be working hard to write more concisely.

But this brings a new problem: there is too much going on in AI land, and sometimes a deep dive is necessary for a tricky ethical, pedagogical, or technical issue.

Solution: I keep you updated with weekly newsletters, and you can get ✨Premium or come to the monthly webinars if you want deep dives.

Custom GPT Course

The course is now live on LearnWorlds (our e-learning host), and you can access it via this link:

I think you’ll like it a lot. Beta feedback has been very positive so far!

If you are a ✨Premium subscriber, your discount code for 20% off is immediately below:

✨Premium Improvements

2025 brings some big improvements to ✨Premium subscriptions — while the price remains the same ($99 per year):

• Free Access to Monthly Webinars

  • Going forward, webinars are free for paying subscribers. (Previously, Premium subscribers got 60%-off discounts.) They’ll also be cheaper — and, once per semester, free — for everyone else.

  • My next webinar will occur at the end of January, on a topic to be decided in polls (see below).

  • My last webinar, on December 6th, concerned “How to Use AI to Improve & Accelerate Student Feedback.”

• Discounts on Courses

  • I’ve been working on interactive courses with Charlie Fuller, an e-learning expert. The first one is on custom GPTs and costs $25, but ✨Premium subscribers get a 20% discount (see above).

  • We plan to release 5-10 more courses in 2025.

• Premium Sections in Weekly (Free) Newsletters

  • After this week, there will be a section in every Monday’s (free) newsletter that is only for Premium subscribers. I’ll include polls to help determine what information Premium subscribers want me to provide there, but it will definitely include prompting techniques, commentary and predictions, and insights into more advanced uses of AI.

• Free 1-Hour Consultation with Me Each Year

  • In addition to my presentations, I consult with dozens of professors each semester about their tech-related pedagogical questions/challenges, and I build custom AI tools for them, too. As a Premium subscriber, you’ll have access to an hourlong free consult each year where we can discuss anything you’d like.

• No Sponsor Advertisements Shown to You

This is in addition to all pre-existing ✨Premium benefits (all 17 Guides and Tutorials accessible here):

• Comprehensive Pedagogy Guides

• AI Use Case Tutorials

• AutomatED AI Tools

• Two New Premium Pieces Per Month

  • I’ve got a bit of a backlog right now, but at least the following 6 pieces will be sent in January and February:

    • Tutorial: Anonymize Canvas Submissions in Microsoft 365

    • Tutorial: How Profs Can Use Copilot in Microsoft 365

    • Tutorial: How Profs Can Use Gemini in Workspace

    • Tutorial: How to Use Google’s LearnLM

    • Tutorial: Zapier for Professors and Learning Specialists

    • Tutorial: How to Teach With Google’s Deep Research

January’s Webinar

As I mentioned at the end of last year, I am going to turn to more advanced topics more generally in 2025, in my ✨Premium content and in my paid webinars. Free newsletters and free webinars will stay more focused on the basics and keeping everyone up to speed.

The next webinar will be paid but free for ✨Premium subscribers. Please let me know via the below poll what you’d like it to cover (after you make a selection, you will be redirected to the website where you can also offer typed comments, if you want):

If I were to do one on custom GPTs, Gems, and Claude Projects, I would focus on topics about custom GPTs, Gems, Projects, etc. that are not covered in my pre-existing Tutorial or the new course. Topics like: API connections, workflow optimization, when to pivot to Zapier and LLMs’ APIs, etc.

🧪 AI Department Course Scheduler:
It’s Here

After 3 months of development, my department course scheduling AI tool is now ready.

The system can handle faculty-to-course assignments, course-to-room scheduling, or both, working directly with your existing spreadsheets and faculty preference data — no format changes needed and no proprietary forms.

Given the costs of developing the tool, running it, maintaining it, and ensuring user satisfaction with its outputs, I am now treating it as a “product” of AutomatED that I’ll offer at a (very low, relative to the market) price.

For the January-March scheduling period, I have about 20 remaining slots available.

I’m offering “beta” participants the following:

• Free initial compatibility check of your existing data (I prove to you that it can do your upcoming semester by doing your prior semester)

• Promised 5-day turnaround on optimized schedules

• Multiple revision cycles for adjustments (you’d be surprised how many contradictions, ambiguities, lacunae, etc. exist in the average dataset I encounter)

• Early access pricing

Interested?

The first step, in most cases, is to email me a sample of your past scheduling materials (Fall 2024 would be ideal) for a compatibility assessment.

But feel free to reach out to discuss first, if you’d like…

Note: I hope to have a self-supported web interface live in February. For now, beta interactions are via email with full support.

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

In this last piece for 2024, I share…

1. the top 10 links you clicked this year

2. the top 4 newsletters by web views this year

3. the top 3 newsletters by click rate this year

4. the top 3 newsletters by email open rate this year

Let’s see what you — and the nearly 5000 other higher educators subscribed to AutomatED — found interesting and useful this year!

Last time I emailed you, I outlined seven tech- and AI-related tasks for professors to consider completing before the spring semester:

• experiment with the latest AI tools

• review past teaching performance

• update your course design

• build custom AI assistants

• organize your non-AI technology

• consider AI's role in your research

• take time to rest

Check it out via the link above if you missed it in the holiday rush…

I also announced that I have been collaborating with learning experience maestro Charlie Fuller on an interactive and engaging course that walks you through building custom GPTs. By the end, you have at least one custom GPT, fully ready to deploy. This course is like giving steroids to my ✨Premium Tutorial on the topic. It will be released next week.

If this interests you, express interest via the poll below so we can keep you in the loop:

Now, for this week, I’ve dug through my analytics dashboard to extract several lists of the top AutomatED content from the past year.

I know it’s easy to have missed important tidbits when so much is going on with tech and AI right now.

And it doesn’t help that I send so many emails... Apparently, I’ve sent 144,400 this year, across all subscribers:

But what do I mean by “top AutomatED content”? I am referring to what you — my 5,427 subscribers (~70% PhDs) — opened and clicked, as well as what garnered the most interest from web viewers. (Most web viewers are not subscribers, and they arrive via direct links or organic search.)

Below are four sections:

1. the top 10 links you clicked

2. the top 4 newsletters by web views

3. the top 3 newsletters by click rate

4. the top 3 newsletters by email open rate

Stay tuned next week for my thoughts on where AI is headed in 2025, what to keep an eye on if you are going to be teaching, and my plans for AutomatED.

Thanks for a great year!

🔗 Top 10 Links from 2024

1. “Microsoft Education AI Toolkit: A navigator for education institutions to plan their AI journey" by Microsoft

2. “Empowering Education Leaders: A Toolkit for Safe, Ethical, and Equitable AI Integration” by the DOE’s Office of Educational Technology

3. “How Close is AI to Taking Over the Role of the Instructional Designer” by Philippa Hardman

4. “Course Design Wizard - a custom GPT designed to help professors and other university/college educators create better assignments, rubrics, and course syllabus policies in the AI era” by me

5. “Learn how to write AI prompts in 5 easy steps with this new course from Google” by Google

6. "Claude 3.5 Sonnet for sparking creativity" by Anthropic (click the below image; it’s a YouTube video)

7. “How I Use ‘AI’” by Nicholas Carlini (Google engineer/researcher)

8. Google AI Studio (where you can access Google Gemini, LearnLM, etc for free)

9. Survey on “Generative AI and Postsecondary Instructional Practices” by Ithaka S+R

10. Moral AI by Jana Schaich Borg, Walter Sinnott-Armstrong and Vincent Conitzer

🌐 Top 4 AutomatED Pieces from 2024,
by Web Views

✨Tutorial: How to Build (Better) Custom GPTs

A comprehensive how-to for the ultimate (and now free) tech tool for personalization.

automatedteach.com/p/tutorial-how-to-build-custom-gpt-tutor

10 Ways I Use LLMs like ChatGPT as a Professor

ChatGPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, custom GPTs - you name it, I use it. Here's how...

automatedteach.com/p/10-ways-to-use-llms-ai-chatgpt-professor

✨Guide: How to Train Students to Use AI

Helping you teach your students to use AI, in general and for field-specific tasks.

automatedteach.com/p/guide-teach-train-students-how-to-use-ai

GPT-4o is Here and GPT-4 + Custom GPTs are Now Free

Plus I release a custom GPT-4o for digitizing anything handwritten.

automatedteach.com/p/gpt4o-custom-gpts-handwritten-student

Keep your meeting data safe

Worried about all the AI bots joining your meetings?

Fellow is the only AI meeting assistant built with security controls and permissions.

Thousands of organizations rely on Fellow to capture their meeting transcripts, summaries, and action items – all in one secure platform.

Claim 90 days of unlimited AI notes today… and stop sharing your meeting data with AI note-takers that lack proper security.

Get started free

🖱️ Top 3 AutomatED Pieces from 2024,
by Click Rate

Claude 3 for Lesson Planning, and Positioning AI Well

Can Sonnet outperform ChatGPT?

automatedteach.com/p/claude-3-lesson-planning-positioning-ai-well

How to Cite AI

Should students shirk credit for AI-generated work? Plus, SearchGPT and the Canvas+Khan collab.

automatedteach.com/p/how-to-cite-ai-mla-apa

AI Training and Developing Good Judgment

I discuss a common misunderstanding, highlight amazing recent AI advances, and ask for your input.

automatedteach.com/p/ai-training-students-judgment

✉️ Top 3 AutomatED Pieces from 2024,
by Email Open Rate

AI Literacy and "Technical" Knowledge

Plus, my upcoming webinar on using AI for feedback.

automatedteach.com/p/ai-literacy-and-technical-knowledge

ChatGPT's Search Engine is Here

Plus, a Dec. 6 webinar on using AI for feedback.

automatedteach.com/p/chatgpt-search-engine-is-here

Don't Let AI Make You Forget "Dumb" Tech

Especially when it can enhance AI. Take the case of Google's underrated Apps Script.

automatedteach.com/p/gemini-gmail-to-google-docs-apps-script

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

In this second-to-last piece for 2024, I…

• outline 7 tasks you might consider doing in the next few weeks, including one that probably shouldn’t be called a ‘task’

• share the latest AI news, including some surprising developments from the big dogs

• and reflect on my free webinar from December 6th (and ask for your opinion about the next one)

Let’s dive in!

💡 7 Tasks to Do Before Spring

When I give talks, trainings, and webinars, many professors report to me that they appreciate it when I reveal more of my own views and predictions on the “state of play” of the AI-education space, especially during Q&A periods (I guess that’s when I loosen up a bit).

I figure now is a good time to do something analogous here, with a focus on what tasks I personally think you should be working on now.

Here they are, in rough order of operations…

1. Experiment With the Latest AI

The AI landscape is constantly evolving — the amount of money, effort, and intellectual firepower involved in developing and improving AI tools is absolutely massive right now — and it has shifted dramatically in just the last few weeks. As I detail in my “Quick Hits” news section below, we're seeing groundbreaking developments from both Google and OpenAI, from autonomous research assistants to more powerful language models to sophisticated video generation tools.

If you haven't hands-on tested these tools recently, your mental model of AI's capabilities is outdated. The rate of development makes staying up to date hard, but being outdated is a matter of degree, and you don’t want to be too far out of date, whether as a teacher or as a researcher.

In other words, this isn't just about staying informed in an abstract sense — it's about understanding how these advances impact your field.

For instance, Google's new Gemini 2.0 (which I explain below) isn't just another incremental update; it's being implemented now to deploy supervised "agentic" capabilities that could transform how we approach complex academic tasks. Google launched "Deep Research," an AI research assistant in Gemini Advanced that can autonomously explore complex topics across the web and compile comprehensive reports with source links. Meanwhile, OpenAI's new premium LLM “o1 pro mode” is showing unprecedented reliability on advanced academic problems. (Below, I ask you to send me any tough problems to feed to the beast; I bought it for a month, for science.)

Take advantage of this time to experiment with these tools in your specific domain.

Sure, you can also read about what they can do, but I think you can’t just read about them — you need to test them with real academic challenges from your field and your own classes, especially given you have unique expertise and knowledge. Your firsthand experience with these tools will help you make informed decisions about how to integrate them into your teaching, how to modify your methods to avoid their pitfalls, and how to guide your students in using them effectively.

Relevant Resources:

ChatGPT’s Search Engine is Here
AI That Can Click and Type?!
GPT-o1’s Arrival is Pivotal
Google’s Custom GPTs (Gems) are Here

2. Review How Last Semester Went

Now that grades are submitted, it's worth taking a systematic look at how your courses went. Tools like ChatGPT's Advanced Data Analysis can help analyze your (anonymized) gradebook data in ways that weren't easily accessible before. Beyond simple grade averages, you can explore correlations between different assignments, identify where students consistently struggled, and spot patterns across different types of assessments.

The data might surprise you. Perhaps those early writing assignments weren't actually predictive of final essay performance, or maybe quiz scores in certain units highlighted conceptual gaps that persisted throughout the term. Understanding these patterns can help you adjust your approach for spring — whether that means restructuring assignment sequences, rethinking assessment types, or identifying where students need additional support.

Just as importantly, reflect on how AI impacted your teaching this past term. What worked about your AI policies and what needed clarification? Were students asking for more guidance on appropriate AI use in specific contexts? Did you notice any patterns in how students incorporated (or misused) AI tools? Taking stock now, while the semester is fresh in your mind, will help you make thoughtful adjustments for spring (a topic I cover in the next item on this list).

Relevant Resources:

✨Tutorial: How to Analyze your Teaching with ChatGPT
✨Guide: How to Ethically Use AI with Student Data
GPT-4o Democratizes Data Analysis

3. Engage in AI-Sensitive Course Design

The break between terms is your chance to fundamentally rethink your courses through an AI lens. With new AI tools emerging every other month and students becoming increasingly sophisticated in their use, last semester's course design might already be outdated. This isn't just about tweaking a few assignments — it's about reimagining your entire course structure, assessment methods, etc.

Start by evaluating each assignment and assessment against current AI capabilities. Which ones are still achieving their intended learning objectives, and which have become vulnerable to AI shortcuts? Consider whether traditionally written assignments might work better as oral presentations, or if take-home essays should shift to in-class writings. Multimedia is still a big challenge for AI. But remember that format changes alone aren't enough — you need to think about content too. Tasks requiring specialized knowledge application or complex, multi-step reasoning often remain more resistant to AI than those asking for simple synthesis or summary, although this gap is shrinking (see above).

But don't just focus on preventing AI misuse. Think about where AI tools could actually enhance learning in your course. Could a custom GPT help students brainstorm research topics? Might ChatGPT serve as a practice partner for language learners? The key is designing a course that thoughtfully incorporates AI where it adds value while maintaining the integrity of core learning experiences that need to remain AI-free.

Finally, reflect on whether you want to fundamentally change the paradigm around your use of student data, with AI or otherwise. If, for instance, there are tons of ways to use student data that you wish you had student consent to carry out, now’s the time to implement a way to acquire and manage informed student consent.

Relevant Resources:

✨Guide: Designing Assignments and Assessments in the Age of AI
✨Guide: How to Train Students to Use AI
✨Guide: How Professors Can Discourage and Prevent AI Misuse
✨Guide: Crafting Your Syllabus' AI Policy
✨Tutorial: Easy Student Consent Management in Microsoft 365
✨Tutorial: Easy Student Consent Management in Google Workspace
3 Strategies to Discourage AI Misuse
The 4-Step Process to Teach Students AI
How to Cite AI

4. Build Custom GPTs, Gems, Projects, … For Each Use Case

Now's also the time to create AI tools tailored to your specific teaching, research, and administrative needs. With custom GPTs, Google Gems, and Claude Projects, you can build AI assistants that understand your context, are encapsulated for a specific use case, and perhaps even shareable. The key is to develop these tools before the semester starts, when you have time to test and refine them.

Start by identifying specific use cases where AI could enhance your teaching, say. Maybe you need a feedback assistant to help grade more efficiently, a course design wizard to help plan assignments, or a tutor to guide students through complex problem-solving.

Building these tools effectively requires careful planning and testing. Your instructions need to be clear and comprehensive, your knowledge base needs to be well-organized and relevant, and you need to experiment with different approaches to find what works best.

Relevant Resources:

✨Tutorial: How to Build (Better) Custom GPTs
Google's Custom GPTs (Gems) are Here
GPT-4o is Here and GPT-4 + Custom GPTs are Now Free

5. Get Your “Dumb” Tech Sorted Out, Too

While we're all excited about AI (or maybe it’s just me…), don't forget about the basic tech tools that can make your life easier. Here are four quick wins to implement during your break:

First, set up automated shortcuts for your daily routines. Whether using Apple Shortcuts, Windows Power Automate, or Google App Script, create simple automations for repetitive tasks: launch your morning browser tabs in one click, display your daily meeting schedule at a glance, or create a pipeline from voice notes to email drafts. These small time-savers add up.

Second, declutter your digital workspace. Clean up your inbox with rules that automatically sort incoming emails (e.g., student emails by course section, department announcements, or committee work). Archive old files, organize your cloud storage, and create a logical folder structure for the upcoming semester.

Third, optimize your notifications. Most of us have our devices interrupting us hundreds of times daily. Take 30 minutes to audit your notification settings: silence non-urgent apps during teaching hours, set up VIP contacts for important messages, and create focus modes for different types of work (grading, research, meetings).

Fourth, configure your browser for efficiency. Set up bookmark folders for each course, install useful extensions like Zotero for research management or Grammarly for typo catching, and organize your tabs into groups by project or course.

Relevant Resources:

✨Tutorial: How to Use AI with Canvas LMS
✨Tutorial: Anonymize Canvas Submissions in Google Workspace
Don't Let AI Make You Forget "Dumb" Tech

6. Reflect on AI’s Role in Your Research

The AI capabilities that are transforming teaching are equally revolutionary for research. With tools like Google's Deep Research assistant (see above), ChatGPT Pro's enhanced academic reliability, and specialized research models being deployed at major institutions, now is the crucial time to consider how AI fits into your scholarly work.

We’re getting to the point that AI can help with almost any text-related tasks, including highly advanced ones like your research. You don’t want to look up in 2026 or 2030 and be woefully behind a paradigm shift in methodologies.

Consider setting up a controlled experiment with your own research: take a small piece of your current work and try processing it through different AI tools. Test Perplexity for literature review, see how ChatGPT Pro handles complex theoretical analysis in your field (or send me a sample challenge for me to do it), or explore how these tools might help with data analysis or writing.

Relevant Resources:

✨Tutorial: How to Draft Grant Applications with Gemini
10 Ways I Use LLMs like ChatGPT as Professor

7. Take a Nap

No, seriously! You deserve it.

📢 Quick Hits:
AI News and Links

1. Google launched "Deep Research," an AI research assistant in Gemini Advanced that can autonomously explore complex topics across the web and compile comprehensive reports with source links. The feature, available now on desktop and coming to mobile in 2025, represents Google's first "agentic" feature that can perform multi-step tasks under user supervision.

This is a development educators should take a close look at — and keep an eye on through 2025. I will do a deep dive on it early in the new year.

2. The launch of Deep Research is part of Google’s announcement of a new frontier model, Gemini 2.0 that is especially well-suited for "agentic" capabilities that allow the AI to take actions on users' behalf (with supervision). Key features include native image/audio output and tool integration. They're also showcasing research prototypes like Project Astra (for phones/glasses) and Project Mariner (for web browsing), though access is currently limited to “trusted testers.” 2.0 Flash, an experimental version, is available now in the Gemini app.

3. Finally, Google updated NotebookLM with three major changes: a new three-panel interface for better content management, interactive voice conversations with AI hosts during Audio Overviews, and a premium "Plus" subscription for power users that offers 5x higher usage limits. The Plus version will be included in Google One AI Premium in early 2025.

4. OpenAI announced ChatGPT Pro, a $200/month tier giving access to their most powerful models including a new more compute-intensive "o1 pro mode" (as well as higher usage limits on "o1," "o1-mini," and "GPT-4o"). The pro mode showed significantly better reliability on PhD-level science questions and coding challenges, with OpenAI announcing plans to award initial grants to medical researchers at leading institutions.

OpenAI, “Introducing ChatGPT Pro”

I went ahead and bought it for one month (for science). Please email me with any hard problems you want me to give to it from your field and I will report back on the results, unedited:

(Also, at the start of the new year, I will be discussing the growing disparities in model access that I predict are coming — as well as why there are still a lot of bright sides of the coming year of AI.)

OpenAI also released Projects, which help you organize your chats, and launched Sora Turbo, an improved version of their text-to-video model, available now on sora.com to ChatGPT Plus and Pro subscribers. The new version is significantly faster and includes features like 1080p resolution, 20-second videos, and a storyboard tool. Notably, all generated videos will include C2PA metadata for transparency and verification, with strict safeguards against misuse like deepfakes. (Be sure to check out Google’s Veo 2, too.)

5. A small study in Computers & Education found that students using ChatGPT for help with essay writing showed markedly different patterns than those working with human teachers. Students using ChatGPT tended to skip evaluating the help received and jumped straight to implementation, while also showing more "executive help-seeking" behaviors (asking for direct answers rather than guidance). The researchers suggest this indicates a need for better scaffolding to ensure students engage metacognitively when using AI tools.

6. Microsoft released Phi-4, a small LM that “excels at complex reasoning in areas such as math, in addition to conventional language processing” and like, Meta Llama 3.3, a new free open-source LLM you can run on your own computer, is comparable in performance to GPT-4. Remember: GPT-4 was released in March 2023 and was far ahead of all others for 6+ months; it’s the model that powered ChatGPT to popular awareness. (This is related to the “bright sides” I mentioned above.)

7. Google DeepMind's new AI weather forecasting model "GenCast" has achieved unprecedented 15-day forecasts, outperforming the European Center for Medium-Range Weather Forecasts (the current world leader) 97.2% of the time. Unlike traditional supercomputer forecasting, GenCast learns weather patterns from 40 years of historical data and can generate probabilistic predictions much faster. (Original paper in Nature here. Discussion in the New York Times here.)

✉️ What You, My Subscribers, Are Saying

My December 6th Webinar
on Feedback & Assessment

Two weeks ago, I hosted my last webinar of 2024 on "How to Use AI to Improve & Accelerate Student Feedback." With nearly 100 educators signed up, we explored concrete strategies for using AI to provide better feedback faster while maintaining student data privacy. The 90-minute session included live demonstrations of ChatGPT, Claude, and custom GPTs for feedback generation, along with detailed discussions of ethical considerations and implementation strategies.

The response was overwhelmingly positive, with attendees especially appreciating the practical, hands-on approach and ready-to-use prompts for different teaching contexts. As with previous webinars, all attendees received the full recording, slides, AI-generated summary, and several Premium resources focused on AI-assisted feedback and student data privacy.

Thanks to all who attended or engaged with me after viewing the recording.

Your Comments

Here are some of your comments:

Thanks for the feedback!

What Should the Next One Cover?

Now the question is:

Please tell me your answer, if you didn’t get a chance to do so last week (note: after you make a selection, you will be redirected to the website where you can also offer typed comments, if you want):

I did get a good comment/question from one attendee of the webinar who is a Premium subscriber. If I were to have a webinar on how to build custom GPTs, the person asked me whether there be a lot of overlap with my pre-existing ✨Tutorial: How to Build (Better) Custom GPTs:

In short: I am going to turn to more advanced topics more generally in 2025, in my Premium content and in my paid webinars. Free newsletters and free webinars will stay more focused on the basics and keeping everyone up to speed, but the webinar at issue here would focus primarily on topics about custom GPTs, Gems, Projects, etc. that are not covered in my pre-existing Tutorial. Topics like: API connections, workflow optimization, when to pivot to Zapier and LLMs’ APIs, etc.

This is especially true since Charlie Fuller and I are about to release a course on basic- and intermediate-level aspects of custom GPT creation (see above).

Graham

Let's transform learning together. If you would like to consult with me or have me present to your team, discussing options is the first step: Schedule Free 30-minute Exploratory Zoom Feel free to connect on LinkedIN, too!