This website uses cookies

Read our Privacy policy and Terms of use for more information.

Why it matters:

Listening to students is increasingly the beginning of the work. And higher ed doesn’t struggle with ideas. It struggles with how it learns from them. The real work is testing what we think we heard quickly, visibly, and with a willingness to be wrong.

Too often, teams move from insight quickly to idea and pilot: skipping the small, fast tests that reveal whether an idea actually meet the needs of those it was designed for.

Without that step, ideas move too fast to pilot and too slow to learn. Pilots are built to show progress, testing is how you learn what’s true.

If we want to truly address challenges in the student experience, we need to get better at learning before we scale. Design-driven change depends on closing that gap.

Go deeper:

Working with a number of education organizations over the last 2 years, the most common challenge I encounter isn’t about listening to students or even pursuing infrastructure to spend more time identifying and ideating against student needs.

The problem, I’ve found, is in the dreaded P word: Pilot.

A team does the work. Real work: time spent with students, or employers, or alumni. They’ve heard things that are a little uncomfortable and clarified pain points. They’ve generated ideas that address the needs they’ve heard and identified. Not perfect ones, but ones grounded in real experiences.

Then the tension of the time spent sinks in. The urgency for action surfaces. Many times the obligation and accountability to funding and the responsibility to respond starts to press for progress. And to be clear, it’s not wrong. It’s coming from a good place.

There’s usually some mix of:

  • we’ve already invested a lot into this, we should move

  • this feels promising, we don’t want to lose momentum

  • we need to show progress, not just talk about ideas

All of that is real. But there’s something else in there too that we don’t always name, which is that testing…actual testing, feels uncomfortable in a different way.

It’s less polished, less official, less confident. A little harder to explain to someone who just wants to know “what you’re doing and when it will be done.” Especially when the tests feel small, or obvious, or time intensive for the perceived payoff.

…But it’s extremely important. Especially when your idea is still just an idea, and whether it’s a good one is more dependent on the response from users than any one person or leaders opinion.

This is the moment I spend a lot of time with teams on; helping them slow down just enough to test what matters before they commit to building something bigger.

So we move past that tension pretty quickly. We give in to the pressure to act. And we skip the part that would have actually helps us learn.

What’s an experiment?

The beauty of testing is that it takes many forms, and their goal is smaller and more incremental than you think.

Your testing starts with the concepts (slightly developed ideas) that you’ve generated. Inside of those concepts are dozens of assumptions that you’ve made about how the world works, what your user needs, how they will respond, and what problem your solution is focused on.

When you’re ready to test a concept through an experiment, you start by naming those assumptions and then ranking the ones that are highest impact and most uncertain.

Then, you’ll design an experiment to confront those assumptions, one by one:

IF WE… (ACTION TO TEST THE ASSUMPTION)

THEN… (HYPOTHESIZED RESULT OF THE EXPERIMENT)

WHICH WE WILL MEASURE BY… (HOW WILL YOU KNOW IF YOU WERE SUCCESSFUL)

OUTCOMES OR GOAL…(MINIMUM THRESHOLD FOR SUCCESS)

And once you realize the momentum and energy this brings to a team, it becomes a lifestyle. Ironically, you’ll feel that momentum of progress you desire in a pilot but often lose by the time it’s ready to launch.

Over time, that testing of the most important assumptions builds confidence and insights that move teams in the direction of user needs.

And more importantly, you’ll see opportunities to experiment and test assumptions everywhere. While listening is a critically important muscle in good design, it only results in change if that learning is responsive, adaptive, creates, and tests new approaches.

This idea was on my mind this week because of how often it shows up in my work:

Before making slides, test the title: Before I’ve formally outlined a keynote, I’ll throw out a title and see how the partner reacts to it. It forces me to clarity on my core ideas and relevance to my stakeholders.

Before the website, test the idea. Then only build the next most important part: I recently posted about ASU+GSV coming to San Diego and asked who was planning to be in town. That turned into a handful of conversations, which turned into a loose idea to put together a local guide, which I’ve now been slowly building as people send recommendations or ask what they should do while they’re here. It even evolved into an experiment around whether local school leaders would want to be listed as “tour sites.” It was just putting something small out there and seeing what came back.

Creating and asking for feedback creates unexpected results: A design student I’m coaching at dtech high school is creating a low cost music experience for students. In testing, she discovered that parents were more likely to choose activities that kept their young kids occupied longer. When kids only played for a few mins and then moved on, she realized she needed to augment the experience with her prototype to attract more time and attention. A few stickers and art supplies to help “personalize” the child’s new cardboard guitar suddenly turned a failed instrument experiment into a multidimensional art project that builds connection to the instrument and gave parents a few mins of uninterrupted progress.

Pilots imply confidence we often don’t have (and require more learning)

Learning clarifies. That’s what helps a team move with confidence. And I think this is where higher ed gets itself into trouble. Not because of a lack of ideas, there are plenty of those. It’s how we try to learn from them.

We love a pilot. But most pilots I see are doing the wrong job. They’re set up to prove something works. And once you’ve framed it that way, you’ve already made it harder to learn anything useful.

And once you call something a pilot, a few things tend to happen:

  • tests too many things at once

  • too big to change once it’s in motion

  • too visible to let it fail quietly

  • too late in the process to ask basic questions about whether the idea makes sense in the first place

So teams end up protecting the idea instead of interrogating it. Which makes total sense. There’s time, budget, reputation all wrapped up in it. We want to solve the problem in one shot. We want to identify the problem, clarify the needs, identify potential responses, and have picked the right one to win.

And as a result of piloting, we miss the chance to learn about the small things that matter most when it’s cheap, fast, and still flexible. Testing is a different posture.

  • It starts with a much simpler question: what needs to be true for this to work?

  • What would we need to learn to know we’re even on the right track?

And then you build the smallest possible way to explore that: put that small test in front of the people who actually matter, and you pay attention to what happens.

Tradeoffs of Testing: Slow and Small Now = Faster, Soon

That kind of testing doesn’t always look like progress from the outside. It can feel slow, even though it’s actually faster. It can feel messy, even though it’s actually more disciplined.

What worries me most is the response that we “don’t need to test” because “we know.” Not because I believe they’re wrong, but because testing is vulnerable. It requires a willingness to be wrong early to learn and be right when it matters later.

But when we believe that “we know” what the answer is going to be, it puts the stakes even higher when you build on top of that potentially false assumption with a bunch of other layers to a piloted idea:

  • What if the students that would have shown up aren’t available when you offer the program? What if they’re not showing up because it’s not what they need?

  • What if the program you’re building exists somewhere else and just needs a partner? Or what if partners aren’t willing to be part of it in it’s current form?

  • What if the idea requires infrastructure that needs more validation before it wins over resources? Or worse, what if it can’t sustain funding without outside help?

Testing pushes us to confront the uncomfortable constraints, miscalculations, and uncertainties in the ideas that we create.

And alongside deliberate feedback from leaders, we can calibrate whether we’re headed in a direction that will ultimately be successful more quickly.

Too often, pilots are treated like a formality we pass through on our way to confident scaling. But testing is where you figure out whether the idea you’re excited about is actually connected to the problem you set out to solve. It’s where you prioritize what matters most. It’s where you avoid building something impressive that no one really needs. Pilots still have a place. They just come later.

After you’ve done enough testing that you’re not guessing anymore, you’re making an informed bet that’s also helped you drive change.

Design-driven change takes place through these tight learning cycles, with clear feedback loops in a deliberate, disciplined way.

And those small adaptations help teams evolve towards stronger student-centered listening, testing, and iteration while changing internal culture.

That’s the work. Not just building better ideas, but building the habit of learning what actually works.

One More Experiment (for today)

ASU+GSV is coming back to San Diego in a few weeks, and I’ve been asking my network who is planning to come. I’m grateful to be joining this year as part of the Project Next Board and I’m looking forward to making the most of the time when so many in my network are in the city I call home.

If you’ll be there, drop a comment in this post or reach out to find a time to connect. If you’re interested, I’d love to give you a copy of the toolkit I use with teams to design experiments and walk you through the process I use.

I’d genuinely love to hear what you’re working on, what you’re testing, what you’re unsure about. Those are usually the most interesting conversations anyway.

I’ve been slowly building that local guide I mentioned: from ASU+GSV HHs to places to get outside, see some SoCal beauty, tour a local school, or even play some golf.

Take a look at my ASU+GSV locals guide, and reach out if it’s helpful, or if there’s something missing you’d like me to include.

If this resonates, share it with someone building student-centered change. Share with colleagues or leaders you trust dedicated to driving change in education.

In learning,

Thank you for being part of this community of over 1,400 curious, creative, changemakers who believe we can improve the student experience through design.

Brian LeDuc is the founder of Learning, Designed, where he serves as a design strategist partnering with learning organizations to improve the learner experience and drive change co-designed with students, educators, communities, and employers.

Want to collaborate? I’d love to help your organization drive change using design through projects, keynotes, design sprints, workshops, and coaching. Learn more.

Reply

Avatar

or to participate