A framework for Mind Club problem design

When I started writing Mind Club problems, I had a strong gut feeling about what I was trying to achieve. I wanted problems that were less about testing knowledge/specific skills and more about practicing problem solving and critical thinking more broadly.

I expected these to be different to problems you encounter at school or university, broader than logic problems, and different to those found on IQ tests and interviews for consulting firms.

I decided to not overthink it and write the first two problem sets based on my intuition. I think it was a good start, but I'd like to refine my approach and inform it by more than just my gut.

So I started digging into problem design and discovered some very interesting work by an education researcher, David H. Jonassen. In his 2000 paper Toward a design theory of problem solving, he identifies 11 problem types.

Problem Types

There is a lot of nuance, but at a high level:

🔗 Logical problems are abstract puzzles where the method must be discovered; the challenge is finding the most efficient sequence of moves or manipulations

🪜 Algorithmic problems are where a known procedure or formula is applied to produce a correct answer; the challenge is selecting and executing it correctly

📖 Story problems are narratives that embed a procedure; the challenge is comprehending the context, identifying the relevant variables, and selecting the right algorithm to apply

📏 Rule-using problems are ones where multiple rules and solution paths are possible; the challenge is selecting and applying the right rules to produce a system-constrained answer

⚖️ Decision-making problems require selecting one option from a set of alternatives by identifying benefits and limitations, weighing options, and justifying the choice

🔧 Troubleshooting problems involve a closed system with one or more faults; the challenge is diagnosing the fault state by generating and testing hypotheses against known fault signatures

🩺 Diagnosis-solution problems involve a more open system with faults and numerous optional treatments; the challenge is selecting, evaluating, and justifying the best remedy

♟️ Strategic performance problems are real-time, complex situations where tactics must be applied to meet an ill-structured strategy while maintaining situational awareness as conditions shift

🏛️ Case analysis problems are ones where an existing situation must be interpreted with incomplete information and no consensual answer; the challenge is building a defensible argument from the evidence available

🏗️ Design problems are ones where something new must be created from a vague goal with few constraints; the challenge is first imposing structure on the problem, then producing a solution that can only be judged as better or worse

🌀 Dilemmas are situations where perspectives are irreconcilable and every solution involves compromise; there is no satisfying answer, only an articulated preference with justification

Application to Mind Club

It seemed like many of these problem types could be applicable to Mind Club, with the exception of Troubleshooting, Diagnosis, and Strategic Performance. This is because Troubleshooting, Diagnosis, and Strategic Performance are more about interacting with the physical world, rather than thinking with pen and paper. They also often require deeper domain expertise, which doesn't suit Mind Club, as I want to make it accessible to people from different career backgrounds.

Before trying to apply this taxonomy as a design guide, I wanted to see how existing Mind Club sessions scored against those 11 types. So I used Claude Code to evaluate each problem according to the criteria in the paper and present the scores on a heat map.

Session 1

It was interesting to see that in the first session, the problems leaned heavily towards Algorithmic and Story.

Perhaps not surprising given my background designing problems in the area of theoretical physics.

One piece of useful feedback I got in the first session was that some of the problems felt like helping their kids with HW!

Session 2

Based on this feedback, I adapted the problem types in the second session. I was happy to see that reflected in the heat map, where there is much less concentration in Algorithmic and Story columns.

I was surprised to see such a large representation of logic type problems. This was not deliberate.

Overall trends

As expected, there was not much representation from Troubleshooting, Diagnosis, and Strategic Performance in either session.

But it was interesting to see that in both sessions, Dilemmas were not well-represented either. This seems like a very interesting problem type to explore.

Session 3

I’m now using this to guide the problems I’m writing for the third session. It’s helping me decide which problems to keep and how to arrange them. These are the ones I have so far.

I think this will lead to a better mix of problems that target the desired areas and help me adapt better to feedback from folks who attend the sessions.

If you're interested in attending a session, upcoming events are on Luma. Full problem sets from past sessions are available in the shop. Or if you want to start light, you can sign up for Mind Club Weekly and receive one problem a week in your inbox.