Musings on Skill Checks

8 posts / 0 new
Last post
deadDMwalking
deadDMwalking's picture
Musings on Skill Checks

Sometimes I think about game design. Rarely, it even results in a good idea! I have good friends that have worked with me on our own D&D replacement, and it makes me happy. There are a lot of things that it does better than D&D. But because it is our creation, we're constantly tinkering. Instead of just letting my mind wander and then potentially presenting a suggestion, I thought it might be interesting to share my thought processes about what we did; why we did it; and what changes might make sense.

The Problem
In 3.x, you track individual skill points. Particularly when creating a high level character with multiple class levels, this can be needlessly time-consuming. 3.x also has characters multiple their initial skill allotment by 4 at character creation. This results in differences between otherwise identical characters (ie, a Rogue 1/Fighter 1 has many more skill ranks than a Fighter 1/Rogue 1). Tracking and optimizing for when you take each level to determine whether a skill could be purchased as a class-skill or cross-class skill is best done by advancing a character from 1st level one level at a time. Doing it right is hard; doing it any other way is wrong.

The Current Solution
In our system, characters have three possible options for skills: untrained/trained/expert. An untrained character rolls d20+Attribute+1/2 Level. A trained character receives a +4 bonus; an expert character receives a +6 bonus.

Thoughts Regarding the current Solution
One of the things that we are exploring is replacing the d20 with 2d10 (added together). That has some interesting effects on the probabilities. In both 3.x and our current system, it is possible for an expert to be +8 relative to someone untrained (ie, an untrained person might have a +3 attribute bonus; the expert using a skill that relies on their best attribute might have a +11. Considering these numbers, it is possible that the novice will outperform an expert. If the novice rolls a Natural 20 and the Expert rolls a 1, the novice has a 23 to the Experts 12. Even if the Expert took 10, he would have a 21 compared to the novice's 23. So let's do some math hammering! How likely is it?

We know that if the Novice rolls a 20 (5% of the time), he'll beat the Expert on anything below a 12. The Expert is 55% likely to roll an 11 or lower. This means that 2.75% of all skill contests result in the Novice rolling a 20 and the expert rolling low enough to fail. If the Novice rolls a 19, the Expert loses on an 10 or lower (5% * 50% = 2.50%). Skipping a bit; a roll of 10 is the lowest result that the Novice can still win; his modified 13 beats the Expert only if the Expert rolls a 1 (for a result of 12).

Novice Rolls a: Number of 'Losing Rolls' for the Expert Probability
10 1 0.25%
11 2 0.50%
12 3 0.75%
13 4 1.00%
14 5 1.25%
15 6 1.50%
16 7 1.75%
17 8 2.00%
18 9 2.25%
19 10 2.50%
20 11 2.75%

Adding all those probabilities together gives us: 16.5% of the time. That's basically one out of every six times. In fact, if the party has four members (3 novices and 1 expert), the expert is only likely to beat EVERYONE 58% of the time. That bothers me - I think the expert should be beating the novices most of the time. But what about take 10? Sure, it bothers me that the Expert could lose but if they're just doing their ho-hum average and the novice is REALLY TRYING the odds shift pretty significantly. Taking 10, the Expert always has a 21. The Novice only wins with a 19 or 20. That's a simple 10% of the time. Against his three companions, the Expert is 'winning' about 73% of the time. Definitely enough that people will notice that they're good, but they're definitely sacrificing their ability to get their 'best result'.

In our system, we allow people to 'take 10' after a roll. It costs a resource, so you can't do it all of the time, but it does address this situation. We already calculated that if the expert rolled they'd usually win even if the novice rolled a 19 or 20. 5.25% of results. Even against three companions, the expert is winning 85% of the time. So in a party of four, the expert 'leads' 5 out of 6 times. That's probably enough for it to feel like the expert is consistently performing better than the companions.

All of the above calculations are assuming a d20 with a flat probability curve. Using 2d10 the novice has approximately a 10% chance of winning against the expert. There is only a 1.74% chance of the novice rolling a 19 or 20, so if the Expert can 'take 10' after the roll, that's the odds of winning. That works out to a 95% chance of coming out ahead against three novice companions. I'm definitely liking those numbers better.

Variations
One thing I've been wondering about is if it would be better to have 4 levels of skills: untrained/trained/expert/master. We currently use diminishing returns (ie, trained is +4, expert is +6). If we followed that existing logic, Expert would only be an additional +1 for a total of +7. Alternatively, we could make Trained +4; Expert an additional +3; Master would then be a further +2 (+4/+7/+9).

Implementing that would result in an additional +1 compared to my original examples (+12 versus +3). With a 9 point difference the odds of losing to a Novice are 13.75% for d20 and 7% for 2d10.

Now, other than the bonus for the various levels of training, it seems like there should be other benefits. There are various ways to adjust the probabilities. I'm focusing on the 2d10 from here on out since that's what I care about most for my game preferences.

Rolling Additional Dice
One potentially simple way of helping adjust the odds is simply adding additional dice, and letting players take the best 2. Trained could be 3d10; expert could be 4d10; master could be 5d10. Of course, the extra dice could start with expert/master; there's some flexibility.

Treating some number of Dice as a Particular Result
Another way is to allow a level of mastery impact the ability to take 10 (either before or after the roll). For example, someone who is trained could 'take 10' or roll, but couldn't take 10 after the roll. An expert could take 10 after the roll. A master could treat one of the dice as a 10 (ie, they'd roll 10+1d10).

Allow 'take x' after the roll
I already mentioned that above, but allowing someone to treat the roll as higher than it was eliminates bad results. In our current system, we do allow it with a limited resource.

Putting it all Together

Untrained: 2d10+attribute modifiers. Characters cannot 'take 10' on a skill that they're not trained in (announced before rolling). Our system allows them to treat the roll as a 10 if they spend a limited resource.
Trained: 2d10+4+attribute modifiers. Characters can take 10 (announced before rolling). Our system allows them to treat the roll as a 10 if they spend a limited resource. If they know taking 10 will succeed, they should just take 10; they should never need to spend a resource on this roll.
Expert: Best 2 of 3d10+6+attribute modifiers.
Master: I don't need a master rank.

So if I have an unskilled novice (+3 ability modifier only) versus an Expert (+5 ability modifier and +6 expert bonus), the difference is +8 (as considered before). An untrained character will have to roll for success. The trained person could take 10 if that would be sufficient. The only thing I don't like is that the trained person won't know if taking 10 'wins' unless the untrained person goes first. That's not satisfying for me.

In any case, that's what I'm thinking about. I'm interested in feedback. I'll let it stew for a day or two myself and see if my thinking changes. I'm not sure I like an extra die roll (seems fiddly) and I'm concerned about not enough difference between Untrained/Trained - I looked at Untrained versus Expert - obviously with the smaller difference a Trained Character dominates less than an Expert.

deadDMwalking
deadDMwalking's picture

So having some more time to think about it, it's not as much about the difference between 'untrained' and 'expert' that bothers me, but between untrained and trained. Training makes a BIG difference. There are a lot of things I couldn't hope to do without training. But tracking all of the things that a PC ought to be able to do is hard. In 3.x, you can only try a 'trained skill' if the DC is 10 or less, which is somewhat limiting. And certainly you'd expect a certain amount of 'learning by osmosis'. How many times can you fight a monster before your learn the weaknesses?

In my mind, I'm thinking that there probably ought to be 'basic skills' and 'advanced skills'. I'm wondering if characters should have 'training' in 'basic skills' automatically? The package of basic skills could vary based on background. For example, literacy could be a basic skill for everyone, but if you're from a barbarian tribe, you might lose literacy as a basic skill but gain Survival. Alternatively, backgrounds could provide training in skills that represent the background.

In a lot of systems, you take a penalty for skills that you have no training in. If we said 'you take a -5 penalty to untrained skills', mathematically, it would be equivalent to increasing all of the DCs by +5 and giving you a +0 bonus. I like avoiding penalties. Rolling with a +0 feels better than rolling with a -5, even if the odds of success are the same.

MinusInnocence
MinusInnocence's picture

Re: trained vs untrained, I think in v3.5 we see plenty of examples of "basic" skills, if the definition of "basic" is "any skill someone with no formal training can use." I do like the idea of adding a few basic skills onto the untrained list to represent someone's background. Being able to use Handle Animal to deal with domesticated animals makes sense for someone who grew up on a farm or ranch, for example. I don't necessarily think the same 1st level commoner should automatically be able to train animals to do learn new tricks, however.

"Every normal man must be tempted, at times, to spit on his hands, hoist the black flag and begin slitting throats." - H.L. Mencken

Talanall
Talanall's picture

I think that 3.X basically is set up in such a fashion that it starts from "stuff anybody should be able to do" at DC 10, and then goes up to DC 15, which basically is what is what a technician or professional is expected to be able to deal with as a matter of routine. It controls for "basic" and "advanced" skills through a combination of setting the DC and setting which skills can be used untrained.

Looking at it this way, DC 11 to 15 represents a task that is impossible if you don't know anything about the area of endeavor to which it belongs, but actually no big deal if you're clued into the field. For example, planing the edges of two boards so they fit together tightly enough for a glue joint seems very difficult until you know the basic woodworking techniques used to do it. Then you realize that it's really something straightforward that anyone who calls himself a woodworker ought to be expected to know. If someone is at 1st level, has a positive ability for the skill, and takes 4 ranks in it, he is a minimally competent professional at that skill. That is, he can routinely perform tasks that will male a layman go, "Wow," and a real master scoff.

Higher DCs in the 16-20 range are representative of tasks that you can't do routinely unless you a really are a master, but this is really just a matter of having taken the Skill Focus feat in that skill. If a representative NPC has 4 ranks of a skill, a +1 bonus from the keyed ability score, Skill Focus, and a masterwork tool, he's at +10 and can do stuff up to DC 20 as a matter of routine--he's taking 10 on his rolls. This is basically where most grown-up NPCs with a little education or training are going to be, at least in campaigns I run. Something like 97% of all NPCs are first-level experts or commoners set up along these lines.

I think more thought probably could have gone into designating which skills are "trained only" for 3.X. Currently the list is Decipher Script, Disable Device, Handle Animal, Knowledge, Open Lock, Sleight of Hand, Speak Language, Spellcraft, Tumble, and Use Magic Device. It's kind of a weird list. I really wouldn't be upset if every skill except Balance, Bluff, Climb, Gather Information, Intimidate, Jump, Listen, Sense Motive, Spot and Swim were added to it.

Wæs se grimma gæst Grendel haten,
mære mearcstapa, se þe moras heold

deadDMwalking
deadDMwalking's picture

I had been planning on coming back to this.

It doesn't really bother me that a novice can beat an expert frequently (well, it does, but not as the biggest problem). Ultimately, very few skill contests are set up as opposed rolls, so it doesn't matter that sometimes a novice can get a better roll, especially if it is a 'trained only' skill so the novice can't actually succeed.

I do think 'initial training' makes a big difference in a bunch of skills. Tiling a bathroom isn't super-hard, but it isn't super-easy, either. It's easy to screw up and take lots of extra time and give up in frustration. But if you've done it once or twice, it's all easy enough once you know what you're doing.

The thing about D&D skills is that they work pretty well when you take 10 on everything - once you actually start rolling, it can get wonky. Iterative probability also causes problems. If you have a 90% chance of succeeding on your skill check, but you have to make 10 of them in a row, your odds of success are only 30%.

I'm thinking that I'm leaning toward skill based abilities that are similar to feats (but different). Speak Language is the type of skill I'm thinking of. It's not one that requires a check (normally) but if you spend a skill point on it you gain the ability. I could see something like ignoring armor check penalties as a 'skill' you could take. Characters that wear heavy armor can train to ignore the penalties (which is effectively like giving them virtual bonus skill points).

For most skills, if you have the training you need, success is never in doubt. Setting the TNs reasonably helps along those lines. I'm still mulling over setting the 'easy' TNs at 20 and giving a pretty substantial 'training bonus' (ie, +10) so with take 10, a trained character succeeds reliably; an amateur can 'get lucky' if they have a good attribute.

Talanall
Talanall's picture

If you're rolling a skill check in D&D, it usually means that you are in a situation where you don't think you could succeed by taking 10, or you're prevented from doing so by environmental factors. Either way, the "swingy" character of a d20 doesn't bother me in context.

If the DC is 20 and you have a +10 bonus to your check, then yes, you can take 10 to succeed on it. But not when someone's trying to kill you. Not when you're on a boat in the midst of a storm. Not when you're on the back of a bucking horse. I have no particular issue with the idea that someone could fail abjectly, under stress, at a task he might otherwise consider routine.

In the example of a character with a +10 modifier against DC 20, it's still a 50/50 chance that he'll succeed or fail. In context, that's actually pretty impressive; if it's a Balance check, we're talking about someone who is successfully walking a tightrope while involved in personal combat. I don't really have a problem with the idea that such an individual could basically just phone it in, outside of combat. In combat, it is and ought to be a chancy outcome.

At least by my standards, you're trying to "fix" something that is a feature, not a bug.

Wæs se grimma gæst Grendel haten,
mære mearcstapa, se þe moras heold

deadDMwalking
deadDMwalking's picture

Talanall wrote:

At least by my standards, you're trying to "fix" something that is a feature, not a bug.

I don't think it's quite that way. First off, the difference between someone who is reasonably good and who isn't isn't set in stone, but at 1st level if you have a good attribute and max ranks, you're probably at +7; if you're average you're probably at +1.

So we're talking about a 6 point difference. Let's say we're talking about a DC 15 (the trained character could make it in normal circumstances by taking 10; the untrained character couldn't make it by taking 10). This is clearly a situation where the training makes a difference - the trained character is 'good' most of the time.

So you take these two characters, put them on a tightrope, then fly by with a manticore. The low-skill person has a 35% chance of making the check; the high-skill person a 65% chance.

We could look at the results as a matrix of sorts:

Both Fall (20%)
Low Skill Person Falls, High Skill Person Succeeds (45%)
High skill person falls, Low skill person succeeds (11%)
Both succeed (25%)

These situations aren't common - in a campaign it may only come up once or twice. If you're making 10 checks, you're going to notice the difference between a skilled person and an unskilled person over time. But in a single check all of the possibilities exist. If your high skilled character falls and the low-skilled person succeeds, how do you feel? Should that really happen 10% of the time? 5% of the time? Never?

This is a matter of 'feel', so I don't know that there is a right answer. What I'm wrestling with is that in the case of two individuals, these numbers seem generally reasonable, but in most cases, there are three unskilled individuals making an attempt compared to one skilled individual. The chances of the skilled individual beating EVERY unskilled individual is in part dependent on how many people get to make checks. Rather than having the skilled individuals make the checks they are most likely to succeed at, there's an incentive to have everyone try.

If you have a Hide bonus of +10 and you're sneaking up on a group of 20 people with a Spot bonus of +0, your odds aren't very good. If you roll a 1-10, it's a near certainty that they'll collectively beat you (someone will roll a 20). If you roll an 11+ there's no chance they'll spot you. You have roughly an 87% chance of winning on a one-on-one match up, but a straight up coin-flip against a large group.

There's a few ways to potentially address that - one is making sure the bonuses for training are sufficiently large that the benefit of additional checks is somewhat mitigated (ie, a +15 versus a +0 is always at least a 75% chance of winning) or having collective skill rules that have the highest bonus person make the check with some other version of contribution (or a combination of the two).

For example, if we want our Rogue to be able to sneak up on a group of untrained people reliably 85% of the time (approximately matching a 1:1 matchup of +10 versus +0), having the group make a single roll with their highest bonus (+0) works. If sneaking up on a group is supposed to be very difficult, having the group make a single check with a +1 bonus per member (+20) changes things dramatically.

So again - a long way to say that I like the individual matchups, but I don't like the way the numbers break when you have a large group. Robin Hood should be able to beat any number of peasants in a skill-based archery competition; if the field is 400 people then one of them is likely to match his feat of hitting the bulls eye sand splitting the arrow. On an individual level, things work well - when taken to the nth power, I think the results get wonky. I'm really interested in the 'party sized space' - I think finding the numbers that help ensure that the 'specialist' succeeds consistently more often than 3 other untrained people is important, without saying 'untrained people can't even try'.

Talanall
Talanall's picture

deadDMwalking wrote:

I don't think it's quite that way. First off, the difference between someone who is reasonably good and who isn't isn't set in stone, but at 1st level if you have a good attribute and max ranks, you're probably at +7; if you're average you're probably at +1.

This is a curious set of numbers you've got here. A "good" ability score is evidently a 16 or 17. Or put another way, it uses the highest possible score from a racially modified elite array of 15 14 13 12 10 8, arranged to suit the designer of the character.

That isn't necessarily a problem for the sake of this discussion, but let's be clear that 3.X was designed with the assumption that PCs would be based on this array, and that your example is really based on a difference between a character who must expend feats to be better at a specific task, and someone who may only have a single rank in the skill, or a positive ability score, or some combination of being objectively unsuited for the task but trained to make up for his natural handicaps.

Quote:

So we're talking about a 6 point difference. Let's say we're talking about a DC 15 (the trained character could make it in normal circumstances by taking 10; the untrained character couldn't make it by taking 10). This is clearly a situation where the training makes a difference - the trained character is 'good' most of the time.

So you take these two characters, put them on a tightrope, then fly by with a manticore. The low-skill person has a 35% chance of making the check; the high-skill person a 65% chance.

We could look at the results as a matrix of sorts:

Both Fall (20%)
Low Skill Person Falls, High Skill Person Succeeds (45%)
High skill person falls, Low skill person succeeds (11%)
Both succeed (25%)

Your math's off.
Both fail: 22.75%
High succeeds, low fails: 42.25%
High fails, low succeeds: 12.25%
Both succeed: 22.75%

Quote:

These situations aren't common - in a campaign it may only come up once or twice. If you're making 10 checks, you're going to notice the difference between a skilled person and an unskilled person over time. But in a single check all of the possibilities exist. If your high skilled character falls and the low-skilled person succeeds, how do you feel? Should that really happen 10% of the time? 5% of the time? Never?

It really kind of depends on the circumstances. Why are the characters on a tightrope? Why is the manticore attacking them? Did the DM guide them into this course of action, or did they come up with it on their own? Probably I'm going to point and laugh no matter what, unless the DM somehow connived to make this look like a good idea.

Quote:

This is a matter of 'feel', so I don't know that there is a right answer. What I'm wrestling with is that in the case of two individuals, these numbers seem generally reasonable, but in most cases, there are three unskilled individuals making an attempt compared to one skilled individual. The chances of the skilled individual beating EVERY unskilled individual is in part dependent on how many people get to make checks. Rather than having the skilled individuals make the checks they are most likely to succeed at, there's an incentive to have everyone try.

Again, that really depends on a bunch of stuff. How long does the check take? How much space is there? What happens if you fail the check?

If it's a Knowledge or Spot or Sense Motive check, then sure, it makes sense to have everyone who can check do so, because there isn't really a reason not to try. But that's HOW THOSE SKILLS WORK. And success at one of those skills does not automatically translate to actionable results for the other characters in a group, especially if it happens in the heat of combat.

If it's a Balance, Climb, Jump, Swim, or Tumble check, there usually is some kind of consequence for failure. Depending on circumstances, the consequence could be anything from "you move slower" to "you plummet to your death." The skills listed above are (mostly) structured in such a way that in the majority of cases, one character in the party will be notably better than his peers at that set of tasks, so that he or she can perform the difficult check, and then contrive to make it easier/safer for his compatriots by stringing a rope or providing some other contrivance.

There's a middle ground with most other skills, in the sense that there usually is either some opportunity cost for attempting the check (it takes up an action) or there's a price for failure. For example Craft skill checks ruin some materials if you miss the DC by 5 or more; Bluff checks that fail in a social setting usually result in an NPC doing something you don't want, and they may irritate the NPC over your mendacity.

Quote:

If you have a Hide bonus of +10 and you're sneaking up on a group of 20 people with a Spot bonus of +0, your odds aren't very good. If you roll a 1-10, it's a near certainty that they'll collectively beat you (someone will roll a 20). If you roll an 11+ there's no chance they'll spot you. You have roughly an 87% chance of winning on a one-on-one match up, but a straight up coin-flip against a large group.

Again, you need context. How far away is the group? How close are you trying to get? The outcome of a Spot check is 20, modified for distance in increments of -1 for every 10 feet after the first. So if you take 10 for a result of 20, and you only need to get to be 30 feet away, that group of 20 people is never going to see you because their effective maximum for Spot checks is only 18. If you have cover or concealment all the way up to 15 feet from their position, that's basically the detection limit for this case.

The rules for determining when two sides of an encounter become aware of one another are quite a bit more complex than you're giving them credit for being; they take into account distance, intervening obstacles in the environment, and ongoing distractions in addition to the simple opposition of Hide and Spot results.

Wæs se grimma gæst Grendel haten,
mære mearcstapa, se þe moras heold