For completion, the remaining |Y-p| cases for determining with 95% certainty that the underlying theoretical probability is not 50% (ie. p != 0.5). This was done by laboriously going through cumulative binomial distribution calculations, where the normal approximation is no longer valid.

|Y-p| = 0.20 --> n > 29

|Y-p| = 0.25 --> n > 19

|Y-p| = 0.30 --> n > 13

|Y-p| = 0.35 --> n > 10

|Y-p| = 0.40 --> n > 8

|Y-p| = 0.45 --> n > 7

Within a generic four-five hour DnD game with a few dozen or so d20 rolls made, one would certainly notice a bonus/penalty of 5 or greater to the d20 rolls.

|Y-p| = 0.20 --> n > 29

|Y-p| = 0.25 --> n > 19

|Y-p| = 0.30 --> n > 13

|Y-p| = 0.35 --> n > 10

|Y-p| = 0.40 --> n > 8

|Y-p| = 0.45 --> n > 7

Within a generic four-five hour DnD game with a few dozen or so d20 rolls made, one would certainly notice a bonus/penalty of 5 or greater to the d20 rolls.