What are range and overflow checks (and errors) in Pascal - michaliskambi/modern-pascal-introduction GitHub Wiki

  • What are Range check errors? What does {$R+} do?
  • What are Overflow errors? What does {$Q+} do?

Overflow and range check errors are, to put it simply, bugs.

  • Overflow (checked by {$Q+}) is when you make an arithmetic operation that results in something outside of the range of the allowed value. For example,

    • Declare A: Int64
    • Set A := High(Int64) - 100
    • Calculate A := A + 200.

    The result is High(Int64) + 100, it cannot fit within the Int64 range. So it makes an overflow error (if compiled within a code that has {$Q+} turned on).

    See the complete code showing this error: https://gist.github.com/michaliskambi/ea4399e9aeb9f23858d22dc58b1391d9

  • Range check error (checked by {$R+}) is when you access the array with an index outside of the allowed range. For example,

    • Declare Arr: array[0..10] of something
    • Access (read or write) Arr[11] or Arr[-1].

  • To confuse matters a bit, some situations that I described above as an overflow error (checked by {$Q+}) are reported by FPC as a range check error (checked by {$R+}).

    For example, repeat my 1st example with Byte (8 bit unsigned integer):

    • Declare A: Byte
    • Set A := 100
    • Calculate A := A + 200.

    The result in 300, as Byte can only hold values in the range 0..255. So this is an error. However, it is reported as a range check error, not an overflow error (and it's affected by whether you used {$R+}, not {$Q+}). The reason for this inconsistency is somewhat internal to how FPC works.

    <internal details> FPC calculates internally using "native" integer size (32 or 64 bit on most of our modern machines), and only an error at this point (at 32-bit or 64-bit calculation) causes overflow. An error later, because the result doesn't fit in Byte range, causes a range check error. See Jonas comments at https://bugs.freepascal.org/view.php?id=28384 . </internal details>

    Fortunately, in most use cases you enable (or disable) both overflow and range checking. So it doesn't matter much which check ($Q or $R) catches which error (overflow at arithmetic operation, or accessing outside of array bounds).

    In the discussion below, I talk about overflow errors to mean all the errors when the calculated result doesn't fit in your ordinal range (regardless if it's Int64, Integer or Byte, and regardless if it's reported under {$Q+} or {$R+}).

Both these errors are detected in debug mode (e.g. in Castle Game Engine build tool debug mode, or default Lazarus debug mode), and cause an exception then. But this detection takes time (quite significant in some cases, e.g. CGE ray-tracer is 1.9 x slower in debug mode). So these checks are turned off in release mode. In release mode we simply assume you will not make these errors.

What will actually happen if these errors occur in release mode?

  • An (unchecked) overflow means that the integer value will "wrap" around zero. It's easy to calculate what this means for unsigned values, using an arithmetic "modulus Something". For example when A is Byte and A = 200 and you do A := A + 100 then A is now equal 300 mod 256 = 300 - 256 = 44. For signed values, you need to understand how negative integers are encoded in memory: https://en.wikipedia.org/wiki/Two%27s_complement . In any case, the result is probably something unexpected (you added something positive to A, but the resulting A may be smaller) and can cause any logical bugs later.

  • An (unchecked) range error, when you access something outside the array bounds, is even more nasty. You are reading outside of the memory than you're supposed to.

    1. Maybe you're reading a memory currently allocated but unused, holding garbage (random garbage, that may even change from program execution to execution).
    2. Maybe you're reading a memory currently allocated and used by another of your variables. So you read or write an unrelated, "innocent" variable, treating it's memory (or part of it's memory) as a different type. This can lead to many nasty bugs later. And it can be used as a security hole in special cases: an attacker can deliberately cause this error to read some adjacent memory location (e.g. containing user's saved password).
    3. Maybe you're reading outside of the allocated memory. In this case you get SEGFAULT (a Unix name for Pascal's EAccessViolation exception).

    Note that by "allocated" above I mean "allocated from the OS", using e.g. malloc from libc on Unix. This does not mean that it's actually allocated by a Pascal routine (like GetMem or just by creating an instance of some class). The Pascal memory manager often has a pool of memory allocated from the OS, and only a subset of this pool is currently used. That's why above we have an extra case when the memory is allocated but unused.

To summarize: you should write your code such that it never causes an overflow or range check error. These mistakes cause clear exceptions in "debug" mode to help you with this (so that these errors don't go unnoticed). So you have to find in code what and why causes an overflow or range check error, and properly secure from it.

Only in rare cases (when you want an addition to "wrap" around), it may be acceptable to turn off these checks. Disable them only for as-small-as-possible piece of code. You can do it e.g. like this:

{$ifopt R+} {$define WAS_DEFINED_R} {$endif}
{$ifopt Q+} {$define WAS_DEFINED_Q} {$endif}
{$R-}
{$Q-}

A := A + 500; // let this addition wrap around the A range

{$ifdef WAS_DEFINED_R} {$R+} {$endif}
{$ifdef WAS_DEFINED_Q} {$Q+} {$endif}
// cleanup symbols
{$undef WAS_DEFINED_R}
{$undef WAS_DEFINED_Q}

In FPC, you could do this a little easier by {$push} and {$pop} compiler directives.

In Castle Game Engine, do {$include norqcheckbegin.inc} and {$include norqcheckend.inc} for this purpose.