Consideration for changing data types

Backward and forward compatibility

Choosing a data-type is a long term decision, as preserving it means keeping backward and forward compatibility. Backward compatibility means a later version can read data from an older version. Froward compatibility means an older version can read data from a newer version.

So choosing the right data type should be done with some planning and considerations concerning possible future changes and expansions. Adding new fields will not break forward compatibility, but of course the older version cannot handle this new data - it will simply be ignored (unless configured to trigger an error).

Nevertheless, it might be necessary to change data types later. Expanding an enum with new values is the simplest way, and will not cause any problems and does not require any special consideration.

But changing or expanding a datatype to a higher number of bits is not always possible, and needs some consideration. See below what kind of type changes are possible without breaking backward compatibility. Forward compatibility will always only work if the provided values are still in the old range.

Fixed Data Types (fixed8..fixed64,sfixed8..sfixed64,float,double)

change item	change options
storage type	fixed length of bits according to type
range expansion	not possible
type change	no compatbile types

Unsigned Integer Types (uint8..uint64)

change item	change options
storage type	variable length
range expansion	expansion to higher number of bits possible
type change	for uintxx possible to intyy with xx<yy

Regular Integer Types (bool, enums, int8..int64)

change item	change options
storage type	variable length, negative numbers always require 10 bytes
range expansion	expansion to higher number of bits possible
type change	to uintxx possible if negative values were never used

Signed Integer Types (sint8..sint64)

change item	change options
storage type	variable length, negative numbers encoded efficiently
range expansion	expansion to higher number of bits possible
type change	no compatible types

String Type (string)

change item	change options
storage type	variable length prefix followed by data
type change	to bytes possible

Bytes Type (bytes)

change item	change options
storage type	variable length prefix followed by data
type change	no compatible types

Message Types (message)

change item	change options
storage type	variable length prefix followed by data
type change	addition of new fields possible

Repeated Types

Packed repeated types store the related ID just once for all elements instead of storing pairs of id and element. That makes packed repeated types more efficent than regular repeated types, but also incompatible to updating from non-repeated types. I.e. regular fields can be changed from optional to repeated, but not to a packed repeated type.