I would like to have a per-user limit for the maximum size of messages that can be sent.

The simplest way to do this is to put something in a system filter along these lines:

if $message_size is above
  "${lookup{$sender_address}lsearch{/some/file}{$value}{10M}}"
then
  fail "Message is larger than $sender_address is allowed to send"
endif

In practice, an additional check that the message has arrived from your local host or local network is probably wise because sender addresses are easily forged.