How can I lookup data from a single file using both single IP addresses and IP address blocks as keys? I want to set smtp_accept_max_per_host by this means, and also include a default.

You cannot do this in a single lookup, because you need separate lookups for individual addresses and address blocks. However, these lookups can be nested in a single expansion string. For example, suppose you are using an lsearch file with entries like this:

192.168.34.35:    4
192.168.34.0/24:  2
*:                1

You can use this setting:

smtp_accept_max_per_host = \
  ${lookup{$sender_host_address}lsearch{/path/to/file}\
  {$value}\
  {\
  ${lookup{${mask:$sender_host_address/24}}lsearch*{/path/to/file}}\
  }}

Note that the first lookup does not have an asterisk on the search type. If you have blocks of different sizes (/24, /26, etc) you have to configure it to do a separate lookup for each size, with just the final one using a default.