Writing Modules - zmap/zmap GitHub Wiki
ZMap can be extended to support different types of scanning through probe modules and additional types of results output through output modules. Registered probe and output modules can be listed through the command-line interface:
--list-probe-modulesLists installed probe modules--list-output-modulesLists installed output modules
Output Modules
ZMap output and post-processing can be extended by implementing and registering output modules with the scanner. Output modules receive a callback for every received response packet. While the default provided modules provide simple output, these modules are also capable of performing additional post-processing (e.g. tracking duplicates or outputting numbers in terms of AS instead of IP address)
Output modules are created by defining a new output_module struct and registering it in output_modules.c:
typedef struct output_module {
`const char *name; // how is output module referenced in the CLI`
`unsigned update_interval; // how often is update called in seconds`
`output_init_cb init; // called at scanner initialization`
`output_update_cb start; // called at the beginning of scanner`
`output_update_cb update; // called every update_interval seconds`
`output_update_cb close; // called at scanner termination`
`output_packet_cb process_ip; // called when a response is received`
`const char *helptext; // Printed when --list-output-modules is called`
} output_module_t;
Output modules must have a name, which is how they are referenced on the command-line and generally implement success_ip and oftentimes other_ip callback. The process_ip callback is called for every response packet that is received and passed through the output filter by the current probe module. The response may or may not be considered a success (e.g. it could be a TCP RST). These callbacks must define functions that match the output_packet_cb definition:
int (*output_packet_cb) (
ipaddr_n_t saddr, // IP address of scanned host in network-order
ipaddr_n_t daddr, // destination IP address in network-order
const char* response_type, // send-module classification of packet
int is_repeat, // {0: first response from host, 1: subsequent responses}
int in_cooldown, // {0: not in cooldown state, 1: scanner in cooldown state}
const u_char* packet, // pointer to struct iphdr of IP packet
size_t packet_len // length of packet in bytes
);
An output module can also register callbacks to be executed at scanner initialization (tasks such as opening an output file), start of the scan (tasks such as documenting blacklisted addresses), during regular intervals during the scan (tasks such as progress updates), and close (tasks such as closing any open file descriptors). These callbacks are provided with complete access to the scan configuration and current state:
int (*output_update_cb)(struct state_conf*, struct state_send*, struct state_recv*);
which are defined in output_modules.h. An example is available at src/output_modules/module_csv.c.
Probe Modules
Packets are constructed using probe modules which allow abstracted packet creation and response classification. ZMap comes with two scan modules by default: tcp_synscan and icmp_echoscan. By default, ZMap uses tcp_synscan, which sends TCP SYN packets, and classifies responses from each host as open (received SYN+ACK) or closed (received RST). ZMap also allows developers to write their own probe modules for use with ZMap, using the following API.
Each type of scan is implemented by developing and registering the necessary callbacks in a send_module_t struct:
typedef struct probe_module {
const char *name; // how scan is invoked on command-line
size_t packet_length; // how long is probe packet (must be static size)
const char *pcap_filter; // PCAP filter for collecting responses
size_t pcap_snaplen; // maximum number of bytes for libpcap to capture
uint8_t port_args; // set to 1 if ZMap requires a --target-port be
// specified by the user
probe_global_init_cb global_initialize; // called once at scanner initialization
probe_thread_init_cb thread_initialize; // called once for each thread packet buffer
probe_make_packet_cb make_packet; // called once per host to update packet
probe_validate_packet_cb validate_packet; // called once per received packet,
// return 0 if packet is invalid,
// non-zero otherwise.
probe_print_packet_cb print_packet; // called per packet if in dry-run mode
probe_classify_packet_cb process_packet; // called by receiver to classify response
probe_close_cb close; // called at scanner termination
fielddef_t *fields // Definitions of the fields specific to this module
int numfields // Number of fields
} probe_module_t;
At scanner initialization, global_initialize is called once and can be utilized to perform any necessary global configuration or initialization. However, global_initialize does not have access to the packet buffer which is thread-specific. Instead, thread_initialize is called at the initialization of each sender thread and is provided with access to the buffer that will be used for constructing probe packets along with global source and destination values. This callback should be used to construct the host agnostic packet structure such that only specific values (e.g. destination host and checksum) need to be be updated for each host. For example, the Ethernet header will not change between headers (minus checksum which is calculated in hardware by the NIC) and therefore can be defined ahead of time in order to reduce overhead at scan time.
The make_packet callback is called for each host that is scanned to allow the probe module to update host specific values and is provided with IP address values, an opaque validation string, and probe number (shown below). The probe module is responsible for placing as much of the verification string into the probe, in such a way that when a valid response is returned by a server, the probe module can verify that it is present. For example, for a TCP SYN scan, the tcp_synscan probe module can use the TCP source port and sequence number to store the validation string. Response packets (SYN+ACKs) will contain the expected values in the destination port and acknowledgement number.
int make_packet(
void *packetbuf, // packet buffer
ipaddr_n_t src_ip, // source IP in network-order
ipaddr_n_t dst_ip, // destination IP in network-order
uint32_t *validation, // validation string to place in probe
int probe_num // if sending multiple probes per host,
// this will be which probe number for this
// host we are currently sending
);
Scan modules must also define pcap_filter, validate_packet, and process_packet. Only packets that match the PCAP filter will be considered by the scanner. For example, in the case of a TCP SYN scan, we only want to investigate TCP SYN/ACK or TCP RST packets and would utilize a filter similar to tcp && tcp[13] & 4 != 0 || tcp[13] == 18. The validate_packet function will be called for every packet that fulfills this PCAP filter. If the validation returns non-zero, the process_packet function will be called, and will populate a fieldset using fields defined in fields with data from the packet. For example, the following code processes a packet for the TCP synscan probe module.
void synscan_process_packet(const u_char *packet, uint32_t len, fieldset_t *fs)
{
struct iphdr *ip_hdr = (struct iphdr *)&packet[sizeof(struct ethhdr)];
struct tcphdr *tcp = (struct tcphdr*)((char *)ip_hdr
+ (sizeof(struct iphdr)));
fs_add_uint64(fs, "sport", (uint64_t) ntohs(tcp->source));
fs_add_uint64(fs, "dport", (uint64_t) ntohs(tcp->dest));
fs_add_uint64(fs, "seqnum", (uint64_t) ntohl(tcp->seq));
fs_add_uint64(fs, "acknum", (uint64_t) ntohl(tcp->ack_seq));
fs_add_uint64(fs, "window", (uint64_t) ntohs(tcp->window));
if (tcp->rst) { // RST packet
fs_add_string(fs, "classification", (char*) "rst", 0);
fs_add_uint64(fs, "success", 0);
} else { // SYNACK packet
fs_add_string(fs, "classification", (char*) "synack", 0);
fs_add_uint64(fs, "success", 1);
}
}