Drivers

The soft deadline for Drivers is Monday, February 24th. The deadline for attending a meeting with your mentor is Tuesday, March 4th.

4.1 Introduction

Now that you have processes and threads working properly, you can begin writing the device drivers for terminals, disks, and the memory devices /dev/zero and /dev/null. The code you will write for this part is in the drivers/ directory. There are two different types of devices: block devices and character devices. Character devices are designed to transmit or receive data one character at a time, while block devices operate by handling fixed-size blocks of data. The disk devices you will be working with are block devices, while the terminals and memory devices are character devices. These are similar because they share the same interface, but there are significant differences in their implementation. In order to make the relationships between devices easier to manage we use some magic.

Remember to turn the DRIVERS project on in Config.mk and make clean your project before you try to run your changes.

Here's a quick introduction to the code structure of this assignment:

• Files you need to modify
  • drivers/disk/sata.c - a block device that handles read and write to disks
  • drivers/tty/ldisc.c - line discipline that handles interaction with terminals
  • drivers/tty/tty.c - tty implementation
  • drivers/memdevs.c - memory devices (/dev/zero and /dev/null)
• Other files (you shouldn't have to modify these)
  • drivers/tty/vterminal.c - virtual terminal implementation
  • drivers/blockdev.c - block devices (reads and writes are in terms of blocks of bytes)
  • drivers/chardev.c - character (byte) devices
  • drivers/keyboard.c - implementation of keyboard interaction
  • drivers/pcie.c - PCIe or peripheral component interconnect express
  • drivers/screen.c - code that actually writes stuff to the screen

4.2 Object-Oriented C

As you look through the struct definitions of the different devices in the header files, you should notice that the structs for more specific types of devices (e.g. ttys and disks, referred to as the sub-struct from here on) contain the structures for the generic devices (e.g. block and character devices, referred to as the superstruct) as fields. These are not pointers; they occupy memory that is part of the sub-struct. This way, we can use memory offsets to get a pointer from substruct to super-struct, and vice versa. So, given a pointer to the struct of a char device which you know is a terminal device, just subtract the size of the rest of the terminal device struct and you have a pointer to the beginning of the terminal device struct. There is a macro provided for the purpose of doing these pointer conversions called CONTAINER_OF. In many cases, a more specific macro is defined using CONTAINER_OF which converts from a super-struct to a specific sub-struct (for an example, see bd_to_tty in drivers/tty/tty.c).

You should also notice that one of the fields in the device structs is a pointer to a struct containing only function pointers. The generic device types (e.g. block and character devices) each specify that they expect certain operations to be available (e.g. read() and write()). This function pointer struct contains pointers to the functions which implement the expected operations so that we can perform the correct type of operation without actually knowing what type of device struct we are working with. The definitions of these function pointer structs are in the C source files of their respective types. Essentially, we are manually implementing a simple virtual function table (which is how C++ implements polymorphism).

4.3 TTY Device

Each tty device is represented by a tty_t struct. A tty consists of a driver and a line discipline. The driver is what interfaces directly with the hardware (keyboard and screen), while the line discipline interfaces with the user (by buffering and formatting their tty I/O). In Weenix, the main purpose of the tty subsystem is to glue together the low level driver and the high level line discipline. The advantage of this is that when a program wants to perform an I/O operation on a tty, it does not need to know the specifics of the hardware or the line discipline. All of these implementation details are abstracted away and dealt with by the functions in drivers/tty/tty.c.

Once you have a working virtual file system (after VFS) you will access the terminals through files, specifically in the files /dev/tty0, /dev/tty1, etc. Then you can read and write to the terminals using the do_read and do_write functions. Until then, you will need to use the chardev_lookup function to get devices and then explicitly call the device’s read/write functions in order to test your code. A convenient way to do this is by using the kernel shell. For more details, see the testing section at the end of this chapter.

4.3.1 Line Discipline

The line discipline is the high-level part of the tty. It provides the terminal semantics you are used to. Essentially, there are two things the line discipline is responsible for – buffering input it receives and telling the tty what characters to print. Buffering input is what allows users to edit a line in a terminal before pressing enter, or, as we call it, cooking the buffer. The buffer for a line discipline is split into two sections, raw and cooked, so that the buffer can be filled circularly. Before a circularly-contiguous segment of the buffer is cooked, the user is able to edit it by editing the current line of text on the screen. When the user presses enter, that segment of the buffer is cooked, a newline is echoed, and the text becomes available to the running program via the read() system call. This is why read() does not return until it receives a newline when reading from a terminal. For simplicity, do not store more input than you can put in the primary buffer (even though you could theoretically use the buffer that the waiting program provided as well).

The other important job of the line discipline is telling the tty which characters to echo. After all, when you type into a terminal, the characters you press appear on the screen. From the user’s perspective, this all happens automatically. From your perspective (you being a kernel hacker), this must be done manually from the tty and the line discipline. The line discipline is also responsible for performing any required processing on characters which will be output to the tty via the write() system call. In Weenix, the only characters that are not just echoed are the newline, backspace, Ctrl+D, and Ctrl-C characters.

4.3.2 TTY Driver

The tty driver is the low level part of the tty, and is responsible for communicating with hardware. When a key is pressed, the appropriate tty driver is notified. If a handler was registered with the driver via the intr_handler function, the key press is sent off to the handler. In our case, the tty subsystem registers the keyboard_intr_handler function with the driver, which calls the line discipline’s tty_receive_char_multiplexer method. Then, after any high level processing (some of which you will be handling in the section above), any characters which need to be displayed on the screen are sent directly to the driver via the vterminal_write function.

The tty driver is already implemented for you. For anyone feeling adventurous, feel free to take a look at drivers/keyboard.c, drivers/screen.c, and drivers/tty/vterminal.c.

4.4 Disk Device Driver

A block device struct is associated with each disk device. This structure contains information the device ID and the block device operations. In Weenix, you can assume that all disk blocks are page-sized (4096 bytes). We have defined the BLOCK_SIZE macro in blockdev.h for you to be the same size as the size of a page of memory; use it instead of a hard-coded value. The functions that you will be writing are sata_read_block and sata_write_block (which are the block device operations), these are the routines that will send a command to the host bus adapter (HBA), to initiate a disk operation. These routines will be used when getting a particular block of a file that is stored on disk, for instance (this will become more relevant for S5FS).

4.5 Memory Devices

You will also be writing two memory devices, a data sink and source. These will not really be necessary until VFS. Still, these fit well with the other device drivers so they are included in this part of the assignment. If you have played around with a Linux/UNIX machine, you might be familiar with /dev/null and /dev/zero. These are both files that represent memory devices. Writing to /dev/null will always succeed (the data is discarded), while reading from it will return 0 immediately (i.e., not read anything). Writing to /dev/zero is the same as writing to /dev/null, but reading any amount from /dev/zero will always return as many zero bytes (’\0’) as you tried to read. The former is a data sink and the latter is a data source. The low level drivers for both of these will be implemented in drivers/memdevs.c.

4.6 Testing

As always, it is important to stress test your terminal code. We have provided a testing file, test/driverstest.c, where we have written one test for you. We recommend that you test the following cases (note that this is not an exhaustive list):

• Test out different scenarios for your line discipline code: typing a character, deleting a character, filling the buffer, typing special characters (ETX, EOT, etc.)
• Make sure that, if the internal terminal buffer is full, Weenix cleanly discards any excess data that comes in.
• Have two threads read from the same terminal, which will cause each thread to read every other line. If you’re not sure why this is, ask. To get the character device that corresponds to the tty (that thread is reading from), using chardev_lookup, after which you can use the character device's cd_ops for reading.
• Ensure that you can have two threads writing to the same terminal.
• Make sure that you can have multiple threads reading, writing, and verifying data from multiple disk blocks. To test your disk code, you will need to use blockdev_lookup, with the argument MKDEVID(DISK_MAJOR, 0). Think of ways that you can write to a disk and display the data stored there.
• Note that Weenix does not currently support Caps Lock.

It is very important that you get this code working flawlessly, since it can be a constant source of headaches later on if you don’t. Note that the disk driver only works with page-aligned data, so you should use page_alloc() to allocate the memory used in your test cases, not kmalloc().

Since you should now have a functional tty, you should try using the kernel shell to test it out. Once you are confident in your tty code, try implementing your own kshell commands to run further kernel tests.

Below is what to add to initproc_run() (#include "test/kshell/kshell.h" at the top of kernel/main/kmain.c):

/* To create a kshell on each terminal */
#ifdef __DRIVERS__
    char name[32] = {0};
    for (long i = 0; i < __NTERMS__; i++)
    {
        snprintf(name, sizeof(name), "kshell%ld", i);
        proc_t *proc = proc_create("ksh");
        kthread_t *thread = kthread_create(proc, kshell_proc_run, i, NULL);
        sched_make_runnable(thread);
    }
#endif

/* Run kshell commands until each kshell process exits */
while (do_waitpid(-1, &status, 0) != -ECHILD)
        ;

The code above will spawn three kshell terminals, to toggle between them, you can use the F1, F2, and F3 keys. If you would like to add your own kshell commands, add the following code to kshell_init:

/* Add some commands to the shell */
kshell_add_command("test1", test1, "tests something...");  
kshell_add_command("test2", test2, "tests something else...");

The commands that you write should be added to commands.c.

4.7 Expected Output

• Drivers tests
  • It is expected behavior for your kshell to boot up with random outputs before your shell prompt (this is because of the tests). To see if the tests pass, you can check the output in the terminal that you used to boot up weenix
  • When you halt weenix should halt cleanly

4.8 Milestone Checklist

Complete before mentor meeting for extra credit on Weenix. Please note that you can choose to complete the interactive grading for Drivers at this meeting (the soft deadline) or at the end of Weenix (hard deadline).

• The kshell starts up
• Line Discipline
  • You are able to type normal characters and see them displayed on the screen.
  • Backspace deletes the last character; ctrl-d cooks and executes command; ctrl-c discards and moves to next line; newline goes to new line, cooks and executes command.
  • Repeat and try different combinations to make sure it works as intended.
  • Type in 127 characters, and make sure you can only type in backspace, ctrl-d, ctrl-c and newline as the 128th character.
• Runs the built-in commands (e.g. “help”, “echo”, …).
• Pressing fn + F1/F2/F3 navigates between different shells.
• ctrl-d without any command exits the current shell (“bye!” shows up).
• Typing the command “exit” exits the current shell (“bye!” shows up).
• Exit all 3 shells and “Weenix has halted cleanly!” shows up.
• Command “halt” exits all shells and “Weenix has halted cleanly!” shows up.
• All given drivers tests pass.

4.9 More Information

The Help Guide provides a lot of useful information on things you may be confused on now and throughout the project. Be sure to check this link first if you are stuck on anything!

4.10 Frequently Asked Questions

• Special character handling
  • Enter on blank line: The kshell prompt should print again
  • Backspace: delete the character and emit (or no-op if there's no character to delete)
  • ETX: remove the uncooked part and have a '\n' written on the terminal
• How do the chardev_ops relate to the functions we write?
  • You are technically writing the functions for chardev_ops! As an example, for a tty, they correspond to tty_read and tty_write. To use them, if you have a pointer to chardev_t, let's say cd, to invoke its operations, you can do cd->cd_ops->read(), see driverstest.c for an example.
• What do the null_read and null_write do?
  • tty_read and tty_write correspond to the cd_ops for a tty, null_read and null_write correspond to the cd_ops for the device /dev/null. You are responsible for implementing the function null_write directly—if you were to invoke the cd_ops->write() on the character device corresponding to /dev/null, it would eventually invoke the null_write function.
• vterminal_key_pressed vs vterminal_write
  • vterminal_key_pressed will get the raw part of the buffer and display that to the vterminal — you should call this function only for non-special characters. vterminal_write is used for displaying the special characters, namely, \n and \b. While \n should be placed in the line discipline buffer, \b should not.