P9 - hpaluch/hpaluch.github.io GitHub Wiki

Plan 9

"Plan 9" is unique system (from UNIX creators). However it may be hard to grasp first.

[!WARNING]

Just by accident found this page: https://orib.dev/gefs.html where are disclosed serious issues with existing Plan9 filesystems, quoting several points:

  • on power loss, the file systems tend to get corrupted, requiring manual recovery steps
  • silent disk corruption is not caught by the file system, and reads will silently return incorrect data
  • they tend to require a large, unshrinkable disk for archival dumps, and behave poorly when the disk fills

I have verified that most of this is also case of hjfs - examining sources under /sys/src/cmd/hjfs:

  • no asserts, no corruption checks - which means that most errors will be unnoticed for long time possibly leading to cascaded massive filesystem corruption in time - manifesting only when it is too late
  • single superblock with no backup
  • no fsck command utility to detect and correct errors (there is commented out command check, but it just counts disk blocks by types and report results - no real check is performed there)
  • missing any kind of checksums in case of metadata and/or data

If you intend to use Plan for any kind of production purpose you should make regular logical backups (that do not depends on filesystem structure - for example tar) to be able to recover data in case of filesystem failure.

Main characteristics:

  • everything is file (including env, sockets)
  • because everything is file there are possible many neat tricks:
    • irc or http client and http server is written in shell (!)
    • even terminal windows uses files for operations, for example /dev/text is completed buffer of your terminal window so you can for example grep it
    • /dev/snarf is clipboard - you can read it or modify it
    • similarly works mouse and keyboard
  • fully UTF-8 compatible (Plan9 authors invented UTF-8)
  • transparently distributed system - remote resources (CPU, files, ...) can be just mounted on local machine and accessed as local files
  • full namespace support - each process has its own view of filesystem

NOTE: There are several distributions of Plan9 including:

  • Bell Labs - original now managed by "Plan 9 Foundation"
  • 9Front - most popular

Bell Labs install

Right now I'm trying original distribution from Bell Labs (now managed by "Plan 9 foundation"). ISO can be found on https://9p.io/plan9/download.html

Notes for KVM/QEMU:

  • machine type i440fx, BIOS mode - important!
  • allocated 1 CPU
  • 1GB of RAM
  • 10GB of IDE disk
  • assigned plan9.iso to IDE CD-ROM
  • network card: e1000

NOTE: I had no luck with SATA/AHCI (they should be supported but maybe different type?)

NOTE: When you are asked for CD-ROM distribution directory I have to:

cd /
exit

Installation will then continue...

After reboot when you will be asked for username default is none which is wrong - you have to use glenda (same username as in 9front Plan9).

Changing screen resolution on the fly:

aux/vga -l 1024x768x8

NOTE: It seems that monitor type is now basically ignored (at least in /bin/termrc)

Mounting 9fat win plan8.ini (to change screen resolution):

  • important - in 9front there is script so simple command 9fs 9fat will do the trick.
  • But on Bell Labs version you have to run 9fat: (yes with colon at the end) ENTER and then any editor...
  • it is actually shell script - see cat /bin/9fat: for contents

Recommended:

  • backup that file with: 
    cd /n/9fat
cp plan9.ini plan9.orig.ini
  • and edit with: 
    acme plan9.ini
  • in Acme scroll down to line vgasize=640x480x8 and change it to something bigger, but still supported by VESA
  • I tried 1024x768x8
  • also change monitor type to VESA: monitor=xga
  • to Save file middle click on Put label
  • next click on Exit
  • verify with cat plan9.ini that file was really changed
  • now I rather unmounted 9fat partition: 
    cd /
unmount /n/9fat # yes, unlike Unix it is "un"mount instead of "u"mount
  • to reboot type: 
    fshalt -r
  • hmm - it did not work... Need to debug /bin/termrc...

Network configuration:

  • it was supposed to start on boot in /bin/termrc
  • but it was commented out
  • to autostart DHCP client on boot you should uncomment block in /bin/termrc with ip/ipconfig command....
  • or manually run: 
    ip/ipconfig
ndb/dns -r
  • use netstat -ni to see network interfaces and assigned IP addresses

Getting free space on disk (using "fossil" filesystem):

con /srv/fscons
# you may need to press ENTER to see "prompt:"
printconfig
fsys main df

To exit con session:

  • press Ctrl-\
  • prompt >>> will appear
  • press q followed by ENTER to close connection

9Front install

This one is now probably most active and most popular.

Quick install:

WARNING! Refrain from using Delete key - it is used as "Terminate" in Plan 9 (similar to Ctrl-C on UNIX). If systems seems to not responding, try to type some character in terminal and then remove it with Backspace - usually it helps.

Really brief installation guide:

  • using KVM, selected Alma 9 as OS, 2 CPUs, 2GB of RAM, 20GB of disk, using VirtioBLK disk, SATA CD-ROM, virtio-net network card
  • NOTE: BIOS+MBR scheme recommended (UEFI+GPT is quite new on Plan9 - use on your own risk)
  • you have to select boot device - default is fine
  • select display resolution - in my case I changed 1024x768x16 to 1440x900x16
  • graphics type: I selected vesa
  • confirmed PS/2 mouse
  • after a while rio GUI should start
  • now you have to enter inst/start to start real installation
  • press ENTER to confirm configfs stage
  • confirm default cwfs64x
  • confirm partdisk with defaults
  • on disk selection there are:
    • sdE0 installation SATA DVD ROM
    • sdF0 target 20GB VirtioBLK disk
  • enter sdF0 as target disk name
  • use mbr as partition scheme
  • you will be asked to create Plan 9 partition - type as suggested w ENTER and q ENTER
  • press ENTER to confirm prepdisk stage
  • accept default of using prepared Plan 9 partition (/dev/sdF0/plan9 in my case)
  • again you can use w ENTER and q ENTER to confirm suggested partitioning
    • NOTE: it is nested partitioning similar to BSD disklabel (where BSD has single primary partition on MBR and inside of that single partition are subpartitions for specific system)
  • press ENTER to confirm mountfs stage
  • confirm fscache selection with ENTER, then fsworm with ENTER and finally other with ENTER
  • confirm yes on Ream (= Format filesystem)
  • next press ENTER for confignet stage
  • press ENTER for automatic (DHCP) configuration
    • ignore no router advs after 3 sols on /net/ether0 error - that's expected when you have no IPv6 router on network.
  • press ENTER to confirm mountdist stage
  • type /dev/sdE0/data as Distribution disk and ENTER
  • press ENTER to confirm Location of archives = /
  • press ENTER for stage copydist
  • press ENTER for stage ndbsetup (basically /etc/hosts and DNS)
  • sysname - press ENTER to accept cirno default
  • press ENTER for stage tzsetup
  • I use CET (Central European Time) for EU
  • finally press ENTER for bootsetup stage
  • press ENTER to confirm 9fat partition
  • type yes ENTER to write master boot record
  • type yes ENTER to confirm Active flag on boot partition (required by some BIOS vendors to boot at all)

Tip: to save installation log you can just type (when asked with [finish] prompt):

!cat /dev/text > /n/newfs/usr/glenda/install.log

After reboot you will find install.log with complete console output in your (glenda's) home directory as install.log. It is very useful to learn Plan 9 you will for example see commands to create new filesystem and setup admin user for it.

And then:

  • finally press ENTER for finish installation

System should automatically reboot. On reboot just:

  • confirm default bootargs (ending with fscache)
  • confirm user glenda
  • you can see sdE0 i/o errors - that's OK, because LibVirt automatically removes ISO on reboot

How to view boot messages with ease:

  • cat them to text file using: 
    mkdir logs
cd logs
cat /dev/kmesg > kmesg.txt
acme kmesg.txt
  • to exit Acme editor do middle-button press on Exit menu.
  • to run Acme again:
    • left-click and hold and select existing acme kmesg.txt command
    • middle-click and hold - move mouse to send - release middle-button to execute ("send")

Remember:

  • to move Window - move mouse to bold frame (cursor will change)
    • press and hold right-button (frame color will change to red)
    • move mouse while still holding middle button will move window
  • to resize Window - move mouse to bold frame and use left-click hold and move.

Installed system under Proxmox (disk: VirtIO BLK, network: VirtIO NET).

  • Do NOT use lvm-thin - copying will take forever. I suspect that there is significant write amplification
  • I use qcow2 on ext4 (because I will need snapshots, otherwise I would use raw), cache: unsafe
  • 1 CPU, 2 GB RAM

There is really good installation guide - and mostly you just press enter when asked for something:

Both installation (and after installation) runs GUI called rio(1) very good intro is at:

How to create New Terminal window:

  • Right click and Hold on empty background
  • move mouse to New menu item and release Right button
  • when Cross (+) appears again Right-click (important) and hold - draw new Window
  • move mouse to draw rectangle of new window. When done - release button and Terminal will start in that rectangle area.

You can use man ITEM for most commands...

System info - run sysinfo script. Most commands are in /rc/bin (try ls /rc/bin).

How to shutdown:

fshalt

Basic terminal tips

Terminal window is called rio(1) and it works like full screen editor.

I strongly recommended watching at least these YouTube Videos for introduction:

NOTE: Autoscroll is disabled by default. You have two choices:

  1. press ENTER to show next page
  2. click-and-hold Middle button and select scroll and release Middle button - this will enable auto-scroll

Selecting text - again 2 choices: a) double-click on text to select word or text inside quotes b) left-click and hold left button - move mouse to select text - but see below for details.

Copy/Cut & Paste using "chords":

  1. left-click and hold left button - select text - still hold left-button
  2. now also press middle button (keep holding left-button) - it will Cut text 2.b) if you want to Copy instead of cut - still hold left and middle button. Additionally press Right button - text will be pasted back - same effect as Copy.
  3. release all buttons (I'm rather releasing from Right to Left)
  4. position mouse to place where you want paste text
  5. left-click, hold and right-click - text will be pasted
  6. release all buttons

Copy/Cut & Paste using menu:

  1. select text with any method - release left-button when selection is done
  2. Middle click and hold to show menu. Select Snarf (Copy) or Cut and release mouse button
  3. Position your mouse to place where you want paste text
  4. Middle click and hold to show menu. Select Paste and release mouse button.

Other menu items:

  • plumbing -> will open selected file in application (typicaly in page viewer)
  • send -> will run command in clipboard

Random tips

  • killing process: 
    ps # find process name
kill NAME | rc
    • please note that kill NAME will just prints command to kill so it has to be piped to "rc" shell to really kill process...
    • it is sometimes handy to first try kill NAME if there is output at all. Then you can select it with mouse, middle-click and choose send to run that script.
  • Use Del key instead of Ctrl-C to stop program
  • To find assigned IP address: cat /net/ndb or netstat -i
  • To see routing tables cat /net/iproute
  • most scripts are under /rc/bin
  • default shell (equivalent of /bin/sh under Unix) is named rc - try man rc

Generating graphical manual pages (from Video tutorial):

  • example: man -t rio | page -w

Building docs

Try:

cd /sys/doc
mk

Should generate lot of *.ps files in /sys/doc

WARNING! OpenSUSE Viewer (gv) is somehow broken (strange rendering of fonts). However you may use ps2df file.ps (included with ghostscript package) and then evince file.pdf.

Convert manual page to PostScript:

man -t fdisk | dpost -f > fdisk.8.ps

On Linux you can use ps2pdf to convert single file from PostScript to PDF.

Serving filesystem from Plan9 server to Linux client

After lot of digging found working example

On Plan9 I run:

ip/ipconfig # enables network
# 9front:
aux/listen1 -tv tcp!*!9999 /bin/exportfs -r /sys/src
# Bell Labs: without "-R" there is fatal error "chdir '.' invalid path"
aux/listen1 -tv tcp!*!9999 /bin/exportfs -R -r /sys/src

On Linux client:

mount -r -t 9p -o version=9p2000,port=9999,trans=tcp IP_ADDRESS_OF_PLAN9_SERVER /mnt/9p
# should see contents of /sys/src
ls -l /mnt/9p

Connecting remotely to CPU server

Default plan installation is so called "standalone terminal" - single user. Although it is possible to run "manually" some services as single user it is not standard way. One has to switch this system to cpu mode...

[!WARNING] It took me so long until I found that you MUST use 9front version of drawterm to connect to 9front version of Plan 9!

Do NOT use version from https://github.com/9fans/drawterm.git to connect to 9front! It will just throw weird error dp9ik@virtual, which means "I don't support DP9IK authentication" (while virtual is so called "authentication domain").

When I hit this note on https://git.9front.org/plan9front/drawterm/HEAD/info.html it perfectly explain my issue:

This is a fork of Russ Cox's drawterm to incorporate features from Plan9front (http://9front.org), most importantly DP9IK authentication support (see authsrv(6)) and the TLS based rcpu(1) protocol.

So you MUST use 9front build of drawterm on your Linux:

git clone git://git.9front.org/plan9front/drawterm drawterm-9front
cd drawterm-9front/
CONF=unix make

Note:

  • under openSUSE LEAP 15.6 I have to install also libXt-devel (to avoid error not finding X11/Intrinsic*.h

Once I have right version of drawterm this guide works perfectly for me:

Example how to connect to 9front Plan9 target (replace TARGET_IP with your Plan9 target):

YOUR_PATH_TO/drawterm-9front/drawterm -a TARGET_IP -s TARGET_IP -h TARGET_IP -u glenda

Other guides - not tests:

Plan9 partitions

Here is brief inside how Plan9 manages partitions/filesystems (looked to it from CloneZilla ISO image):

MBR disk layout:

$ fdisk -l /dev/vda

Disklabel type: dos
Disk identifier: 0x00000000

Device     Boot Start      End  Sectors Size Id Type
/dev/vda1  *       63 41929649 41929587  20G 39 Plan 9

That partition appears as FAT16, but reported FAT16 size is significantly smaller than 20GB (because after that FAT16 are real Plan9 partitions):

# file -s /dev/vda1

/dev/vda1: DOS/MBR boot sector, code offset 0x58+2, \
 OEM-ID "Plan9.00", sectors/cluster 8, reserved sectors 2, \
 root entries 512, Media descriptor 0xf8, sectors/FAT 100, \
 sectors/track 63, heads 255, \
 hidden sectors 63, sectors 204800 (volumes > 32 MB), \
 serial number 0, label: "PLAN9      ", FAT (16 bit)

We can use standard linux mount -r /dev/vda1 /mnt/source to see filesystem:

$ df -h /dev/vda1

Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1       100M  5.2M   95M   6% /mnt/source

$ ls -o /mnt/source/

total 5252
-rwxr-xr-x 1 root    8088 Jul 12 08:34 9bootfat
-rwxr-xr-x 1 root 5356664 Jul 12 08:34 9pc64
drwxr-xr-x 3 root    4096 Jul 12 08:34 efi
-rwxr-xr-x 1 root     512 Jul 12 08:34 pbs.bak
-rwxr-xr-x 1 root     113 Jul 12 08:34 plan9.ini
  • The plan9.ini is text configuration file.
  • 9bootfat is 2nd stage bootloader (that boots kernel 9pc64
  • 9pc64 is Plan9 kernel

But where is "real" partition table? It is in 2nd sector of this small FAT:

$ dd if=/dev/vda1 skip=1 count=1 of=plan9-part.bin
$ hexdump -C plan9-part.bin

00000000  70 61 72 74 20 39 66 61  74 20 30 20 32 30 34 38  |part 9fat 0 2048|
00000010  30 30 0a 70 61 72 74 20  6e 76 72 61 6d 20 32 30  |00.part nvram 20|
00000020  34 38 30 30 20 32 30 34  38 30 31 0a 70 61 72 74  |4800 204801.part|
00000030  20 6f 74 68 65 72 20 32  30 34 38 30 31 20 36 31  | other 204801 61|
00000040  36 35 34 38 34 0a 70 61  72 74 20 66 73 63 61 63  |65484.part fscac|
00000050  68 65 20 36 31 36 35 34  38 34 20 31 32 31 32 36  |he 6165484 12126|
00000060  31 36 37 0a 70 61 72 74  20 66 73 77 6f 72 6d 20  |167.part fsworm |
00000070  31 32 31 32 36 31 36 37  20 34 31 39 32 39 35 38  |12126167 4192958|
00000080  37 0a 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |7...............|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200

Plan9 nested partition is really ACIIZ string, excerpt from man fdisk:

Plan 9 partitions (and Plan 9 disks on non-PCs) are them- selves divided, using a textual partition table, called the Plan 9 partition table, in the second sector of the partition (the first is left for architecture-specific boot data, such as PC boot blocks). The table is a sequence of lines of the format part name start end, where start and end name the starting and ending sector. Sector 0 is the first sector of the Plan 9 partition or disk, regardless of its position in a larger disk. Partition extents do not contain the ending sector, so a partition from 0 to 5 and a partition from 5 to 10 do not overlap.

It is standard ASCIIZ string:

$ tr -d '\0' < plan9-part.bin | cat -v

part 9fat 0 204800
part nvram 204800 204801
part other 204801 6165484
part fscache 6165484 12126167
part fsworm 12126167 41929587

Notice that 1st line - 204800 is real FAT16 size is sectors (around 204800/2 => 102400 MB). Other Plan9 partitions are hidden beyond that small FAT16 partition.

Grep variants

There is programmer's grep called just g - that recursively search programmer's files (sources scripts, ...). Example to search for nil (NULL) under /sys/include:

g nil /sys/include

...

Tracing cwfs

Looking into sources under /sys/src/cmd/cwfs/* found how to enable cwfs tracing:

  • open new console with messages from kernel using cat /dev/kprint (it will block - waiting for new messages)

  • now run this:

    con -C /srv/cwfs.cmd
# now you are in command channel. Try 'statw' to get basic stats...
statw
# use "chatty level" to enable debug level:
chatty 7

  • now do something on filesystem (create file/dir, etc..)

  • and watch window with /dev/kprint...

  • still on con you should set chatty 0 before exit

  • to exit con do this:

    • press Ctrl-\ and ENTER to get con prompt >>>
    • type q and ENTER to exit con

Resources

Interesting stories: