OpenBSD

TODO ...

Important to know:

NOTE: OpenBSD disklabel (that is mapped to devices under /dev is limited to total 16 entries, where first 8 slices ('a' to 'h' are always defined manually taking into account special meaning for:

a is typically root filesystem
b is typically swap
c is "virtual" entry representing whole disk (when some tools need access whole disk they can use /dev/DISKc device - because normally only slices are accessible as devices)
'a' to 'h' (excluding 'c') are always manually defined by user in disklabel editor

There are also available additional 8 slices:

'i' to 'p' that are automatically created by kernel from existing GPT partitions and/or MBR partitions. But they can be changed/reused by user (adding 8 additional slices to access areas on disk)

Thank to kind souls on mailing list I got this clarification:

any disklabel entry (from a to 'p' excluding c) may point to any area on disk - even outside of single OpenBSD (GPT or MBR) partition. This can be used to access any part of disk including (but not limited) to other partitions.
although 'i' to 'p' are automatically created in memory from remaining MBR or GPT partitions they can be manually overridden by user.

So important conclusion is is that total 14 slices can be used to access any partition (or actually even non-partition area of disk).

Here are references:

my original inaccurate post: https://marc.info/?l=openbsd-bugs&m=170297639319139&w=2
follow-ups with corrections:
- https://marc.info/?l=openbsd-bugs&m=170297691819467&w=2
- https://marc.info/?l=openbsd-bugs&m=170299306330367&w=2

Notes on environment

My /etc/doas.conf:

permit nopass :wheel

this permits anyone in wheel group to run doas without password.
for example doas id

My favourite packages:

pkg_add curl wget mc lynx vim--no_x11

NOTE: tmux is part of base system, so there is no Package for it.

My /etc/rc.conf.local:

ntpd_flags=NO
smtpd_flags=NO
sndiod_flags=NO
slaacd_flags=NO

More challenging package qemu_ga under Proxmox VE Host - because it uses latest PCI serial devices by default:

install is easy:
```
pkg_add qemu-ga
```
however you must publish QEMU agent port device as ISA (OpenBSD has no PCI serial driver, that is default on Proxmox/KVM host).

Use any of:
- in Proxmox VE go to VM -> Options -> Qemu Guest Agent -> Edit
  - select Enabled
  - check on Advanced
  - change Type: to ISA
- or you can directly edit .cfg - here is difference:
```
c1
< agent: 1
---
> agent: 1,type=isa
```
power-cycle VM

verify that OpenBSD guest kernel detected com port:

dmesg | grep COM
com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo

now you have to specify this com0 port (as /dev/cua00) to QEMU:

add these two lines to your /etc/rc.conf.local

pkg_scripts=qemu_ga
qemu_ga_flags="-t /var/run/qemu-ga -m isa-serial -p /dev/cua00 -f /var/run/qemu-ga/qemu-ga.pid"

important are both MODE (-m isa-serial) and Path to device (-p /dev/cua00 )
start service /etc/rc.d/qemu_ga start
ensure that ok appear
back in Proxmox - go to VM Summary - you should quickly see assigned IP address (proof that qemu agent is running)

or you can try in Proxmox shell:

qm guest cmd MY_VM_ID info
# should produce lot of JSON

or something (OpenBSD 7.2 guest):

qm guest cmd MY_VM_ID get-osinfo

example output:

{
   "kernel-release" : "7.2",
   "kernel-version" : "GENERIC#728",
   "machine" : "amd64"
}

Congratulations - you are now running QEMU Guest Agent!

My post should be soon published here soon:

https://forum.proxmox.com/threads/backing-up-openbsd-vm.34534/page-2#post-560977

Notes on USB pendrive install

When you install OpenBSD from USB pendrive there are two important things:

You have to master .img image (NOT .iso! - it will not boot!), for example install73.img
When you are asked for location of installation sets:
- you have to answer disk (which can be easily overlooked)
- then you have to specify disk - usually sd1 (second "SCSI")
- then you will be presented with disklabel - there should be only 2 partitions
  - a with FFS
  - MSDOS partition
- select a partition
- now you can continue as usual - the offered path should be fine.
  - confirm installation without signature check.

Linux to OpenBSD glossary

Here I will show few Linux commands and their OpenBSD equivalents:

List of open files:

Linux: lsof
OpenBSD: fstat

The fstat command can be also used to list process listening on TCP/UDP sockets:

fstat | fgrep internet

root     sshd       17658    3* internet stream tcp 0x0 *:22
root     sshd       17658    4* internet6 stream tcp 0x0 *:22
_dhcp    dhcpleased 64686    4* internet dgram udp *:0
_dhcp    dhcpleased 64686   10* internet dgram udp 192.168.0.102:68
root     dhcpleased  1517    8* internet dgram udp *:0

Show assigned IP addresses:

Linux: ip -br a
OpenBSD: netstat -in

Used swap:

Linux: cat /proc/swaps

OpenBSD: pstat -s, Example output:

Device      512-blocks     Used    Avail Capacity  Priority
/dev/sd0b      4196288        0  4196288     0%    0

Become root (other traditional options is command su):

Linux sudo
OpenBSD doas

Top with process arguments:

Linux: top -c
OpenBSD: top -C

Quickly reboot system (requires root privileges):

Linux: init 6 or reboot
OpenBSD: reboot

Power down system (requires root privileges):

Linux: init 0 (Debian, SUSE) or poweroff (Arch, Alpine,..)
OpenBSD: halt -p

Show PCI devices:

Linux: lspci -v
OpenBSD: pcidump -v (requires root privileges)

Strace alternative

descirbed here: http://www.yak.net/fqa/298.html

use these two commands:

cd
# -i - follow child processes
ktrace -i /bin/ls
# and then decode create created 'ktrace.out'
kdump -f ktrace.out

OpenBSD only commands:

systat show nice kernel utilization stats (interrupts, forks memory ...)
- general overview: systat vmstat
- disk IO: systat iostat
- tcp/udp overview: systat netstat
- hardware sensors: systat sensors (on MSI-7250, there are: admtemp0,kate0,lm1)
- you can also use Left and Right arrows to switch views inside systat

getting list of block devices (disks, floppies, CD ROMs)...

$ sysctl hw.disknames

hw.disknames=cd0:,sd0:1d9203b36031e65b,fd0:

how to list changed files in /etc/:
```
sysmerge -db
```
how to merge changed filesi in /etc/ interactively:
```
sysmerge -d
```

Same commands:

all process tree: ps axf

Notes on partitioning

You have to manually ensure proper partition align:

for recent HDD align on 4KB = 8 sectors - so your partition start and partition size should be dividable by 8 (without reminder)
for SSD you should align on 1MB = 2048 sectors - so your partition start and size should be dividable by 2048 (without reminder)

I simply use calculator on mobile phone to verify that numbers are properly aligned.

Notes on MBR install

If you install OpenBSD to clean disk - without existing loader on MBR - you need to install MBR boot using:

fdisk -u wd0
# confirm Yes

Otherwise your BIOS will tell you famous Missing operating system....

Booting OpenBSD from GPT+BIOS

Officially it is not possible to boot OpenBSD from GPT via BIOS (only EFI supported). However if you have working Linux + Grub2 installation you can nicely boot it via GRUB.

Update - May 2023: This example now uses AHCI SATA3 controller (was on-board nVidia MCP55 SATA2 controller). Therefore disk name changed from wd0 to sd0.

Here is example of my complex setup. I have installed these OS in specified order (which matches partition order):

Proxmox VE (Linux KVM) on LVM with ext4
OpenBSD
NetBSD
FreeBSD
Arch LInux

Here is GPT partition layout from Linux fdisk (I used it instead of OpenBSD because it knows all GPT ids):

Command (m for help): Disk /dev/sda: 447.13 GiB, 480103981056 bytes, 937703088 sectors
Disk model: KINGSTON SA400S3
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 227D1FFD-48E0-40EE-9D5D-0AF5EF25AE11

Device         Start       End   Sectors  Size Type

# PROXMO VE - Grub - allows GPT+BIOS boot
/dev/sda1         34      2047      2014 1007K BIOS boot

# unused in case of lvm/ext4 - reserved for UEFI:
/dev/sda2       2048   2099199   2097152    1G EFI System

# LVM holding Proxmox VE rootfs ans swap:
/dev/sda3    2099200 314572800 312473601  149G Linux LVM

# OpenBSD UEFI Boot (not used in my case)
/dev/sda4  314574848 314779647    204800  100M EFI System

# OpenBSD disklabel holding rootfs and swap
/dev/sda5  314779648 417179647 102400000 48.8G OpenBSD data

# NeetBSD rootfs (direct partition using wedge)
/dev/sda6  417181696 517844991 100663296   48G NetBSD FFS

# NetBSD swap (direct partition using wedge)
/dev/sda7  517844992 534622207  16777216    8G NetBSD swap

# FreeBSD GPT Boot (not used in my case)
/dev/sda8  534622208 534623231      1024  512K FreeBSD boot

# FreeBSD rootfs
/dev/sda9  534623232 639479807 104856576   50G FreeBSD UFS

# FreeBSD swap
/dev/sda10 639479808 656257023  16777216    8G FreeBSD swap

# Arch Linux rootfs (it uses swap from Proxmox)
/dev/sda11 656257024 761114623 104857600   50G Linux filesystem

Important! You should ensure that all swaps and filesystems (with exception of boot loaders) are properly aligned!

for HDD 4KB alignment (sector multiple of 8) is recommended
for SSD 1MB alignment (sector multiple of 2048) is recommended

If partitions are not properly aligned than both read and write amplification will occur (disk will have to do significantly more I/O than normally).

Here is how i verify that under Bash or OpenBSD SH:

GRUB BIOS - start on sector 34 not aligned to 1MB - modulo is 34 (should be 0 when aligned)
```
echo $(( 34 % 2048 ))

34
```
OpenBSD disklabel - properly aligned to 1MB:
```
echo $(( 314779648  % 2048 ))

0
```

Regarding OpenBSD installation: I normally booted OpenBSD 7.3 CD, selected Manual partition and created both another EFI + OpenBSD main partition (EFI created just to satisfy installer, we will not use it later).

You have to create both

small EFI Sys partition (ID 0xEF in fdisk)
and OpenBSD partition for disklabel (ID 0xA6 ) holding all OpenBSD partitions.

without EFI Sys partition the installation program will refuse to continue and exit with error.

On PC, disklabel starts on 2nd sector (offset 512 from partition start) as can be seen in /usr/src/sys/arch/amd64/include/disklabel.h

#define	LABELSECTOR	1	/* sector containing label */
#define	LABELOFFSET	0	/* offset of label in sector */
#define	MAXPARTITIONS	16	/* number of partitions */

It is necessary, because under MBR it is expected that 1st sector contains boot loader and its offset can't be changed.

Then I manually edited disklabel using provided disklabel command, with following layout:

#                size           offset  fstype [fsize bsize   cpg]
  a:         98205696        314779648  4.2BSD   2048 16384 12960 # /
  b:          4194304        412985344    swap                    # none
  c:        937703088                0  unused
  i:             2014               34    boot
  j:          2097152             2048   MSDOS
  k:        312473601          2099200 unknown
  l:           204800        314574848   MSDOS
  m:        100663296        417181696 unknown
  n:         16777216        517844992 unknown
  o:             1024        534622208 unknown
  p:        104856576        534623232 unknown
...

Important! These nested partitions (in disklabel) are usually called "slices" to distinguish them from MBR or GPT partitions. Please note that:

only slices a:,b:,d: to h: are "real" slices that are usable for OpenBSD where b: slice should be always reserved for swap (because it is expected by kernel)
slice c: always represents whole disk and is synthetic

sliced starting from i: are mapped MBR partitions of GPT partitions!

so for example:

#                size           offset  fstype [fsize bsize   cpg]
  i:             2014               34    boot

is actually Grub GPT partition in my case:

Device         Start       End   Sectors  Size Type
/dev/sda1         34      2047      2014 1007K BIOS boot

these fake slices are created in /usr/src/sys/kern/subr_disk.c one of those functions:

// fake GPT slices
int
spoofgpt(struct buf *bp, void (*strat)(struct buf *), const uint8_t *dosbb,
    struct disklabel *lp, daddr_t *partoffp)
{
...
n = 'i' - 'a';  /* Start spoofing at 'i', a.k.a. 8. */
...
}

// for MBR
void
spoofmbr(struct buf *bp, void (*strat)(struct buf *), const uint8_t *dosbb,
    struct disklabel *lp, daddr_t *partoffp)
{
...
n = 'i' - 'a';
...
}

notice that those "spoofed" slices always start from 'i:' so you can use only slices from a: to h: (with exception of c: which is always whole disk) for OpenBSD

Again - real OpenBSD slices (a: and b:) should start on 4KB boundary for HDD and 1MB boundary on SSD.

Disclaimer: having just 1 partition for complete filesystem is NOT recommended by OpenBSD maintainers. However I use it for evaluation. Just be warned...

And finished installation.

Then under Proxmox VE (Debian) I created this GRUB2 manual menu entry in /etc/grub.d/40_custom

#!/bin/sh
exec tail -n +3 $0
# XXX This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry "OpenBSD SSD" {
        set root='hd0,gpt3'
        insmod ufs2
        insmod bsd
        set root='(hd0,gpt5)'
        # wd0a for MCP55, SiI3112 and others
        # sd0a for AHCI or real SCSI
        kopenbsd -r sd0a /bsd
        boot
}

Notes:

for AHCI or true SCSI you diskname will be sd0a instead of wd0a. The wd0a is typically for legacy IDE drivers (including non-AHCI SATA controllers).
first set root=hd0,gpt3 is necessary to hint Grub where it can find GRUB modules (otherwise insmod will throw confusing error messages) - so (hd0,gpt3) is Linux installation with Grub2
second set root=(hd0,gpt5) is pointer to OpenBSD partition where is disklabel
then just little magic

You have to run update-grub in your Linux so /boot/grub/grub.cfg is actually updated. Please ignore plenty of warnings:

/usr/sbin/grub-probe: \
  warning: Discarding improperly nested partition (hostdisk//dev/sda,gpt5,bsdXX).

The grub-probe thinks that BSD uses directly partitions without disklabel, which is true for FreeBSD and NetBSD but OpenBSD - it uses disklabel even on GPT (so only 1 GPT partition can be used for OpenBSD).

Then reboot and if you are lucky, your OpenBSD OS should start from GPT+BIOS disk :-)

Notes on packages

Smartctl

You have to append whole disk label (c - virtual disklabel partition containing whole disk) to device name when using smartctl:

pkg_add smartmontools
# for IDE, ATA, and non-AHCI SATA
smartctl -a /dev/wd0c
# for AHCI SATA or SCSI
smartctl -a /dev/sd0c

Virtualization

According to https://www.openbsd.org/faq/faq16.html#Prerequisites

A CPU with nested paging support is required to use vmm(4). Support can be checked by looking at the processor feature flags: SLAT for AMD or EPT for Intel.

NOTE: Some low-level info on AMD SVM is here:

https://www.amd.com/system/files/TechDocs/24593.pdf

I'm out of luck (I know that my CPU has not SLAT) - confirmed empty output on:

dmesg | egrep '(VMX/EPT|SVM/RVI)'

The dmesg shows a bit confusing message:

vmm0 at mainbus0: SVM

However verified that:

rcctl enable vmd
rcctl start vmd

vmd(failed)

# and in /var/log/messages
vmd: /dev/vmm: Operation not supported by device

So KVM still rules...

How to use tags in Vim

Vi and Vim allows easy jump to function or variable declaration using so called tags file. Here is example how to use them for OpenBSD kernel:

ensure that you have installed Vim with pkg_add vim--no_x11
apply this small patch to ignore non x86 architectures:

diff -u -p -r1.52 Makefile
--- kern/Makefile       1 Oct 2022 23:49:38 -0000       1.52
+++ kern/Makefile       28 May 2023 14:49:26 -0000
@@ -7,6 +7,8 @@ ARCH=   alpha amd64 armv7 arm64 \
        luna88k macppc octeon \
        sparc64
 
+ARCH= amd64 i386
+
 all:
        @echo "Doing nothing.  Did you want make syscalls?"

Now prepare tags file (may take few minutes):

cd /usr/src/sys
make tags

Now go to directory where are tags and edit some file with vim, for example:

cd /usr/src/sys/arch/amd64
ls -l tags
# use same relative path as in tags file:
vim ../../dev/ic/wdc.c

Now move cursor to declaration of your interest, for example:

void
wdcstart(struct channel_softc *chp)
HERE            ^

And press Ctrl-]
you will be automatically moved to ../../dev/ic/wdcvar.h with struct channel_softc declaration
to return back to wdc.c just press Ctrl-t

And that's it!

If in doubt try :help - and use :q to return from help.

OpenBSD trap when installing from libvirt/KVM IDE CD

I have found that OpenBSD 7.3 produces fatal trap:

wdc_atapi_start: not ready, st = 50 fatal protection fault in supervisor mode
trap type 4 code 0 rip ffffffff810089d9 cs 8 rflags 10282 cr2 2377bf000 cpl 6 rsp ffff8000150a2f50
gsbase 0xffffffff818fbff0  kgsbase 0x0
panic: trap type 4, code=0, pc=ffffffff810089d9
syncing disks...6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6  giving up

dump to dev 17,1 not possible
rebooting...

Explained here:

https://marc.info/?l=openbsd-bugs&m=168564039922088&w=2

Detailed blog is available here: OpenBSD i8254 trap

OpenBSD as GUI Workstation

I plan to evaluate Arch Linux, FreeBSD, NetBSD and OpenBSD as alternative to container dictatorship that is appearing on Linux (not mentioning systemd). So here is my experience with OpenBSD as workstation.

Sound support

Official page is here: https://www.openbsd.org/faq/faq13.html

Here is my on-board sound-card - on MSI-7250 board with nVidia MCP55 chipset:

$ dmesg | fgrep Audio

azalia0 at pci0 dev 6 function 1 "NVIDIA MCP55 HD Audio" rev 0xa2: apic 2 int 11

WARNING! When using VM under libvirt/KVM the default ich6 will not work:
dmesg | fgrep azalia
azalia0 at pci0 dev 4 function 0 "Intel 82801FB HD Audio" rev 0x01: apic 0 int 11
azalia0: No codecs found
What is strange that it is there: https://man.openbsd.org/OpenBSD-7.3/azalia.4

Intel 82801FB/GB/H/I/JI (ICH6, ICH7, ICH8, ICH9, ICH10) for this libvirt configuration:
<sound model='ich6'>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</sound>
FIX: had luck with this one (see https://libvirt.org/formatdomain.html):
<sound model='ac97'>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</sound>
Now sndioctl -d works and dmesg shows:
dmesg | grep -iE '(auich0|ac97)'

auich0 at pci0 dev 4 function 0 "Intel 82801AA AC97" rev 0x01: apic 0 int 11, ICH
ac97: codec id 0x83847600 (SigmaTel STAC9700)
audio0 at auich0
Wow!

More information is on man azalia - link here:

https://man.openbsd.org/azalia.4

NOTE: to have working audio ensure that daemon sndiod(8) is running (should be default).

To see mixer settings try:

sndioctl -d

009:input[0].level=0..255 (120)
010:input[1].level=0..255 (120)
011:input[0].mute=0..1 (0)
012:input[1].mute=0..1 (0)
005:output[0].level=0..255 (120)
006:output[1].level=0..255 (120)
007:output[0].mute=0..1 (0)
008:output[1].mute=0..1 (0)
001:server.device=0
002:server.device=1
003:server.device=2
004:server.device=3

It looks promising - output levels are non-zero and mute=0 (so should be off).

By following official manual we can try to play sound using aucat(1) with aucat -i file.wav

To get some sound WAV files you can try:

doas pkg_add asterisk-core-sounds-en-wav

To list available files you can try:

pkg_info -L asterisk-core-sounds-en-wav

And finally try (in plain shell - /bin/sh):

for i in /usr/local/share/asterisk/sounds/en/*.wav;do aucat -i $i;done

Wow! It works!

Or you can try to install my favourite:

doas pkg_add mpg123

And try my favourite Heavy Metal radio (from Arch Linux:

mpg123 http://stream.antenne.de:80/heavy-metal

Or even better to record songs for later play (it is OK as long it is for personal use):

doas pkg_add streamripper
... and ...

X-Window support

NOTE:

You can also run X-Window apps remotely without running X-Server on OpenBSD by using SSH with X11 forwarding. However you must first enable it - on /etc/ssh/sshd_config put this line:
X11Forwarding yes
And restart sshd using: doas /etc/rc.d/sshd restart

Then you can connect from other Unix running X-Window with ssh -X REMOTE_OPENBSD_IP or sometimes (Cygwin) ssh -Y REMOTE_OPENBSD_IP.

Running X-Window GUI on OpenBSD:

You need to have right graphics card. What is right is not easy to define. I'm unlucky one to have NVidia card and despite 2D driver nv(4) support it is slow because it defaults to XAA acceleration that is no longer supported by X-Server. Please see text below for analysis and workaround (using EXA acceleration).

Or go directly to my post: https://marc.info/?l=openbsd-tech&m=168685386609760&w=2

Getting running startx(1)

Disclaimer: recommended way to start X-Window is to use xenodm, so follow instructions below on your own risk!
you need to ensure that aperture (direct access to card memory) is allowed. Here is my content of /etc/sysctl.conf:
```
machdep.allowaperture=2 # See xf86(4)
```
please be aware that you are not allowed increase aperture on booted system, because its securelevel is > 0 (must be less or equal 0, which is on early boot only). So if you changed /etc/sysctl.conf you have to reboot your OpenBSD to apply those changes.
next: X-server must run as root to have bot access to /dev/xf86 and right to do I/O (this privilege is normally rised with amd64_iopl(2) call). The later requires root privilege unconditionally.

so I have to do this:

doas chmod o-rwx /usr/X11R6/bin/Xorg
doas chgrp wheel /usr/X11R6/bin/Xorg
doas chmod u+s /usr/X11R6/bin/Xorg
ls -l /usr/X11R6/bin/Xorg

-rwsr-x---  1 root  wheel  3067520 Mar 25 17:55 /usr/X11R6/bin/Xorg

WARNING! Above is security risk (Xorg server is notoriously known for various security bugs)! However I at least minimize it that only group wheel (that is already powerful) may run it.
now on local console you should be able to use startx - as bonus it will run fvwm(1) manager which I really like - it is there since early 90ties and it does exactly what it should do.
this is additional benefit of using startx instead of xenodm because xenodm starts cwm Window manager as default which I'm no familiar with...
to see how fast or slow your X-Window are you can run OpenGL demo, command glxgears you will see current FPS on terminal where you run it.

Here is important output from /var/log/Xorg.0.log

[   110.231] (--) NV: Found NVIDIA GeForce 210 at 06@00:00:0

Important cause of slow rendering - XAA acceleration fails:

(II) LoadModule: "xaa"
(WW) Warning, couldn't open module xaa
(EE) NV: Failed to load module "xaa" (module does not exist, 0)

The above behaviour (trying only XAA acceleration) as default is properly described on manual page: https://man.openbsd.org/nv.4

Option "AccelMethod" "string" -> Default: XAA

Here is reason why XAA always fails: https://www.phoronix.com/news/MTA0NDg to quote:

18 January 2012

The XAA 2D acceleration architecture is finally set to be stripped out of X.Org Server 1.13 and upstream open-source X.Org drivers.

and from Xorg.0.log:
```
X.Org X Server 1.21.1.6
```

So we have to enable EXA acceleration (that is still supported):

to enable EXA we have to create file, for example /etc/X11/xorg.conf.d/15-nv-exa.conf with contents:
```
Section "Device"
	Identifier "nv"
	Driver "nv"
	Option "AccelMethod" "EXA"
EndSection
```

and again startx - check in /var/log/Xorg.0.log that there are these messages:

(**) NV(0): Option "AccelMethod" "EXA"
...
(II) EXA(0): Offscreen pixmap area of 201320448 bytes
(II) EXA(0): Driver registered support for the following operations:
(II)         Solid
(II)         Copy
(II)         UploadToScreen

however in my case this driver has quite high CPU usage for some time... But anyway it seems to be better. TODO: How to benchmark... However it was rather Firefox issue with pledge(2) and unveil(2) (more available on text below).

Whole analysis and workaround posted here: https://marc.info/?l=openbsd-tech&m=168685386609760&w=2

One followup, responded that only G80 supports EXA. Other chipsest are out of luck:

https://marc.info/?l=openbsd-tech&m=169009691330774&w=2

Gory details for peoople that want to browse X11 sources:

you can find source-code here: https://cdn.openbsd.org/pub/OpenBSD/7.3/xenocara.tar.gz
use these instructions to unpack it: https://www.openbsd.org/faq/faq5.html
the 'nv' driver is under /usr/xenocara/driver/xf86-video-nv/src

for example:

cd /usr/xenocara/driver/xf86-video-nv/src
fgrep 'GeForce 210' *

nv_driver.c:  { 0x10DE0A23, "GeForce 210" },
nv_driver.c:  { 0x10DE0A65, "GeForce 210" },

How to build Xenocara (I plan to play little with NV driver to know what exactly is slow):

WARNING! Xenocara build will overwrite all files in existing /usr/X11R6 installation and also in /etc/X11/!

therefore dedicated machine/VM is strongly recommended for building it !
install fresh OpenBSD 7.3 (install73.iso or install73.img)
I used VM with 2 CPUs and 1GB of RAM, 50GB disk.
you may allow SSH login for root because Xenocara must be run as root anyway (for build it will use build user, however intermediate build results are installed as root into /usr/X11R6/ target - which is likely reason why Xenocara must be invoked as root).
I use single slice layout ('a' for '/' - 2GB, and 2GB used for slice b as swap)
Deselect X-Win packages from sets (with -x*) - they will be overwritten anyway and also skip games set (with -g*)
after reboot install at least curl(1), I also add vim(1)
```
pkg_add curl vim--no_x11
```
run system binary updates with: syspatch just to be sure...

install sources

# speed-up unpacking tars a bit - valid if you use single filesystem
mount -u -o async,noatime /
cd
curl -fLO https://cdn.openbsd.org/pub/OpenBSD/`uname -r`/src.tar.gz
curl -fLO https://cdn.openbsd.org/pub/OpenBSD/`uname -r`/sys.tar.gz
curl -fLO https://cdn.openbsd.org/pub/OpenBSD/`uname -r`/xenocara.tar.gz
tar xpfz src.tar.gz -C /usr/src
tar xpfz sys.tar.gz -C /usr/src
mkdir -p /usr/xenocara /usr/xobj
tar xpzf xenocara.tar.gz -C /usr/xenocara
chgrp -R wsrc /usr/src /usr/xenocara
chgrp -R wobj /usr/obj /usr/xobj

now BACKUP your system - really!!! (in case of VM - snapshot is handy)

after snapshot/backup boot, and issue (mostly following release(8) guide):

# all as root
cd /usr/xenocara
make bootstrap
make obj
# now run in tmux - it will take lot of time:
time make build
# build time:
# - bare-metal: 2 core AMD X2 @ 2GHz, 8GB RAM, SATA3 SSD disk: 2h:30m (same time with HDD)
# -         vm: 2 core Celeron @ 1GHz, 2GB RAM, libvirt/kvm: 3h:30m

Here is little patch to know what card I have:

--- /usr/xenocara/driver/xf86-video-nv/src/nv_driver.c.orig	Wed Jun 14 16:49:32 2023
+++ /usr/xenocara/driver/xf86-video-nv/src/nv_driver.c	Wed Jun 14 16:51:31 2023
@@ -941,8 +941,8 @@ NVPciProbe(DriverPtr drv, int entity, struct pci_devic
         name = "Unknown GPU";
 
     xf86DrvMsg(0, X_PROBED,
-               NV_NAME ": Found NVIDIA %s at %2.2d@%2.2d:%2.2d:%1.1d\n",
-               name, dev->bus, dev->domain, dev->dev, dev->func);
+               NV_NAME ": HP1: Found NVIDIA %s at %2.2d@%2.2d:%2.2d:%1.1d id=%d (0x%x)\n",
+               name, dev->bus, dev->domain, dev->dev, dev->func, id, id);
 
     if(NVIsG80(id))
         return G80GetScrnInfoRec(NULL, entity);

And here message from /var/log/Xorg.0.log:

fgrep HP /var/log/Xorg.0.log

(--) NV: HP1: Found NVIDIA GeForce 210 at 06@00:00:0 id=282987109 (0x10de0a65)

TODO: How to rebuild just driver:

it seems that this command is run in every directory:

su build -c 'exec make  -f Makefile.bsd-wrapper all'

I will try it for /usr/xenocara/driver/xf86-video-nv/
Here is nice hint for libX11: https://ftp.openbsd.org/pub/OpenBSD/patches/7.3/common/005_libx11.patch.sig
```
cd /usr/xenocara/lib/libX11
make -f Makefile.bsd-wrapper obj
make -f Makefile.bsd-wrapper build
```
so I can likely use something similar for driver.

How to restore corrupted X11:

if you interrupted X11 build it may end up with corrupted installation
the easisest way is to reinstall 'x*` sets:

rename old dirs:

mv /etc/X11 /etc/X11.old
mv /usr/X11R6 /usr/X11R6.old

boot from OpenBSD 7.3 CD (install73.iso) or directly boot bsd.rd (RAMDISK) kernel from your disk
select Upgrade
select proper disk (if you have more than one
select media for sets (in my case cd0)
deselect all sets with -*
select only X11 sets with x*
confirm installation without signature
confirm that you want skip upgrade of kernels (bsd*) and confirm that you want skip upgrade of base73
reboot after install

Useful applications:

you will likely to need install at least browser, for example:

doas pkg_add firefox-esr
# Firefox expects that this directory already exists
mkdir -p ~/.local/share/applications

WARNING! In my case there were too tight permissions (using pledge(2) and unveil(2) that prevented Firefox to create and access important directories under ~/.local and ~/.config As workaround I added disable keyword at the start of these files:
- /etc/firefox-esr/pledge.main
- /etc/firefox-esr/unveil.main

you can watch and scan for Permission denied using commands like:

ktrace /usr/local/bin/firefox-esr
# try to open file dialog - it will be empty and exit
# after exit:
kdump -f ktrace.out > kdump-firefox.lst
# WARNING! Those files are huge! in my case .out has 260MB and .lst has 780MB!
# now you can try:
fgrep -B 4 denied kdump-firefox.lst

these calls failed because of pledge/unveil:
```
~/.local/share/mime
~/.config/gtk-3.0
```
also in such setup I have no way to import Bookmarks from other Firefox...

or mp3 player and mixer:

# Warning! Has lot of dependencies
doas pkg_add audacious cmixer

How to add Firefox to Fvwm's menu:

to convert icons from .png to .xpm install ImageMagick using:
```
doas pkg_add ImageMagick
```

convert .png to .xpm:

cd /usr/local/share/pixmaps/
doas convert -scale 16 firefox-esr.png mini.firefox-esr.xpm

copy system wide configuration file to your home directory:

cp /usr/X11R6/lib/X11/fvwm/.fvwmrc ~/.fvwmrc
chmod u+w ~/.fvwmrc

apply following patch:

--- /usr/X11R6/lib/X11/fvwm/.fvwmrc	Sat Mar 25 17:58:48 2023
+++ .fvwmrc	Sat Jun 17 10:36:48 2023
@@ -5,7 +5,7 @@
 
 # be sure to customize these paths for your installation:
 ModulePath /usr/X11R6/lib/X11/fvwm
-PixmapPath /usr/X11R6/include/X11/pixmaps
+PixmapPath /usr/X11R6/include/X11/pixmaps:/usr/local/share/pixmaps
 IconPath   /usr/X11R6/include/X11/bitmaps
 
 EdgeResistance 250 10
@@ -95,6 +95,9 @@
 ######################## Menus ###################
 AddToMenu RootMenu	"Root Menu"	Title
 +			"XTerm%mini.xterm.xpm%"		Exec exec xterm
++			"Firefox ESR%mini.firefox-esr.xpm%"	Exec exec /usr/local/bin/firefox-esr
++			"Audacious mp3"	Exec exec /usr/local/bin/audacious
++			"Audio mixer"	Exec exec xterm -e /usr/local/bin/cmixer
 #+			"Rxvt"		Exec exec rxvt
 +			""		Nop
 #+			"Remote Logins"	Popup Remote-Logins

WARNING! You may have troubles when doing Copy & Paste of above patch, because some characters are spaces and some are tabs.

Optional: run also xload (shows system load) on X11 startup:

diff -u /etc/X11/xinit/xinitrc{.orig,}
--- /etc/X11/xinit/xinitrc.orig	Sat Jun 17 10:06:49 2023
+++ /etc/X11/xinit/xinitrc	Sat Jun 17 10:08:58 2023
@@ -31,6 +31,7 @@
 fi
 # start some nice programs
 xclock -geometry 50x50-1+1 &
+xload -geometry 60x60-1+62 &
 xconsole -iconic &
 xterm -geometry 80x24 &
 fvwm || xterm

Problems:

Spotify does not work:
- OpenBSD does not support binary DRM plugin (called widevine) required to use DRM content in Spotify, Netflix,....
- free ncspot package requires Premium account. It will throw error Login failed with reason: Premium account required

Unsorted tips

Installing sets after install

WARNING! It is not that easy as just expanding set.tar.gz, because some files have different target directories, especially those under /etc - because of merging files.

So please avoid temptation to just unpack .tar.gz somewhoere (I know what I'm talking about)...

Recommended way is described here: https://www.openbsd.org/faq/faq4.html#FilesNeeded

Adding a File Set After Install

If you chose to skip some file sets at install time, you might realize later that you really do need them after all. Simply boot bsd.rd from your root file system and choose (U)pgrade. When you get to the list of file sets, select the ones you need.

exFAT support

Motivation:

having dedicated disk for backup
problem is which filesystem is supported by all operating systems
my candidate is exFAT (supported well under Linux and of course under Win7+)

Seems to be there: https://romanzolotarev.com/openbsd/exfat.html

# to install
doas pkg_add exfat-fuse
# to list files in installed package
pkg_info -L exfat-fuse

Example for my disk - finding disklabel (last partition it correct in my case):

# getting list of disks, cd's and Floppy!

sysctl hw.disknames

hw.disknames=cd0:,sd0:0de15a47afbba18c,sd1:c8447de41b7ea394,sd2:,fd0:

dmesg | fgrep sd1
# yes looks like right disk

# how it looks under fdisk
Disk: sd1       Usable LBA: 34 to 1953525134 [1953525168 Sectors]
   #: type                                 [       start:         size ]
------------------------------------------------------------------------
...
  12: Microsoft basic data                 [  1830813696:     83886080 ]

# Finding right partition name:
doas disklabel sd1

...
#                size           offset  fstype [fsize bsize   cpg]
...
p:        209714168       1491076096 unknown

### IT IS WRONG letter - see text below...

doas mkdir -p /mnt/exfat
doas /usr/local/sbin/mount.exfat-fuse /dev/sd1p /mnt/exfat/ 

FUSE exfat 1.2.8
WARN: partition type is not 0x07 (NTFS/exFAT); you can fix this with fdisk(8).
ERROR: exFAT file system is not found.

PROBLEM:

OpenBSD disklabel support only 16 slices - from a: to p:
but becuase synthetic slices starts from i: you are simply out of luck if you have GPT with more than 8 partitions (it is my case)

Here is correct partition id:

from https://en.wikipedia.org/wiki/Microsoft_basic_data_partition
it should be EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 which is basic data partition (BDP)
under Linux it is Microsoft basic data

doas fdisk sd1
Disk: sd1       Usable LBA: 34 to 1953525134 [1953525168 Sectors]
   #: type                                 [       start:         size ]
------------------------------------------------------------------------
 12: Microsoft basic data                 [  1830813696:     83886080 ]

The problem - not all partitions are shown in disklabel (only from i: to p:) so I can't access it.

Using firewall

Here is my /etc/pf.conf (modified from default version, comments omitted):

set skip on lo
block return log	# block stateless traffic
pass		# establish keep-state
# Block all connections to this host, but SSH
block return in log on ! lo0 proto tcp to port != ssh
# Port build user does not need network
block return out log proto {tcp udp} user _pbuild

Important commands:

to validate pf.conf: pfctl -n -f /etc/pf.conf
to apply pf.conf: pfctl -f /etc/pf.conf
to dump status: pfctl -s all

To decode /var/log/pflog

from https://www.openbsd.org/faq/pf/logging.html
decode existing log entries: tcpdump -n -e -ttt -r /var/log/pflog
decode in real-time: tcpdump -n -e -ttt -i pflog0

Wayland on OpenBSD

Read this article:

https://x61.sh/log/2024/05/27052024192206-sway_openbsd.html

Unfortunately you need DRM supported device - according to man drm only these cards have DRM support (OpenBSD 7.5):

amdgpu(4)     AMD Radeon GPUs using the amdgpu kernel driver
intel(4)      Intel integrated graphics chipsets
radeon(4)     ATI/AMD Radeon video driver

Note that nvidia (nouveau) is not there. Also KVM QXL is not there. If you have unsupported hw - when you run startsway.sh it will end-up with errors:

[ERROR] [wlr] [libseat] [libseat/backend/noop.c:57] Failed to open device: Device not configured
[ERROR] [wlr] [backend/session/session.c:321] Failed to open device: '/dev/dri/card0': Device not configured
[ERROR] [wlr] [backend/session/session.c:410] Unable to open /dev/dri/card0 as DRM device
[ERROR] [wlr] [backend/backend.c:210] Found 0 GPUs, cannot create backend
[ERROR] [wlr] [backend/backend.c:376] Failed to open any DRM device
[ERROR] [sway/server.c:137] Unable to create backend

Please note that unlike current Linux (with devtmpfs and others) the existence of device /dev/dri/card0 does NOT mean that there exists driver for it(!). Devices are allocated statically on OpenBSD (some optional with /dev/MAKEDEV script).

Resources

Interesting resources:

https://www.openbsd.org/innovations.html
https://man.openbsd.org/gcc-local.1
- I really appreciate:
The -O2 option does not include -fstrict-aliasing, as this option causes issues on some legacy code. -fstrict-aliasing is very unsafe with code that plays tricks with casts, bypassing the already weak type system of C.
- strict aliasing basically means that two pointers are not allowed to point to same address (data). If you violate that - for example write data using 1st pointer and read data using 2nd pointer the compiler may not work properly (it may use value cached in register for example). This situation may occur even when you write one member of union and read data using other member of same union (!)
https://playground.boxtec.ch/lib/exe/fetch.php/publ/practical_gsec_2508.pdf
https://learn.microsoft.com/en-us/azure/virtual-machines/linux/create-upload-openbsd
https://calomel.org/ids_mtree.html

OpenBSD - hpaluch/hpaluch.github.io GitHub Wiki

OpenBSD

Notes on environment

Notes on USB pendrive install

Linux to OpenBSD glossary

Notes on partitioning

Notes on MBR install

Booting OpenBSD from GPT+BIOS

Notes on packages

Smartctl

Virtualization

How to use tags in Vim

OpenBSD trap when installing from libvirt/KVM IDE CD

OpenBSD as GUI Workstation

Sound support

X-Window support

Unsorted tips

exFAT support

Using firewall

Wayland on OpenBSD

Resources

⚠️ GitHub.com Fallback ⚠️

OpenBSD - hpaluch/hpaluch.github.io GitHub Wiki

OpenBSD

Notes on environment

Notes on USB pendrive install

Linux to OpenBSD glossary

Notes on partitioning

Notes on MBR install

Booting OpenBSD from GPT+BIOS

Notes on packages

Smartctl

Virtualization

How to use tags in Vim

OpenBSD trap when installing from libvirt/KVM IDE CD

OpenBSD as GUI Workstation

Sound support

X-Window support

Unsorted tips

exFAT support

Using firewall

Wayland on OpenBSD

Resources

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️