OpenAMP remoteproc Subgroup Meeting Notes 2022 - OpenAMP/open-amp Wiki

Table of Contents

Introduction

This sub-group covers areas such as remoteproc, rp-message, virtio, big buffers, etc. The meeting cadence is every 2 weeks on Thursdays at 11 am Eastern US time.

2022-08-11

  • Bill Mills (Linaro)
  • Tanmay Shah (Xilinx)
  • Ben Levinsky (Xilinx)
  • Ed Mooring (Independent)

Kernel

  • Mathieu can not join today
    • His last patch queue update was on July 27 and is below
    • He reported at that time that the list is unlikely to change before Aug 15
  • [PATCH] Add support for MT8195 SCP 2nd core
  • [PATCH] Add Xilinx RPU subsystem support
  • [PATCH] remoteproc: imx: add start up delay
  • [PATCH] remoteproc: imx_rproc: support i.MX8QM/QXP
  • [PATCH] Add support for WASP SoC on AVM router boards

library

  • Arnaud is out, should be back next week
  • EM: Tammy and XXX have been submitting multiple PRs
    • EM: Tammy's work is fixups for safety certifiablility coding standards
    • EM: May be related to host restart
  • BL: Xilinx needs remoteproc & rpmsg in u-boot
    • BM: other people have remoteproc support, not sure about rpmsg
  • BL: remoteproc recovery from host failure, seems like some stuff may be missing
    • BM: should be a supported use case but would not be surprised is things were missing
    • BM: We need test cases for Host orderly shutdown and
    • EM: Need arch approach
  • BL: upstream kernel rpmsg driver, how does it get activated?
    • BM: activated by name match on channel announcement, see tty_rpmsg.c line 216 or similar

2022-07-14

  • Bill Mills (Linaro)
  • Mathieu Poirier (Linaro)
  • Tanmay Shah (Xilinx)
  • Tammy Leino (Mentor/Siemens)
  • Ben Levinsky (Xilinx)
  • Arnaud Pouliquen (ST)

Kernel

  • [PATCH] Introduce PRU remoteproc consumer API
    • MP: should finish this one soon
  • [PATCH] Introduction of rpmsg_rx_done
    • MP: Request Arnaud to look at this WRT flow control etc
  • [PATCH] Add support for MT8195 SCP 2nd core
  • [PATCH] Add Xilinx RPU subsystem support
    • MP: will get to this one this period
  • [PATCH] remoteproc: restructure the remoteproc VirtIO device
    • MP: may get to this one before next call
  • [PATCH] remoteproc: imx: add start up delay
  • [PATCH] remoteproc: imx_rproc: support i.MX8QM/QXP
  • Other
    • MP: Thanks to Arnaud for jumping into upstream virtio changes that have broken rpmsg

general openamp lib

  • MP: things are slowed slowed down for summer
  • TL: when does Xilinx update openamp lib from upstream
    • BL TS: at least 1 per year, 20XX.1

2022-06-30

  • Bill Mills (Linaro)
  • Mathieu Poirier (Linaro)
  • Tanmay Shah (Xilinx)
  • Tammy Leino (Mentor/Siemens)
  • Ben Levinsky (Xilinx)
  • Arnaud Pouliquen (ST)
  • Suman Anna (TI)
  • Ed Mooring (Independent)
  • Hari Nagalla (TI)
  • Segei Korneichuk (AMD/Xilinx)
  • Loic Pallardy (ST)

Kernel

  • [PATCH] remoteproc: pru: Enable PRU cores on K3 AM62x SoCs (June 02, Kishon)
  • [PATCH] remoteproc: restructure the remoteproc VirtIO device (June 3rd, Arnaud)
  • [PATCH] Introduce PRU remoteproc consumer API (June 6th, Puranjay)
  • [PATCH] Add support for MT8195 SCP 2nd core (June 8th, Tinghan)
  • [PATCH] Introduce helpers for carveout management (June 10th, Chris Lew)
    • Looking for people to review this one.
  • [PATCH] virtio pci support VIRTIO_F_RING_RESET (June 29th, Xuan Zhuo)
https://lore.kernel.org/lkml/[email protected]/t/

Status emails: OK for everyone?

general openamp lib

  • EM: AP added mmio stuff on virtio-mmio
  • EM: AP Flow control
  • EM: AP recycle buffers
    • Pending Buffers when endpoint is destroyed
    • Host: just clean up in descriptor table
    • Device:
    • Busy status: reserved field points to descriptor idx
  • AP: virtio memory buffer

2022-06-02

x Bill Mills (Linaro) x Mathieu Poirier (Linaro) x Tanmay Shah (Xilinx) x Tammy Leino (Mentor/Siemens) x Ben Levinsky (Xilinx) x Arnaud Pouliquen (ST) x Suman Anna (TI) x Ed Mooring (Independent) x Hari Nagalla (TI) x Segei Korneichuk (AMD/Xilinx) x Loic Pallardy (ST)

Kernel

  • MP: back to patch review cycle
    • Pang's patch set, what to do after crash
    • Arnaud's virtio restructure, maybe wait until other work is done?
      • 1st step toward making it a platform device then declare in DT
      • MP: also fixes the DMA api usage to be more clean
    • TI PRU consume API
    • Xilinx remoteproc v7

general openamp lib

AP: no big work except virtio-exp and flow control TL: Petalinux 2019.2/1 remoteproc.h has different struct in trace resource TL: endpt problem was running out of memory

virtio-exp

AP: virtio-exp what is the policy BM: lets follow the coding standards from the start in the future BM: This branch should be continous history AP: Will accpet Dan's PR EM: OK for me

flow control for virtio

AP: Don't really like the last patch AP: Other features, aligned with BM: Do we need a feature bit? AP: BM: NAK what you don't like AP: control ept as name

rpmsg restart

AP: How to move forward on virtio? move meeting and 3 people

2022-05-19

Bill Mills (Linaro) Tanmay Shah (Xilinx) Tammy Leino (Mentor/Siemens) Ben Levinsky (Xilinx) Segei Korneichuk (AMD/Xilinx)

Kernel

  • Mathieu could not attend today so we will skip kernel topics

openamp lib

  • AP: Interest in virtio from multiple parties
    • Dan has responded w/ pointers to his previous work
    • Dan is expected
  • AP: VDEV status
    • Ben has ok'ed
    • BM:Does this violate the spec?
      • OK to skip checks if no new kick from host
      • I would like to hold off until we know if this follows the spec or not
  • AP: remote end point disconnect call rpmsg-dev not just EP callback
    • Custom rpmsg device

rpmsg restart

  • Crash & Recovery
    • BM: the kernmel cleans up after remote crash
      • does the library support this also?
      • A lot depends of the library user but

flow control for virtio

  • AP: will push flow-control work, RFC to kernel and PR to openamp
    • will depend on QC patches that are pending

2022-05-05

Bill Mills (Linaro) Mathieu Poirier (Linaro) Tanmay Shah (AMD/Xilinx) Tammy Leino (Mentor/Siemens) Ben Levinsky (AMD/Xilinx) Arnaud Pouliquen (ST) Ed Mooring (Independent) Segei Korneichuk (AMD/Xilinx)

rpmsg restart

  • BL: customer wants to re-establish the remoteproc
    • Linux crashes and want R5 to stay alive
    • What to do when Linux comes up again
      • Xilinx remoteproc does not re-init on queue, need to relook at
  • AP: ST rproc does this, take a look at that

Kernel

  • MP: Looking at Arnaud set destination address 20 lines
  • MP: done with most easy ones
  • MP: still leaves harder ones like:
    • Pang's support for attach then crash what to do
    • Fix DMA leak when remoteproc is going down
  • MP: Xilinx remoteproc, Suggest V5 for address Rob's comments

openamp lib

  • AP: Release is done, looking at pending PR
    • Split RX buffer & rings and TX Buffer & rings areas
    • Each side needs to know the state
    • use case is not Linux
  • AP: remove check of device status

Other topics

  • AP: no update on flowctrl
    • 20 line patch to set destination is alternative to full flow control

R5 attach/detach

  • Ben wants pointers to the ST code on both

AP

  • added Ben to PR at openamp

Tanmay

  • back compatibility at source level and wire level?
    • BM: per TSC, back compat are wire level (once upstream). Any exception needs TSC approval.
    • EM: source level is 2 years back

2022-04-21

Bill Mills (Linaro) Mathieu Poirier (Linaro) Tammy Leino (Mentor/Siemens) Ben Levinsky (Xilinx) Arnaud Pouliquen (ST) Ed Mooring (Independent) Segei Korneichuk (AMD/Xilinx)

Kernel

  • MP: cleared most of the backlog, left
    • AP: set dest address on first message rx
      • maybe wait on flow control patches coming
    • Pang: support for self recovery
    • Mediatek: AngeloGioacchino Del Regno, Fix circular locking dependency
      • Still trying to figure out / verify the issue
    • Hangyu Hua fixing double free

openamp lib

  • AP: approaching release, some pull request still pending
    • EM: objectionable terms, #defines, cmake options etc
    • TL: split child memories
      • support for suspending one side of the conversation
      • probably RTOS to RTOS only feature
  • TL: asking what to do with legacy NS feature bit negotiation
  • AP: zero copy issue
  • TL: Assert vs error propagation has similar issue

flow control for virtio

rpmsg restart

  • ran out of time

2022-04-07

Bill Mills (Linaro) Mathieu Poirier (Linaro) Tanmay Shah (Xilinx) Ben Levinsky (Xilinx) Arnaud Pouliquen (ST) Suman Anna (TI) Ed Mooring (Independent) Segei Korneichuk (AMD/Xilinx)

Kernel

  • MP: got pulled into other things
  • MP: still on todo list: control flow and mediatek
  • MP: 11 patch sets in queue, will do easy ones first
  • MP: Shengjiu Wang: correct behavior on sysfs
    • start -> sysfs/state, currently return BUSY if already running
  • MP: Puranjay Mohan, TI pruss: 5000 lines
    • SA: Is RFC of multiple subsystems
      • Remoteproc is just one
  • AP: WRT flow control
    • working on proof of concept to add the same flow control as QC for vdev
    • QC channels are point to point so flow control is better defined
    • hard part for vdev is multipoint to point

openamp lib

  • EM: code freeze on April 18
    • EM: Remove objectionable terms
    • EM: Tamey's fixes are pretty much going is as they come in.
  • BL: rpmsg ept recovery, working on proposal
    • single end point dies
    • remoteproc crash detection & recovery
    • remoteproc char driver
    • rpmsg char returns error after crash
    • EM: RTOS to RTOS does not have support for this
    • rpmsg only when Linux crashes
    • library does not have crash / recovery support
    • today's support is in platform specific code in kernel
      • mailbox crash code (relies on self report)
      • HW watchdog remoteproc -> interrupt to host
      • Exception report to host (IOMMU fault)
    • AP: Vdev status should be managed
    • MP: crash detection is all vendor specific

2022-03-24

Bill Mills (Linaro) Mathieu Poirier (Linaro) Tanmay Shah (Xilinx) Tammy Leino (Mentor/Siemens) Arnaud Pouliquen (ST) Suman Anna (TI) Ed Mooring (Independent) Hari Nagalla (TI) Segei Korneichuk (AMD/Xilinx)

Kernel

  • MP: Arnaud's refactor of rpmsg-char is now accepted in rproc-next
  • MP: TI ipc-only patches are accepted
  • MP: on todo AP's virtio refactor
  • MP: all in all 8 to 10 on list. expect to be done with all before next meeting
  • AP: comments on QC control flow patch series, we should do virtio transport also
    • Could be a solution to connect ack issue in openamp

openamp lib

  • AP: new issue from Tammy about zero copy
    • needs API to drop buffer
TL: Will have some time to make new PR
    • found about 10 issues, all publicly documented in openamp issues
    • what are the requirements for a PR?
    • AP: currently: don't break what is there, test on your platform
  • AP: code freeze in mid April

general virtio expansion

  • BM: we have had several meetings over the past 2 month on how to join the
  • openamp lib with the general virtio support done by Wind River in the Zephyr fork
  • The plan as of now is
    • move Zephyr fork to github/openamp
    • modify Zephyr code to use virtio layer from openamp lib
    • add virtio-mmio to openamp lib (using Vincent's and WR's code)
    • add SCMI server code as 1st openamp resident driver
    • get this far and re-evaluate
    • work toward moving as much code to openamp lib as makes sense but maintain
good Zephyr integration (like device model integration etc)

2022-03-10

Bill Mills (Linaro) Mathieu Poirier (Linaro) Tanmay Shah (Xilinx) Tammy Leino (Mentor/Siemens) Arnaud Pouliquen (ST) Suman Anna (TI) Ed Mooring (Independent)

Kernel

  • MP: spent most of review time on Arnaud's refactor patches
    • MP: looked at this very hard w/ focus
  • MP: started looking at Pang's series for refactor startup & crash recovery
    • stopped review due to quality issue and asked for better series
    • put at end of queue
  • MP: started looking at Suman patch
    • Suman: any chance of hitting 5.18
    • MP: probably not
  • MP: next mediatek 8186 8183
  • MP: Daniel kestrel v2 (Borjn looked at v1 assume he will do v2)
  • SA: Will Arnaud's refactor be for 5.18?
    • MP: His work is coming in 4 waves
    • MP: Arnaud please drop the RFC for next cycle
    • MP: when you move to next wave, keep wave 1 in series

library

  • EM: library is
  • EM: PR 347 new API for "wait_tx_buffer"
    • Should the issue be solved by local side or remote side?
    • if local should it be at app, bsp, or lib level
  • EM: Tammy has been looking at certification issues

Virtio Library add on

  • AP: Really want to avoid adding a new library
    • already too complex with libmetal
  • WM: motivation for extra lib was
    • keep existing lib focused
    • allow more freedom of config to make app dev's job easier
    • WM: example ST added a new lib to make app dev job easier
    • AP: ST's extra library was to simplify libmetal startup
    • EM: adding a new layer in the library to make app dev life easier is OK
  • WM: why can't we create the Zephyr glue in openamp
    • AP: cmake and west support
    • AP: could use git submodule
    • AP & BM: try west manifest not in Zephyr

2022-02-24

Bill Mills (Linaro) Mathieu Poirier (Linaro) Tanmay Shah (Xilinx) Tammy Leino (Mentor/Siemens) Arnaud Pouliquen (ST) Hari Nagalla (TI)

Kernel

  • MP/TS: Xilinx Remoteproc V3 finished review ready for V4
    • TS: Target V4 in 2 weeks
  • MP: Someone interested in AP's work on virtio restructure
    • on queue for next week
    • Pang's patch on recovery is also on review list for next week
      • AP has tested Pang's patch, looks OK
      • MP: Request review from TI/Hari
  • MP: Xuan Zhuo: support virtio ring reset
  • MP: Christoph Konstaph: refactor static memory
  • MP: Suman R5 TI K3 IPC only mode

Library

  • AP: Tanmay's suggestions for testing
  • AP: Jeramie no buffer handling, can call API or wait
    • Hookable function to wait
  • TL: what is the policy for API parameter checking?
    • BM: In some situations the library size must be very small so need to keep 
        that in mind.  Could be an option?
    • AP: suggest Tammy point out the specific instances she sees and then discuss

2022-02-10

Bill Mills (Linaro) Mathieu Poirier (Linaro) Tanmay Shah (Xilinx) Hari Nagalla (TI)

Note

Short meeting today. Lots of people are out so only went over kernel aspects.

Kernel

  • MP: got Tanmay V3 remoteproc for Xilinx
    • will probibly review this next week as below item is a bit stalled
  • MP: Pang reworking crash/attach/detach
  • MP: Arnaud reworking remoteproc virtio
    • Asking Arnaud, Pang, and Hari to cross check the above
  • MP: sysfs remoteproc read-only from Puranjay at TI
    • done on a per remoteproc basis
    • coming from AM335x wakeup M3 which is not for user use, it handles suspend/resume low level work

2022-01-27

Bill Mills (Linaro) Tammy Leino (Siemens: past Nucleus, network and higher level stack) Tanmay Shah (Xilinx) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) Suman Anna (TI) Hari Nagalla (TI)

Kernel

  • MP: still in a backlog, 12 series 50/50 between coresight and rp
  • next up K3 M4F
  • Pang trying to complete attach/detach
    • MP asked HW support from attach/detach
  • Arnaud's next gen: rpmsg char driver
  • AP: will look at attach/detach patch from ST POV
    • will need to enhance it
  • AP: new version of rpmsg-char
    • Qualcomm also has fixes for rpmsg-char so need to figure out order

Library

  • EM xtensa support still going on
    • API change to add "const"
  • AP: Vincent's changes for SCMI
  • AP: multi endpoint on one channel broken
    • BM where is the broken part?
      • protocol?
      • libopenamp?
      • kernel API?
      • rpmsg_char?
    • We really need a rpmsg spec independent of kernel & openamp library

2022-01-13

Bill Mills (Linaro) Tanmay Shah (Xilinx) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) Loic Pallardy (ST) Hari Nagalla (TI)

Kernel

  • MP: in a backlog 11 patch sets between coresight and remoteproc
    • also doing rust maintenance
  • MP: still keeping an eye on Arnaud's rpmsg char driver rework
    • AP: to resend same version rebased to tip on Jan 24
    • MP will ping Bjorn at that time

Library

  • AP: new platforms to support RISC-V & extensa
  • AP: will add CI tests for these platforms
  • AP: library can define independent buffers for RX and TX
    • each rpmsg instance can be configured independently
    • new API for configurable sizes keeping existing API for legacy
  • AP: PR to remove deprecated APIs will be put off due to 2 year policy

New Topics

  • BM: Looked at Zephyr on virtio: https://github.com/danmilea/zephyr
    • Should compare with Vincent Guittot's SCMI on Zephyr work
    • Coexistence with independent virtio layers
      • need to share mailboxes? Shared Mem carve-outs?
    • WR code has virtio-mmio
    • WR has not yet shared kernel code for mailbox and memory access from user space
⚠️ **GitHub.com Fallback** ⚠️