OpenAMP remoteproc Subgroup Meeting Notes 2021 - OpenAMP/open-amp GitHub Wiki
This sub-group covers areas such as remoteproc, rp-message, virtio, big buffers, etc. The meeting cadence is every 2 weeks on Thursdays at 11 am Eastern US time.
Bill Mills (Linaro) Tanmay Shah (Xilinx) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) Loic Pallardy (ST) Suman Anna (TI)
- TS: Xilinx R5 V2
- MP: TI R5 device tree binding went through remoteproc tree, w/o Rob's sign off
- Verified the above with Suman
- TS: changed to use std power domain bindings etc
- MP: will address Rob's comments
- MP: make bindings in current patch the baseline and define sysdt based on that
- MP: deal with TCM statically as is and deal with dynamic power of TCM later w/ sysdt
- TS: V3 after holiday
- MP: TI R5 device tree binding went through remoteproc tree, w/o Rob's sign off
- MP: Arnaud v8 dec 7: working with Bjorn
- MP: Arnaud rpmsg tty: progressing with tty maintainer
- EM: a few open PRs, little things
- EM: PR for dynamic buffer size for non-Linux use case
- enable RX and TX buffers be different sizes
- dynamic at lib init time (like from main() )
- EM: terminology change, "API" change => config #defines
- Second pass on comments and documentation
- EM: already renamed the default branches and adjusted CI
- AP: Stratos meeting SCMI virtio by Vincent
- Modified openamp virtio we think, needs to be confirmed
- MP: two VMs Linux/Linux or Linux/Zephyr (Zephry as SCMI server)
- LP: would like to apply to TFA or coproc
- LP: ARM would like to use kernel allocated memory even if it needs memcpy
- LP: OpenAMP should do work here to define approaches better
- https://docs.google.com/drawings/d/1YSuJUSjEdTi2oEUq4oG4A9pBKSEJTAp6hhcHKKhmYHs/edit
- AP: ST github meta layer to use upstream kernel, TFA, U-boot, OPTEE
- https://github.com/STMicroelectronics/meta-st-stm32mp-oss
- AP: adding Zephyr as well
- WAM: next meeting will be Jan 13
Bill Mills (Linaro) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) Loic Pallardy (ST) Suman Anna (TI) Hari Nagalla (TI)
- MP: Tanmay's Xilinx R5
- need to fix checkpatch and compiler warnings
- on deck for review
- MP: Arnaud's refactor of rpmsg_char driver
- Bjorn will review next as high priority
- MP: Arnaud's fix for used length
- 5.16 rpmsg over virtio does not work anymore
- Legacy specifically requires ignoring length check
- AP's fix is a copy of other legacy virtio implementations
- AP also has fix in openamp but kernel should still be made legacy compat
- MP: Looked at Hari's patch and applied
- MP: removing Ohad from maintainers
- EM: Few pull requests, mostly minor changes
- Branch renames, in progress (nothing in libmetal)
- Ed working on terms in openamp
Bill Mills (Linaro) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) Loic Pallardy (ST) Hari Nagalla (TI)
- MP: 5 patch sets I am looking at now
- Arnaud's rpmsg_char driver v7
- Arnaud misc docs & kprintf
- iMX resource table
- R-Car remoteproc
- AP: a few things
- fix race condition issue
- fix vring id, in remoteproc
- Where does remoteproc from lib get tested?
- QEMU test using
- Nuttx uses remoteproc from library
- usage with C++ (from ST usage)
- Multi-virtio in DT in Zephyr PR, from Carlo from baylibre
- https://github.com/zephyrproject-rtos/zephyr/pull/40365
- LP: we should make progress on kernel side so we don't diverge
- BM: MCU to MCU STM32H747 on upstream Zephyr
- AP: someone wants to use rpmsg between two boards
- iMX and RPI probably using UART
Bill Mills (Linaro) Tanmay Shah (Xilinx) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) Suman Anna (TI) Hari Nagalla (TI)
- MP: Looking at "Initial Renesas R-Car remoteproc support" series by Julien Massot
- pretty simple, only 330 lines including DT
- MP: watching Bjorn/Arnaud rpmsg_char refactor control / data
- MP: Bjorn's input seems OK
- AP: Agree mostly, not closed yet is creation of default end-point
- MP: will let Bjorn review as we (MP/AP) co-designed
- MP: AP should send new version ~11/09 with not-decided section highlighted
- AP: 1st priority is to close rpmsg_char (not the virtio via DT)
- SA: we are using rpmsg_char
- HN: Have not looked yet, will try to look
- SA: Still owe patches on IPC only mode for TI platform
- scope is R5 and DSPs across all of K3
- EM: Release 2021.10
- EM: Terminology changes are targeted to do now
- Rename master branch still pending
Bill Mills (Linaro) Tanmay Shah (Xilinx) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) x Loic Pallardy (ST) Suman Anna (TI) Hari Nagalla (TI)
- AP: RPMSG spec document
- Bjorn is suggesting a specific use for the field
- defer for a month or so
- MP: rpmsg tty driver, picked up by Greg KH
- MP: Deepak intro signaling API to rpmsg, using GLINK today
- want to do flow control, decided to reuse RTS like flow control on rpmsg_char
- Not tested on Virtio, hard codes the signaling
- MP: looking at vdev from DT patch
- looks good but need multiple reviewers (Rob etc)
- Release at end of Oct
- AP: has finished his testing
- Build in different env
- Zephyr & Bare-metal GCC, Keil compiler, MISRA like tools
- Make my own copy to test in ST env for Bare-metal
- Make a copy for Zephyr w/ some glue
- When does Zephyr pick up the change?
- Minimize Zephyr clean-up
- Doing this is Openamp-ci
- will send PR to Zephyr after OpenMAP release
- EM: will do his testing and then do release notes
- test Virtual Xilinx HW, Baremetal on R5 w/ Linux, test examples
- goes direct to openamp github
- AP: has finished his testing
- AP: suppress inline functions in headers for rust support
- Applies to Python and maybe other languages also
- Today moving inline to *.c files for everybody
- Increase libmetal, decrease libopenamp, overall size increase
- Maybe the start of libmetal rework to skinny down for minimal systems
- EM: Xang Cadence DSP support (iMX 8)
- BM: open toolchain for build tests?
- AP: the patch is very small for libmetal and not that different than libmetal patches we carry for other platforms that we don't test for
- AP: Would be good to build test for NuttX
Bill Mills (Linaro) Tanmay Shah (Xilinx) Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Loic Pallardy (ST) Ed Mooring (Independent) Suman Anna (TI) Hari Nagalla (TI)
- MISRA-C for libmetal & libopen-amp is under investigation, it is not a committed requirement
- Step 1: pick the rules
- Align to Zephyr rules
- Step 2: leverage open source tool cppcheck as much as possible
- expect ~50% coverage of the rules
- Zephyr is not using cppcheck so no help there
- Xen has pull requests comming in to add support
- wait a bit for Xen to work things out then see what we can use
- Step 3: add a commercial tool to get better coverage
- Zephyr is partnering with Parasoft
- As openamp libraries are used in Zephyr, see if we can get checks done that way
- Questions for Zephyr:
- what SW components are going to be checked
- when are the checks going to run (nightly/weekly CI, every pull-request?)
- Step 1: pick the rules
- Maintainer update
- rpmsg remoteproc tree is now official shared
- https://git.kernel.org/pub/scm/linux/kernel/git/remoteproc/linux.git/
- it is being pulled for linux-next and will be source for next merge cycle
- rpmsg_tty
- a bit of push back on rpmsg for of i2c/spi/gpio
- group opnion is rpmsg *is* appropriate for low speed peripherals
- rpmsg should *not* be used for high speed or complex peripherals like virtio-net, virtio-blk, virtio-gpu etc
- rpmsg is less HW resource intensive than multiple virtio vdevs
- rpmsg can work over other transports like serial link to remote MCU
- Problems with multiple virtio vdevs
- doorbell sharing is harder
- mbox API is not easy to share
- memory requirements are harder
- Suggestion: show what HW would look like for I2C, 2 UART, GPIO and 3 SPI with a virtio vdev for each vs one rpmsg
- ST multiple vdev
- DT vdev
- Xilinx remoteproc vdev-rpmsg for userspace w/ vdev-rpmsg for kernel
- same IRQ for both -> use different IRQs
- 2nd channel triggers uio
- use DT to do one or the other for now?, Tanmay: OK
- Xilinx remoteproc driver is being worked on but still waiting
- Upstream kernel support for ST MP1 w/ remoteproc
- upstream kernel does not use SCMI for power & clk control
- current official ST firmware does have SCMI server which conflicts with kernel
- MP is using a special firmware delivery from ST so does not see the issue
- upstream should work if firmware SCMI server is disable, AP will try
- MP is looking at 3 patch sets, all from AP
- rpmsg_char driver
- virtio rework
- rpmsg_tty (v9 coming this week)
- release at end of Oct
Mathieu Poirier (Linaro) Arnaud Pouliquen (ST) Ed Mooring (Independent) Bill Mills (Linaro) Tanmay Shah (Xilinx) Suman Anna (TI) Hari Nagalla (TI) Bruce Ashfield (Xilinx)
- Terms update
- Bill to send discussion from TSC list to openamp-rp list
- Ed/Arnaud to figure out plan to rename github branches from master -> main after the October release
- Xilinx R5 driver upstream
- TS: Start small with just remoteproc driver, add rpmsg in second round
- HN: TI: Heads up, New Driver AM64x for Cortex-M4 coming soon
- AP: Joint kernel tree at kernel.org
- Mathieu and Bjorn will be able to push to this tree
- In place but not yet defined as the official path
- Baylibre support for NAI APU
- Interesting work that uses DRM w/ remoteproc and rpmsg
- Getting ready for release
- Possible PR:
- Compiler inlining help
- New Linux demos
- Possible PR:
- rpmsg tty
- patches for new version will be coming soon
- new version is simpler (got rid of sideband control)
- old version had support for RTS/CTS signaling
- define vdev in DT (see presentation)
- Mathieu Poirier (Linaro)
- Arnaud Pouliquen (ST)
- Ed Mooring (Indipendent)
- Bill Mills (Linaro)
- Loic Pallardy (ST)
- Etsam Anjum (Mentor)
- Bruce Ashfield (Xilinx)
- Xilinx R5 driver upstream
- New Xilinx person, Tanmay
- ST rpmsg char driver enhancement
- Not in pull for 5.15, don't know why
- Other patches missing
- MP: will give update next meeting
- ST needs multiple virtio
- virtio net, audio etc
- use DMA api to allocate to vdev
- i.MX DSP driver
- several others waiting for next patch series version
- AP: Greg from NXP to run MISRA
- NXP IAR tool used to find intital issues
- ACTION: Talk to TSC
Missing notes, see transcript: https://drive.google.com/file/d/15bj4tdvoWNNBneTYKcg_ICEn4ZvGOalM/view?usp=sharing
Missing notes, see transcript: https://drive.google.com/file/d/12q9Bc_jr__iFqpVCbPflr3gF_r8W6jP0/view?usp=sharing
x Mathieu Poirier (Linaro) x Ed Mooring (Xilinx) x Bill Mills (Linaro) x Suman Anna (TI) x Hari Nagalla (TI)
- Hari Nagalla is new from TI on IPC
- Hari will take over most of work from Suman
- Suman has a new role but will still attend openamp meetings at least for a few months
- MP: reviewed patch from Martin S: Always on feature for amlogic media processor
- needs minor fixup in V2
- MP: reviewed Suman's recent patch, not much out of the ordinary
- MP: 7 line patch for mediatech T8195, will probably get to review this
- MP: Will be on vacation Aug 6 to 23
- Patches Qualcomm specific handled by Bjorn
- SA: what is the status of rpmsg char driver updates?
- MP & AP agree work is done
- makes it a module
- split control & data path
- keeps qualcomm mode untouched
- Bjorn has not picked up
- looks good for 5.15 (but next LTS _might_ be 5.14)
- EM: lots of linting fixes, a few real bug fixes
- Lots of ambitious stuff but author goes away after a bit so don't complete
- Manual Cache aware work got accepted, uses existing libmetal primitives
- Mathieu Poirier (Linaro),
- Arnaud Pouliquen (ST),
- Ed Mooring (Xilinx),
- Bill Mills (Linaro),
- Suman Anna (TI),
- Etsam Anjum (Mentor),
- Loic Pallardy (ST)
- MP: Working w/ Suman & Arnaud on their patches
- MP: Rob sent patch on bindings changing for spec stuff but should not
- SA: Have taken first look, confused a bit about what Rob is trying to do
Will follow up with Rob
- MP: Sidharth char driver fix working with Greg KH
- SA: 2020.10 U-boot will support early boot of R5
- AM64x, will be picked up for next
- lack of reserved memory binding in yaml casing dt check problems
-m is making checks more strick
- ipc-only mode for "all" SOCs
- AP: working with MP on rpmsg
- limit the ability of userspace to delete create/destroy rpmsg device
- AP/EM: PR for not inline (for rust compatible)
- globally disable static inline
- change to inline with external C ref
- Why not use a disabled inline option
- Need to optimize libmetal in general
- AP: ~900 Bytes for libmetal on ST platform
- Cached memory with manual coherency ops
- config switch
- libmetal?
- vrings
- config switch
- Stratos
- virtio in Zephyr using virtio layer in openamp
- Vincent G.
- Note: Zephyr openamp repo is an automated reformating
- No functional patches are carried
- No pull requests accepted there
- virtio in Zephyr using virtio layer in openamp
- Mathieu Poirier (Linaro)
- Arnaud Pouliquen (ST)
- Bill Mills (Linaro)
- Suman Anna (TI)
- Etsam Anjum (Mentor)
- BM is working on a doc / spec structure
- looking at Zephyr, Yocto Project, and EBBR
- Started with EBBR, moving build CI to github actions
- Will be a prototype for OpenAMP-docs
- AP: to send Bill a ODP version of
- EA: docs he wrote are now Incorporated into OpenAMP wiki
- AP: Libmetal PR #159 Add Symbols to Library
- User is interested in Rust integration and static inline is cause problems
- AP: request to manage the cache (request in Open-AMP but probably will be libmetal)
- AP: redirect syscall between two Linux systems??
- Proxy does semi-hosting
- Perhaps proxy use case is better handled by app srv model
- Perhaps make a better demo / example app
- Generic Service user Generic Service pub
- Proxy does semi-hosting
- MP: waiting for next rev from everyone
- AP next rev for rpmsg char enhancement
- SA IPC only mode on TI platforms
- AP: ioctrl can create lots of RPMSG device
- AP is this safe?
- could create lots of end points, some could bind to existing drivers
- Lets discuss when we have code to look at
- AP is this safe?
- Mathieu Poirier (Linaro)
- Arnaud Pouliquen (ST)
- Ed Mooring (Xilinx)
- Bill Mills (Linaro)
- Loic Pallardy (ST) first 30 min
- Suman Anna (TI) joined after 30 min
- Ed is retiring, willing and able to stay on as maintainer
- all agreed keeping Ed as maintainer would be good
- Bill to follow up with TSC if needed
- Github actions CI
- today is checkpatch & build test only
- some linux user space test on libmetal?
- not running linux to linux tests today but probably could
- no easy pass/fail results today
- Would be nice to test on QEMU
- Possible on Xilinx QEMU but is appears all or nothing
- Xilinx ZynqMP is not AMP usable upstream yet
- Using generic QEMU or generic upstream on Xilinx QEMU would break log jam
- Would use of Zephyr's user space mode help at all?
- Linux 5.13 now has virtio i2c
- AP hacked Resource table vitiodev to new ID and it was enumerated as I2C device
- would be nice to do this all the way for a demo
- multiple virtio devices would be good. 1 for rpmsg and 1 for I2c
- AP already have work that does multi virt queue from Resource table
- Doing in resource table makes i2c appear to user space but kernel can't bind devices to it (at boot anyway)
- AP has work to declare remoteproc Virtio devs in DT but review/acceptance is slow/hard
- ST has demo based on RPMSG I2C using 2 LCD panels
- Zephyr on M4 uses one LCD and presents I2C interface to Linux
- Linux can update the other LCD panel but can't see the first one at all even if it tries.
- From a couple of years ago, using ST demo code
- Suman K3 R5 incremetal patch for AM64 SOC
- seems stuck
- MP we should be able to push it through
- Mathieu Poirier (Linaro),
- Arnaud Pouliquen (ST),
- Ed Mooring (Xilinx),
- Bill Mills (Linaro),
- Suman Anna (TI),
- Resource table
- SA what is the current state?
- AP I thought we should first write up a spec for the current state
- AP has written in google doc, it is on the mail list and sent to Bill also
- BM Suggest we use Sphinx and ReSTuctured text like the kernel docs, EBBR, DT spec
- BM will look into this
- BM we really need an official spec in OpenAMP
- Some discussion of overlap between kernel & OpenAMP spec
- Need to define terminology clearly as it is confusing.
- Ex: from a virtio POV the M4 is the "host" and Linux is the "guest"
Missing notes, see transcript: https://drive.google.com/file/d/1XpFa6L6yeIz9CWn1lEm0PGE4nN0ghZgf/view?usp=sharing
Mathieu Poirier (Linaro), Arnaud Pouliquen (ST), Ed Mooring (Xilinx), Bill Mills (Linaro), Loic Pallardy (ST), Etsam Anjum (Mentor), Suman Anna (TI)
RPMsg buffer size negotiation based on Xiang Xiao's proposal (rpmsg.pdf)
- BM: discuss point without Xiang participation?
- MP: can start discussion without him, but will need to share discussion with him
- AP: sump-up of Xiang Xiao's proposal is to use the VirtIO configs space to give requirement about RPMsg buffer size
- RX and TX RPMsg buffer sizes can be different.
- Bjorn finds this reasonable, main concerns about proposal is that each size should provide its expected TX size.
- LP:
- ST proposed the same few years ago: [PATCH] virtio_rpmsg: make rpmsg channel configurable
- same concern from Bjorn + would like that RPMsg clients can request a tuned size for RPMsg.
- LP feedback: not suitable for small system with memory constraint (need to pre-allocate buffers)
- BM: size defined by client and pre-allocated buffer is 2 different codes.
- EM: Xiang's patch seems rejected as not complicated enough, seems because no negotiation
- BM: if each side negotiates own size, the resource table entry could be view as minimum size, host can tune it bigger
- LP:
- no dynamic allocation possible for some remote processor
- the remote processor know the max size it can support (RX +TX)
- if need bigger buffer should use some other mechanism ( indirect buffers)
- it is the rule of the resource table to give size information because fixed in coprocessor firmware
- BM: a feature bit has also been defined, for which purpose?
- AP: to enable the use of this configuration or not( so default size)
- LP: when ST proposed similar patch: Bjorn other concerns was how to access to this config:
- need to know the structure on both side
- need a version
- RPMsg has direct access to the resource table. strong link between RPMsg and resource table format.
- BM:
- Would be nice to have representation at VirtIO level to abstract from remoteproc
- initial size seems needed at RPMsg init.
- LP: Right, at least for announcement
- MP:
- mainly agree with that, else need a buffer rebuilt step
- like Bjorn suggestion keeping things in VirtIO, so without strong coupling
- We need a patchset to restart the conversation and move forward
- LP: agree, a new series rebased to relaunch discussion, including new comment such as version field in structure.
- EM:Could be done in the library and ignore kernel, with a versioning for compatibility?
- BM: still need a proposal working for the Linux Kernel
- EM:
- A versioning to recognize other firmware
- The OpenAMP library could take the lead over the kernel in term of feature
- AP:
- We need to ensure that we not fork in term of features
- main risk to integrate features without kernel Acknowledgement
- BM:yes, pretty fundamental for feature that impact both.
- AP:
- 2 ways to implement the VirtIO , legacy(split) and Packed virtqueue.
- seems that packed virtqueue imposes static allocation and so could have to take into account for buffer sizes
- Packed virqueues not supported in Linux kernel remoteproc
- BM:
- the problem of the VirtIO config is that there is a very poor implementation in the remoteproc framework
- in other VirtIO get config and set config are used in a very dynamic way for negotiation.
- AP: do we need this kind of implementation?
- BM: if implement such negotiation in virtio-rpmsg for the buffer size, that would be an easy statement for other system.
- BM: i assume fw_rsc_config struct is part of the resource table
- AP: part of an area at the end of vdev struct
- LP: it a free area, just need to know offset and size. free in R/W access
- in ST patch, update it before remote processor kick, relying on loacl and remote constraints
- allow also asymmetric buffer allocation for RX and TX
- BM: The story could be:
- With remoteproc, information comes from the resource table, Linux reads config and update it if needed before the kick
- Without remoteproc (using a VirtIO device) could be similar with a negotiation between host and guest, guest gets what has been negotiated
- LP: make sense
- BM: to document if we implement with remote proc only as a first step
- LP: Legacy support could be based on size of the config.
- BM: So what would need to be change for this patchset
- AP: This patchset is a good starting point for discussion on mailing list, could propose to Xiang to resent it.
- LP: would be nice to point over old patch to merge both, to have a common solution.
- AP: I need compare both.
- BM:
- looking at OpenAMP side, access to the config area is hardcoded
- what about number of buffers for RX and TX is it symmetric?
- SA:
- number of buffers pick-up from the Vrings
- concerning config space could be have different config structures ?
- what about extend vring structure ? so, with a new version of the vdev resource type, as for 64bits support
- BM: for the config space we can have feature bits and/or version field.
- LP: yes, these two fields would be useful
- BM: any issue for 64-bit? For 64 bits no upstream right?
- SA: TI has not issue with 64 bits.
- BM: can address could be in 64 bits
- AP: RFC from Clement Leger [Openamp-rp] [RFC,] Resource table and 64 bits addresses/fields
- BM: what do you think about a new vdev version with 64bit support.
- AP: means that we would support buffer size negotiation only with a new version of the resource table
- SA: resource table actually hardcoded for 32 bits. Need a new structure to properly support 64 bits
- BM: is this something that block you Suman?
- SA: no, address is in 32-bit space even if it is a 64-bit processor
- Discuss Xiaoxiang's issue/feature and their suggested solution
- Did have short meeting but Xiaoxiang could not make it so defered to March 11
No meeting on this day.
Mathieu Poirier (Linaro), Ed Mooring (Xilinx), Bill Mills (Linaro), Suman Anna (TI), Loic Pallardy (ST) Bruce Ashfield (Xilinx),
- Arnaud Pouliquen (ST),
- Guennadi Liakhovetski (Intel), virtio audio using rpmsg
- Bjorn Andersson (Linaro)
- [email protected]
- Get back to patch of Dec 18th: detach
- Arnaud is only one to have commented
- Pang has sent new versions for iMX8.M (two versions)
- Martin Bl* has patches for amlogic
- Arnaud 2 line patch taking long to review, return EBUSY when device is busy
- Ben: Xilinx R5 new version
- Alex and few others Qualcomm: Let Bjorn handle
- Suman PRU support: change make checkpatch happy
- targeting V2
- Adapting to mcu sync will be later in month
- Arnaud work to add ioctl to rpmsg char driver
- MP: will try to take a look
- LP: it would be good to get 2nd opinion from MP
- LP: we need more information from Bjorn
- EM: some bug fixes
- xiaoxiang: TCP 3 way handshake
- Servers know of the client connect and disconnect
- Special meeting to talk about this perhaps at a new time?
- Meeting on 1/28 will be canceled
- Next normal meeting with be Feb 11
- Look at possibility to have a special meeting on protocol enhancement
- Look for a time as early in Ed's day as possible so we are not too late for China