Critical Sections on the ESP8266 - mhightower83/Arduino-ESP8266-misc GitHub Wiki
DRAFT - Still editing and Organizing thoughts
TODO: Add info about memory fences
Critical Section Issues on the ESP8266
I think of a critical section as a section of code that needs to be protected from some asynchronous event that would cause the code to be reentered or some element that the critical section code is already working with is accessed or modified, where such access would result in incorrect behavior. Common examples are updating a link list, maintaining a shared counter between foreground and IRQ handler, etc.
I see three cases that need to be handled:
- quickly set a value atomically
- run a block of code atomically
- bit-banging applications
The ESP8266 does not have any atomic instructions. Thus interrupts must be turned off then back on to protect an operation. The ESP8266 has multiple interrupt levels. While there are levels 0 to 15 for the Xtensa architecture, the ESP8266 supposedly only uses up to 2. INTLEVEL 1 hardware interrupts, INTLEVEL 2 debug, INTLEVEL 3 NMI?. Thus, a level of 3 would turn off all normal interrupts. However, people that do bit-banging code, have reported the need to use level 15 to resolve some latency issues. I assume the NMI interrupt is what was causing problem; however, if that were the case, I would have thought they would have said so. Based on Makuna's comments, I think INTLEVEL of 15 should be limited to bit-banging applications. Do INTLEVEL 3 for the general block of code protection. For the quick setting of a variable, maybe do INTLEVEL 15 is the safest choice. My thinking at the moment is we have no atomic instructions and INTLEVEL 15 works for bit-banging. This is the closest thing we have to being able to perform an atomic operation. It would give the most protection for doing quick exchange, increment, or flag set.
One additional note, because of the need to let the WiFi code run so as not to drop packets, interrupts off should be limited to no more than 10us.
ets_intr_lock/unlock - Is it Safe?
These functions are not documented beyond their prototype in ets_sys.h. They are used in an example in the NONOS SDK without explanation. By their name you would think they were the correct choice for handling a critical section; however, they do not do what you think they might. The main problem here is that ets_intr_unlock does not restore the previous state of INTLEVEL. It does a rsil a2, 0 setting INTLEVEL to 0, effectively interrupts enable. This also makes it useless for the nested lock case.
This is the ROM disassembly code for ets_intr_lock and ets_intr_unlock:
40000f74 <ets_intr_lock>:
40000f74: 206300 rsil a2, 3
40000f77: 31feff l32r a3, 40000f70 ; ( 3fffdcc0 )
40000f7a: 2903 s32i.n a2, a3, 0 ; save prior level
40000f7c: 0df0 ret.n
40000f7e: 0000
40000f80 <ets_intr_unlock>:
40000f80: 206000 rsil a2, 0
40000f83: 0df0 ret.n
Note, the saving of prior level and then it is never used. After some time a hypothetical use occurred to me: A function that is called from the foreground or an ISR (interrupt service routine) could use that value to make a decision on whether to do a memcpy that could take over 10us. When called from the foreground, interrupts could be safely turned back on for the transfer. When called from an ISR, a flag could be set to indicate that processing had been deferred. Possibilities like this and the lack of documentation on the proper use causes me some concern that by using it, some conflicted behavior could be created.
Currently, in the Arduino ESP8266 Core, I see ets_intr_lock/unlock used to handle critical sections in the umm_malloc and the lwIP stack. In the case of umm_malloc when malloc, free, calloc, or realloc are called with interrupts off, interrupts are already back on when the function returns. That is probably not a good thing when called from an ISR. There are also macros that define os_intr_lock/unlock to ets_intr_lock/unlock.
IMHO XTOS_SET_MIN_INTLEVEL/XTOS_RESTORE_INTLEVEL is a superior choice. It keeps or uses the higher INTLEVEL on entry and allows for restoring the original INTLEVEL on exit.
XTOS_SET_MIN_INTLEVEL and XTOS_RESTORE_INTLEVEL
I like the macros XTOS_SET_MIN_INTLEVEL and XTOS_RESTORE_INTLEVEL for critical code blocks. The following is an adaptation of XTOS_SET_MIN_INTLEVEL and XTOS_RESTORE_INTLEVEL from xtruntime.h. I don't think this clipping qualifies as a "... substantial portions of the Software", but just in case this links to Xtensa's copyright/license.
/*
* A derivative work from xtensa's xruntime.h
*
* unsigned XTOS_MIN_INTLEVEL(int intlevel);
* This macro conditionally sets a new interrupt level,
* when 'intlevel' is greater then the interrupt level
* present at the start of the call.
* The 'intlevel' parameter must be a constant.
* This macro returns a 32-bit value that must be passed to
* XTOS_RESTORE_INTLEVEL(unsigned restoreval) to restore
* the previous interrupt level.
*/
#ifndef __STRINGIFY
#define __STRINGIFY(a) #a
#endif
#define XTOS_SET_MIN_INTLEVEL(intlevel) \
({ \
unsigned __tmp, __tmp2, __tmp3; \
__asm__ __volatile__( \
"rsr.ps %0\n" \
"movi %2, " __STRINGIFY(intlevel) "\n" \
"extui %1, %0, 0, 4\n" \
"blt %2, %1, 1f\n" \
"rsil %0, " __STRINGIFY(intlevel) "\n" \
"1:\n" \
: "=a" (__tmp), "=&a" (__tmp2), "=&a" (__tmp3) : : "memory" ); \
__tmp;})
#define XTOS_RESTORE_INTLEVEL(restoreval) \
do { \
unsigned __tmp = (restoreval); \
__asm__ __volatile__( \
"wsr.ps %0\n" \
"rsync\n" \
: : "a" (__tmp) : "memory" ); \
} while(false)
Conclusion?
While I like XTOS_SET_MIN_INTLEVEL and XTOS_RESTORE_INTLEVEL, do we need the extra overhead? Is it necessary?
For atomic operations that do not involve critical waveform timing (bit-banging), is (isn't?) simply setting INTLEVEL to 3 and restoring the previous INTLEVEL, when done, enough?
For atomic operations on one or two variables use INTLEVEL 15 and get out. If you are doing PWM using NMI, you might think about using INTLEVEL 3 instead and be sure the NMI code does nothing that could corrupt your atomic variables.
Maybe it would be best to use a macro to define INTLEVEL:
#ifndef DEFAULT_CRITICAL_SECTION_INTLEVEL
#define DEFAULT_CRITICAL_SECTION_INTLEVEL 3
#endif