Software caching - adapteva/epiphany-sdk GitHub Wiki

The software caching feature is an way of allowing bigger programs to run efficiently on the Epiphany co-processor. As of the 2014.08 release, this feature is built into the GCC toolchain, but turned off by default.

Current way of optimizing for space on an Epiphany core.

The short answer to this is hacking a linkerscript. Although it can seem scary, it is a good option. The linkerscript is there to tell the tools "Where things go". So we can hand tune the location of every object file in our program.

Here is a linkerscript snippet, taken from the official distribution:

GNU_C_BUILTIN_LIB_RO . :
{
	*mulsi3.o(.text  .rodata)   *modsi3.o(.text  .rodata)
	*divsi3.o(.text  .rodata)   *udivsi3.o(.text .rodata)
	*umodsi3.o(.text .rodata)  _*.o(.text  .rodata)
} > EXTERNAL_DRAM_0

We can see that the whole of the newlib C library is stored in external memory. The user can optimize this by moving the objects that he/she needs to internal memory:

INTERNAL_GNU_C_BUILTIN_LIB_RO . :
{
	*modsi3.o(.text  .rodata)
} > INTERNAL_RAM

GNU_C_BUILTIN_LIB_RO . :
{
	*mulsi3.o(.text  .rodata)
	*divsi3.o(.text  .rodata)   *udivsi3.o(.text .rodata)
	*umodsi3.o(.text .rodata)  _*.o(.text  .rodata)
} > EXTERNAL_DRAM_0

What if this could be done on demand?

This is what the software caching is doing. It allows one to keep a whole library in external memory and still get decent performance. In a nutshell, when your code hits a function call, it asks a cache manager where this function is. If the function is in external memory, the cache manager will pull it into the internal memory before executing this function.

There will of course be an overhead, copying code from external to internal memory comes at a cost. But, the next time your code hits the same function. It will jump directly to it, giving the best performance.

You may ask what happens when the core actually runs out of code space? As you would expect, the cache manager will unload a previously loaded function to make room for the new one.

Bear in mind that this feature is new in release 2014.08. However we hope it can make a big difference and welcome bug reports for it to be as stable as possible.

About the new toolchain

The 2014.08 release incorporates this functionality. If you wish to examine the code, you will find it on the following branches of the standard components:

  • gcc: epiphany-gcc-4.8-software-cache
  • binutils: epiphany-binutils-2.23-software-cache
  • newlib: checkout epiphany-newlib-1.20-software-cache
  • cgen: epiphany-cgen-1.1-software-cache

Using software caching

The software cache is straightforward to use. It requires two changes to your standard procedure for building programs:

  1. Add -fsoftware-cache to the gcc command line
  2. Use a modified linker script which sets aside space for the cache manager and its data

Simple examples

To start, we have put together two examples. You can clone these from GitHub

git clone https://github.com/embecosm/parallella-software-caching-example.git

You will find two directories, math_example and basic_math. Both include a Makefile to build the script and a modified linker script to allocate space for the cache manager. In each case just type make and the program will be built.

match_example just calls the fmod funtion twice, and times how long it takes each time. The second time is notably quicker, because the function is now in memory.

basic_math is similar, but exercises a wide range of functions to give a fuller assesment.

Power use of software caching

You can use software caching on all your programs. Use the linker script in the examples, modified if desired for your specific application. We've tried it successfully on the standard Epiphany examples.

However this only applies sofware caching to application code (which helps a lot). It is possible to build the standard libraries with software caching. To do this, you'll need to rebuild the toolchain, specifying -fsoftware-cache to be added to the target CFLAGS. To do this use the build-toolchain.sh script in the SDK directory, adding the --target-cflags option:

./build-toolchain.sh --target-cflags "-g -O2 -fsoftware-cache"

(of if you prefer use -Os for a smaller library)

Bugs and ehancements

Being new, we expect there will be bugs we have missed. We also hope this proves useful and you may have suggestions for improvements. These should all be recorded in the SDK issue tracker