A cookbook: patching Premake4 behavior - premake/premake-4.x GitHub Wiki

Fixing (patching) existing Premake4 behavior

Sometimes, actually most of the time, when you have a premake4 binary, you won't be interested in re-building it every time you come up with a cool new feature. Alternatively Premake4 may be misbehaving and you simply need to fix its behavior.

Fortunately, since Premake4 uses Lua, and functions are first class language elements in Lua, the task of patching Premake4 behavior isn't quite as daunting as it may seem.

Prerequisites

If you want to understand this article, you need to make sure to have read and have a firm grasp on A Bit Of Meta. If you don't, go there and read up on it. Additionally you should have a firm grasp on Lua as a programming language if you want to make use of these tips and tricks here.

Additionally, try to familiarize yourself with the Lua code of Premake4, by having a look at the src folder. If you also want to dive into the Lua functions Premake4 defines at the C level, you also need to look into src/host, otherwise this subfolder can be ignored.

Generic approach: hook Premake4 functions

Goal: you found a small non-local Premake4 function whose behavior you'd like to alter.

Solution: the straightforward method here is to re-assign that non-local function to your own. For all the behavior you want to retain, you can then call the original function - if you made sure to save that in a local variable prior to re-assigning the Premake4 function - and alter the return value as needed.

Example: naming solutions and projects

Goal: You would like to name your projects <prjname>.vs8.vcproj (vs8 == VS 2005) and solutions <slnname>.vs8.sln instead of risking to overwrite the existing projects and solutions when generating those for vs2008 subsequently. I.e. sometimes it's good to have the version of Visual Studio as part of the generated file names.

Solution: We hook the premake.project.getbasename() function and use a Lua table to map action names to the name of the suffix to use.

local orig_getbasename = premake.project.getbasename
premake.project.getbasename = function(prjname, pattern)
    -- The below is used to insert the .vs(8|9|10|11|12|14|15) into the file names for projects and solutions
    if _ACTION then
        name_map = {vs2002 = "vs7", vs2003 = "vs7_1", vs2005 = "vs8", vs2008 = "vs9", vs2010 = "vs10", vs2012 = "vs11", vs2013 = "vs12", vs2015 = "vs14", vs2017 = "vs15"}
        if name_map[_ACTION] then
            pattern = pattern:gsub("%%%%", "%%%%." .. name_map[_ACTION])
        else
            pattern = pattern:gsub("%%%%", "%%%%." .. _ACTION)
        end
    end
    return orig_getbasename(prjname, pattern)
end

NB: If you wanted to use the action name straight, you would make use of _ACTION instead of name_map[_ACTION] above.

You may also be interested in the recipe Overriding the basename of my project does not work below

Generic approach: replace arbitrary output to Premake4-generated files

Goal: After scouring the source for some function you can hook, you find that all the sensible choices have been declared a local function by Jason, the Premake4 author. However, up the call path (i.e. when finding the caller of your local function and the caller of that and so on) you find some non-local function. What can you possibly do?

Solution: Since the non-local function doesn't return a value you could alter and the above method of hooking only works for those cases, you'd have to copy large parts of the Premake4 implementation from those .lua files and reimplement the whole thing. Not optimal, if you prefer to keep it DRY. Thankfully Jason seems to have foreseen this limitation and introduced the io.capture() and io.endcapture() functions of which the oft used _p() function (aka io.printf()) is aware of. Eventually, when you follow the call path of said local function you will find a non-local one that you can hook. However, if you just wanted to change a tiny bit of output from one particular callee, the standard approach would be to capture the output, alter it and then output it just as the original (now hooked) function would.

This allows you to do the following.

  1. Call io.capture() to start capturing output via _p()
  2. Call whatever vanilla Premake4 functions or generate your own output by means of _p() and io.printf() (in fact the same function)
  3. Call io.endcapture() and retrieve its return value to get the contents that were generated in the previous step
  4. Manipulate the captured contents
  5. Output the altered contents using io.write()

Or in code:

io.capture() -- this sets io.captured = ''
-- ... some output via _p() either by Premake4 function or your own code
local captured = io.endcapture() -- returns captured output and resets io.captured to nil
-- ... manipulate captured contents
io.write(captured)

However, there is a caveat. If you, like I, hook a lot it may lead to nested calls to io.capture(). Unfortunately io.capture() sets the global variable io.captured to an empty string '' and io.endcapture() resets it to nil. So if you hooked functions premake.foo() and premake.bar() (not actual names!) and foo() called bar() or vice versa, but both your hooked implementations made use of io.capture() and io.endcapture(), you'd end up with io.captured being nil in the outer function. Duh! So you break the capture of the outer function this way.

Luckily there is a fix. You can store the contents of io.captured in a local variable prior to calling io.capture() and then decide after io.endcapture() whether to restore that original value and append the newly captured one or whether to output it via io.write().

local old_captured = io.captured -- save io.captured state
io.capture() -- this sets io.captured = ''
-- ... some output via _p() either by Premake4 function or your own code
local captured = io.endcapture()
-- ... manipulate captured contents
if old_captured ~= nil then
    io.captured = old_captured .. captured -- restore outer captured state, if any
else
    io.write(captured)
end

Not too hard, is it?

Putting the lesson to practical use

Problem description: Premake4 has an issue with putting the appropriate entry point in for Visual Studio ConsoleApp projects which make use of Unicode.

Goal: We should fix that by capturing the output at strategical points and alter it. The code needs to be implemented once for Visual Studio versions before 2010 and those newer.

Solution: Patch functions premake.vstudio.vc2010.link and premake.vstudio.vc200x.VCLinkerTool) to fix the erroneous default behavior:

local orig_vc2010_link = premake.vstudio.vc2010.link
premake.vstudio.vc2010.link = function(cfg)
    if cfg.flags and cfg.flags.Unicode then
        local old_captured = io.captured -- save io.captured state
        io.capture() -- this sets io.captured = ''
        orig_vc2010_link(cfg)
        local captured = io.endcapture()
        assert(captured ~= nil)
        captured = captured:gsub("(<EntryPointSymbol>)(mainCRTStartup)", "%1w%2")
        if old_captured ~= nil then
            io.captured = old_captured .. captured -- restore outer captured state, if any
        else
            io.write(captured)
        end
    else
        orig_vc2010_link(cfg)
    end
end

local orig_vc200x_VCLinkerTool = premake.vstudio.vc200x.VCLinkerTool
premake.vstudio.vc200x.VCLinkerTool = function(cfg)
    if cfg.flags and cfg.flags.Unicode then
        local old_captured = io.captured -- save io.captured state
        io.capture() -- this sets io.captured = ''
        orig_vc200x_VCLinkerTool(cfg)
        local captured = io.endcapture()
        assert(captured ~= nil)
        captured = captured:gsub('(EntryPointSymbol=")(mainCRTStartup)', "%1w%2")
        if old_captured ~= nil then
            io.captured = old_captured .. captured -- restore outer captured state, if any
        else
            io.write(captured)
        end
    else
        orig_vc200x_VCLinkerTool(cfg)
    end
end

This illustrates quite nicely how to use the above learned lesson and put it to practical use. In this case to fix erroneous Premake4 behavior.

By the way, on the line:

captured = captured:gsub('(EntryPointSymbol=")(mainCRTStartup)', "%1w%2")

we're using the gsub() method of the captured variable. That method is documented here. To learn about the patterns used (essentially Lua's very own regular expression flavor), you should check out the patterns tutorial.

NB: as an alternative one could simply remove the EntryPointSymbol attribute (VS 200x) or element (VS 201x) instead of replacing its value. The effect is much the same since Visual Studio knows which default entry point name to use depending on the project type.

Rename Makefile to GNUmakefile

Goal: you want to use the name GNUmakefile for generated make files, because Makefile is picked up by all kinds of make tool flavors, but GNUmakefile only by GNU make.

Solution: scouring the Lua source of Premake4 for Makefile (the default name generated by Premake4 for gmake), we find the function _MAKE.getmakefilename in src/actions/make/_make.lua which we can override. In order to override it, we create a scope (do ... end) inside of which we assign only to local variables. We store the original function inside a local variable and re-assign the global function. Our own function makes use of the original one and only replaces the return value whenever the return value is "Makefile".

do
    local orig_getmakefilename = _MAKE.getmakefilename
    _MAKE.getmakefilename = function(this, searchprjs)
      local x = orig_getmakefilename(this, searchprjs)
      if x == "Makefile" then
        return "GNUmakefile"
      end
      return x
    end
end

Overriding the basename of my project does not work

Goal: The function premake.project.getbasename was actually introduced by a patch I had proposed. This means there are still old Premake4 versions out there not having this functionality.

Solution: In order to fix these, you can include the following in premake4.lua.

if not premake.project.getbasename then
    print "Magic happens for old premake4 versions without premake.project.getbasename() ..."
    -- override the function to establish the behavior we'd get after patching Premake to have premake.project.getbasename
    premake.project.getbasename = function(prjname, pattern)
        return pattern:gsub("%%%%", prjname)
    end
    -- obviously we also need to overwrite the following to generate functioning VS solution files
    premake.vstudio.projectfile = function(prj)
        local pattern
        if prj.language == "C#" then
            pattern = "%%.csproj"
        else
            pattern = iif(_ACTION > "vs2008", "%%.vcxproj", "%%.vcproj")
        end

        local fname = premake.project.getbasename(prj.name, pattern)
        fname = path.join(prj.location, fname)
        return fname
    end
    -- we simply overwrite the original function on older Premake versions
    premake.project.getfilename = function(prj, pattern)
        local fname = premake.project.getbasename(prj.name, pattern)
        fname = path.join(prj.location, fname)
        return path.getrelative(os.getcwd(), fname)
    end
end

This, in fact, copies and alters the behavior of the old Premake4 versions which don't have premake.project.getbasename() and causes some other Premake4 functions to make use of that newly established function.

Generate projects using the XP toolset in modern VS versions

Goal: You want to tell Premake4 to generate a project on Visual Studio 2012 or newer, which targets Windows XP.

Solution: We need to replace the <PlatformToolset> element and append _xp to the existing value in it. This can be achieved by adding a new option --xp which triggers our behavior and patching the premake.vstudio.vc2010.configurationPropertyGroup() function to append the _xp as mentioned before.

do
    newoption { trigger = "xp", description = "Use XP-compatible toolchain for newer Visual Studio versions." }

    premake.vstudio.vc2010.configurationPropertyGroup = function(cfg, cfginfo)
        local old_captured = io.captured -- save io.captured state
        io.capture() -- this sets io.captured = ''
        orig_vc2010_configurationPropertyGroup(cfg, cfginfo)
        local captured = io.endcapture()
        assert(captured ~= nil)
        local toolsets = { vs2012 = "v110", vs2013 = "v120", vs2015 = "v140", vs2017 = "v141" }
        local toolset = toolsets[_ACTION]
        if toolset then
            if _OPTIONS["xp"] then
                toolset = toolset .. "_xp"
                captured = captured:gsub("(</PlatformToolset>)", "_xp%1")
            end
        end
        if old_captured ~= nil then
            io.captured = old_captured .. captured -- restore outer captured state, if any
        else
            io.write(captured)
        end
    end
end

Ability to use old-style (i.e. embedded) debug information for static libraries

Goal: Premake4 does not allow you to use OldStyle for the debug information. However, this type of debug information is incredibly useful for static libraries, as the debug information gets embedded into the object files which are stored in the static library. So our goal is to override the existing setting for the debug information format and set it to OldStyle for StaticLib projects, provided the Symbols flag is set.

Solution: we override premake.vs2010_vcxproj and premake.vstudio.vc200x.Symbols to make it work for VS201x and VS200x respectively. Observe:

do
    -- Premake4 doesn't allow to set old-style debug information, which is
    -- useful for our static libraries, so let's patch that (VS201x)
    local orig_premake_vs2010_vcxproj = premake.vs2010_vcxproj
    premake.vs2010_vcxproj = function(prj)
        if (prj.kind == "StaticLib") and prj.flags and prj.flags.Symbols then
            local old_captured = io.captured -- save io.captured state
            io.capture() -- this sets io.captured = ''
            orig_premake_vs2010_vcxproj(prj)
            local captured = io.endcapture()
            assert(captured ~= nil)
            -- Old style: OldStyle, PDB: ProgramDatabase, PDB + Edit & Continue: EditAndContinue
            captured = captured:gsub("(<DebugInformationFormat>)(%a+)(</DebugInformationFormat>)", "%1OldStyle%3")
            if old_captured ~= nil then
                io.captured = old_captured .. captured -- restore outer captured state, if any
            else
                io.write(captured)
            end
        else
            orig_premake_vs2010_vcxproj(prj)
        end
    end
    -- ... same functional patch for VS 200x
    local orig_vc200x_Symbols = premake.vstudio.vc200x.Symbols
    premake.vstudio.vc200x.Symbols = function(cfg)
        -- Old style: 1, PDB: 3, PDB + Edit & Continue: 4
        local retval = orig_vc200x_Symbols(cfg)
        if (retval ~= 0) and (cfg.kind == "StaticLib") and cfg.flags and cfg.flags.Symbols then
            return 1
        end
        return retval
    end
end

As you can see this also makes use of the trick explained above regarding capturing, while retaining any possible outer captures.

Fix issue #9 (error C1052 due to name clash of compiler and linker PDB)

Problem: When compiling a project again after changing a source file, without doing a full rebuild, the compiler tells us about error C1052. The cause for this is that Premake4 names the compiler's PDB file by the name the linker's PDB file defaults to. This effectively causes the linker to overwrite the PDB of the compiler and upon the next compilation the compiler chokes, because the linker's PDB file is incomatible.

Solution: We can easily fix the problem by patching premake.vs2010_vcxproj. We capture the output of the original function (making sure to save and restore possible nested captures) and then remove the <ProgramDataBaseFileName /> element from the output, effectively forcing it to use the default name again.

do
    -- Premake4 sets the PDB file name for the compiler's PDB to the default
    -- value used by the linker's PDB. This causes error C1052 on VS2017. Fix it.
    local orig_premake_vs2010_vcxproj = premake.vs2010_vcxproj
    premake.vs2010_vcxproj = function(prj)
        local old_captured = io.captured -- save io.captured state
        io.capture() -- this sets io.captured = ''
        orig_premake_vs2010_vcxproj(prj)
        local captured = io.endcapture()
        assert(captured ~= nil)
        captured = captured:gsub("%s+<ProgramDataBaseFileName>[^<]+</ProgramDataBaseFileName>", "")
        if old_captured ~= nil then
            io.captured = old_captured .. captured -- restore outer captured state, if any
        else
            io.write(captured)
        end
    end
    -- ... same as above but for VS200x this time
    local function wrap_remove_pdb_attribute(origfunc)
        local fct = function(cfg)
            local old_captured = io.captured -- save io.captured state
            io.capture() -- this sets io.captured = ''
            origfunc(cfg)
            local captured = io.endcapture()
            assert(captured ~= nil)
            captured = captured:gsub('%s+ProgramDataBaseFileName=\"[^"]+\"', "")
            if old_captured ~= nil then
                io.captured = old_captured .. captured -- restore outer captured state, if any
            else
                io.write(captured)
            end
        end
        return fct
    end
   -- wrap the two relevant functions to filter out the ProgramDataBaseFileName attribute
    premake.vstudio.vc200x.VCLinkerTool = wrap_remove_pdb_attribute(premake.vstudio.vc200x.VCLinkerTool)
    premake.vstudio.vc200x.toolmap.VCLinkerTool = premake.vstudio.vc200x.VCLinkerTool -- this is important as well
    premake.vstudio.vc200x.VCCLCompilerTool = wrap_remove_pdb_attribute(premake.vstudio.vc200x.VCCLCompilerTool)
    premake.vstudio.vc200x.toolmap.VCCLCompilerTool = premake.vstudio.vc200x.VCCLCompilerTool -- this is important as well
end

Another generic approach to cheat in case of local functions and variables

There are plenty of places where Premake4 by default uses local functions and variables. This makes our life harder when patching its behavior. While it's easy to patch any function which has a global name of some kind, patching local functions is somewhat cumbersome.

The problem I ran into was issue #10. Go read it, I'll wait.

Now, the problem I ran into here was that the local function resource_compile(cfg) in vs2010_vcxproj.lua was local and all of its callees were as well. However, I knew that it was using a single global function twice: _p() (an alias for io.printf() in Premake4).

Now, that means we could be patching _p() _globally and look for the start trigger ('<ResourceCompile>') and the end trigger ('</ResourceCompile>') to alter default behavior. Alas, by overriding _p() globally we're putting quite a brake on the overall performance of Premake4. And using the io.capture() method described elsewhere on this page will incur a similar performance overhead.

So it'd be nice if we could patch this function only locally. This way we don't receive the performance penalty while retaining all the flexibility.

However, in order to do anything half-way locally we need to find a global function which we can use as inroad into the resource_compile() function. So looking for callers of resource_compile() we find local function item_definitions(prj) ... unfortunately also local. Fortunately looking for the caller of this caller already yields a global function. We're in luck: function premake.vs2010_vcxproj(prj).

So we're going to

  • override premake.vs2010_vcxproj() to hook into _p() within the scope of it
  • in our patched _p() we'll look out for the start trigger ('<ResourceCompile>') and the end trigger ('</ResourceCompile>')
  • we need to either call the local functions preprocessor() and resinclude_dirs() or simply take their implementation into our patch

On to glory!

do
    -- Borrowed from setLocal() at https://stackoverflow.com/a/22752379
    local function getLocal(stkidx, name)
        local index = 1
        while true do
            local var_name, var_value = debug.getlocal(stkidx, index)
            if not var_name then break end
            if var_name == name then 
                return var_value
            end
            index = index + 1
        end
    end
    local orig_premake_vs2010_vcxproj = premake.vs2010_vcxproj
    premake.vs2010_vcxproj = function(prj)
        -- save original _p function under this new name
        local orig_p = _G._p
        -- provide an upvalue used by our replacement _p below
        local besilent = false
        -- we patch the global _p function here
        _G._p = function(indent, msg, ...)
            -- look for indent values of 2, this should already lower performance impact
            if indent == 2 and msg ~= nil then
                -- ... with msg value of <ResourceCompile>
                if msg == "<ResourceCompile>" then
                    local cfg = getLocal(3, "e") -- with LuaSrcDiet the cfg variable is named e
                    if cfg == nil then
                        cfg = getLocal(3, "cfg") -- without LuaSrcDiet it'd be cfg of course
                    end
                    assert(type(cfg) == "table" and cfg["resdefines"] ~= nil)
                    -- spit the original line out (i.e. <ResourceCompile>)
                    orig_p(indent, msg, ...)
                    -- bump indentation
                    local indent = indent + 1
                    -- this is our inlined preprocessor() function from vs2010_vcxproj.lua
                    -- the key difference is that _we_ actually make use of resdefines here
                    if #cfg.defines > 0 or #cfg.resdefines then
                        local defines = table.join(cfg.defines, cfg.resdefines)
                        orig_p(indent,'<PreprocessorDefinitions>%s;%%(PreprocessorDefinitions)</PreprocessorDefinitions>'
                            ,premake.esc(table.concat(premake.esc(defines), ";")))
                    else
                        -- note how we're using orig_p in all cases where normally _p would be used
                        orig_p(indent,'<PreprocessorDefinitions></PreprocessorDefinitions>')
                    end
                    -- this is our inlined resinclude_dirs() function from vs2010_vcxproj.lua
                    if #cfg.includedirs > 0 or #cfg.resincludedirs > 0 then
                        local dirs = table.join(cfg.includedirs, cfg.resincludedirs)
                        orig_p(indent,'<AdditionalIncludeDirectories>%s;%%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>'
                                ,premake.esc(path.translate(table.concat(dirs, ";"), '\\')))
                    end
                    -- the following line is important, it tells this whole function
                    -- to swallow any and all input it receives, until we decide
                    -- otherwise
                    besilent = true
                end
                -- ... or msg value of <ResourceCompile>
                if msg == "</ResourceCompile>" then
                    -- don't swallow input anymore ...
                    besilent = false
                    -- fall through
                end
            end
            if not besilent then -- should we be silent?
                orig_p(indent, msg, ...)
            end
        end
        -- do something else here, if needed
        orig_premake_vs2010_vcxproj(prj)
        _G._p = orig_p
    end
end

While this is quite a mouthful, I added inline comments to explain what's going on. However, if you're interested in the 30,000 ft view, consider this original implementation of resource_compile():

do
    local function resource_compile(cfg)
        _p(2,'<ResourceCompile>')
            preprocessor(3,cfg)
            resinclude_dirs(3,cfg)
        _p(2,'</ResourceCompile>')
    end
end

What we're doing with the above patch is this:

  • we sense the call of _p(2,'<ResourceCompile>') and output this line
  • we then execute the
    • inlined preprocessor() and
    • inlined resinclude_dirs functions
  • at this point we silence _p(); since all output should go via our own patched function, that's possible
  • and last but not least when we sense the call of _p(2,'</ResourceCompile>') we return the value of our besilent local variable to false, causing any output to pass ... until we see <ResourceCompile> and start the above logic again
Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️