RQs - pooyanjamshidi/ase17 GitHub Wiki

RQ1: Does the performance behavior stay consistent across environments? Yes -- Performance models are frequently linearly related, primarily for hardware and simple workload changes. Moreover, we also identified transferable knowledge for more severe changes. This observation strongly indicates that although the overall behavior might not be consistent across the environment, individual influences of configuration options and interactions stay consistent. This opens the potential for nonlinear transformation functions and a focused sampling of measurements in the target environment

RQ2: Is the influence of features on performance consistent across environments? Yes -- for all environmental changes, the significance and importance of configuration options stay consistent across environments. Again, this is good news for transfer learning as the key elements in a model will not change, but the coefficients might need to be relearned.

RQ3: Are the interactions among features preserved across environments? Yes -- for all environmental changes, in which configuration options are interacting, the interactions stay consistent across environments. In other words, the prediction models containing interactions trained on the studied software systems have a very similar structure when built independently in different environments. Even for systems with large environment changes for which we could not find any relatedness with the previous metrics, here, we found a high similarity in the occurrence of interactions among configuration options.

RQ4: Are the configurations that are invalid in the source environment with respect to non-functional constraints also invalid in the target environment? Yes -- usually a considerable part of configuration space is invalid and they are largely invalid across environments. Therefore, there is a potential to avoid a considerable part of the configuration space for configuration tasks.