Continuous program optimization: Design and evaluation

This paper presents a system in which the already executing user code is co ntinually and automatically reoptimized in the background. using dynamicall y collected execution profiles as a guide. Whenever a new code image has be en constructed in the background in this manner, it is hot-swapped in place of the previously executing one. Control is then transferred to the new co de and construction of yet another code image is initiated in the backgroun d. Two new runtime optimization techniques have been implemented in the con text of this system: object layout adaptation and dynamic trace scheduling, The former technique constantly improves the storage layout of dynamically allocated data structures to improve data cache locality. The latter incre ases the instruction-lever parallelism by continually adapting the instruct ion schedule to predominantly executed program paths. The empirical results presented in this paper make a case in favor of continuous optimization, b ut also indicate some of the pitfalls and current shortcomings of continuou s optimization. If not applied judiciously, the costs of dynamic optimizati ons outweigh their benefit in many situations so that no break-even point i s ever reached. In favorable circumstances, however, speed-ups of over 96 p ercent have been observed. It appears as if the main beneficiaries of conti nuous optimization are shared libraries in specific application domains whi ch, at different times, can be optimized in the context of the currently do minant client application.