Previous work has demonstrated that for games defined by differential or di
fference equations with a continuum of strategies, there exists a G-functio
n, related to individual fitness, that must take on a maximum with respect
to a virtual variable v whenever v is one of the vectors in the coalition o
f vectors which make up the evolutionarily stable strategy (ESS). This resu
lt, called the ESS maximum principle, is quite useful in determining candid
ates for an ESS. This principle is reformulated here, so that it may be con
veniently applied to matrix games. In particular, we define a matrix game t
o be one in which fitness is expressed in terms of strategy frequencies and
a matrix of expected payoffs. It is shown that the G-function in the matri
x game setting must again take on a maximum value at all the strategies whi
ch make up the ESS coalition vector. The reformulated maximum principle is
applicable to both bilinear and nonlinear matrix games. One advantage in em
ploying this principle to solve the traditional bilinear matrix game is tha
t the same G-function is used to find both pure and mixed strategy solution
s by simply specifying an appropriate strategy space. Furthermore we show h
ow the theory may be used to solve matrix games which are not in the usual
bilinear form. We examine in detail two nonlinear matrix games: the game be
tween relatives and the sex ratio game. In both of these games an ESS solut
ion is determined. These examples not only illustrate the usefulness of thi
s approach to finding solutions to an expanded class of matrix games, but a
ids in understanding the nature of the ESS as well. (C) 2000 Academic Press
.