Mean field games are limit models for symmetric N-player games with interaction of mean field type as N . .. The limit relation is often understood in the sense that a solution of a mean field game allows to construct approximate Nash equilibria for the corresponding N-player games. The opposite direction is of interest, too: When do sequences of Nash equilibria converge to solutions of an associated mean field game? In this direction, rigorous results are mostly available for stationary problems with ergodic costs. Here, we identify limit points of sequences of certain approximate Nash equilibria as solutions to mean field games for problems with Itô-type dynamics and costs over a finite time horizon. Limits are studied through weak convergence of associated normalized occupation measures and identified using a probabilistic notion of solution for mean field games.