GROUPS WITHOUT FAITHFUL TRANSITIVE PERMUTATION REPRESENTATIONS OF SMALL DEGREE

A subgroup H of a group G is core-free if H contains no non-trivial no rmal subgroup of G, or equivalently the transitive permutation represe ntation of G on the cosets of H is faithful. We study the obstacles to a group having large core-free subgroups. We call a subgroup D a ''de dekind'' subgroup of G if all subgroups of D are normal in G. Our main result is the following: If a finite group G has no core-free subgrou ps of order greater than k, then G has two dedekind subgroups D-1 and D-2 such that every subgroup in G of order greater than f(k) has non-t rivial intersection with either D-1 or D-2 (where f is a fixed functio n independent of G). Examples show that the dedekind subgroups need no t have index bounded by a function of k, and the result would not be t rue with one dedekind subgroup instead of two. We exhibit various rela ted properties of p-groups and infinite locally finite groups without large core-free subgroups, including the following: If G is a locally finite group with no infinite core-free subgroup, then every infinite subgroup of G contains a non-trivial cyclic normal subgroup of G. We a lso exhibit asymptotic bounds for some related problems, including the following: If a group G has a solvable subgroup of index n, then G ha s a solvable normal subgroup of index at most n(c) (for some absolute constant c). If G is a transitive permutation group of degree n with c yclic point-stabilizer subgroup, then \G\ < n(7). (C) 1997 Academic Pr ess.