Abstract
Studies on the evolution of aposematic coloration (prey coloration advertising for unpalatability) have mainly focused on predator psychology in simplified single‐prey species systems. We chose, instead, to model population dynamics on the community level. We studied the invasion by an aposematic phenotype in the presence and absence of another prey species. The single‐prey and two‐prey models differed in two major ways. First, with two prey species the invasion was possible only with a weak aposematic signal, whereas with a single prey species there was no such an upper limit for signal strength. Second, with a single prey species, increase of the aposematic phenotype always resulted in rapid extinction of the predator. Resource value and growth rate of the alternative prey species affected the invasion. These results suggest that community structure is an important determinant of the conditions for invasion of aposematism, and may have contributed to its initial evolution.