1. Introduction
In ecological system, predation and competition are often assumed to be
the important factors that affect species coexistence [7, 8, 11,
25]. It is further investigated thoroughly in [1, 12, 13].
Gurevitch et al. [6] showed that predator can promote coexistence by
lowering the strength of competition. It is a common phenomenon that
predator can affect prey populations by direct killing. Recent field
studies show that the indirect effect of predator species on prey
species has a major impact than direct killing [2-5, 14]. Thus, it
is reasonable to incorporate the fear effect in the model focussed on
the role of predator regarding coexistence of competing species. This
type of mechanism can slow down the competition in respect of resource
competition. Thus avoidance behaviour developed by fear usually
stimulates coexistence provided prey partition resources, but not
predators, whereas it weaken coexistence if prey partition predators but
not the resources. Zanette et al. [29] carried out experiments on
song sparrows and observed 40% reduction in offspring production due to
fear from the predator. With this fact in mind, Wang et al. [27]
first developed the predator-prey model incorporating the cost of fear
into prey reproduction. They found that the cost of fear has no impact
in dynamical behavior when predation follows Holling type I response
function whereas it can stabilize the system by discarding periodic
orbits considering Holling type II response function. Since then several
studies are found in predator-prey models by introducing fear component
in prey reproduction. Wang and Zou [28] investigated a predator-prey
model with the cost of fear and adaptive avoidance of predators and
established that both strong adaption of adult prey and the large cost
of fear induces destabilizing effect while large population of predators
stabilize the system. Sasmal and Takeuchi [22] discussed the
dynamics of a prey-predator model incorporating two facts: fear effect
and group defense. Mondal et al. [16] analyzed the predator-prey
model considering both the effects of fear and additional food and
showed stability of equilibrium points and Hopf bifurcation. Zhang et
al. [30] investigated the influence of anti- predator behavior due
to fear of predators to a Holling type II prey-predator model allowing a
prey refuge and demonstrated the global stability analysis of the
equilibria of the model and showed Hopf bifurcation. Previous studies
[16-19, 27, 28, 30] are mainly confined in two species that cannot
properly explain the fear effect when multiple species are present. So
present study attempts to investigate the predator fear which affects
prey behavior when prey species are in competition. This study also
address the question of species coexistence.
Takeuchi and Adachi [24] studied the following two competing prey
and one predator model in Lotka-Volterra form:
\begin{equation}
\frac{dx_{1}}{\text{dt}}=x_{1}\left(r_{1}-x_{1}-\alpha x_{2}-\varepsilon y\right),\nonumber \\
\end{equation}\begin{equation}
\frac{dx_{2}}{\text{dt}}=x_{2}\left(r_{2}-\text{βx}_{1}-x_{2}-\mu y\right),\nonumber \\
\end{equation}\(\frac{\text{dy}}{\text{dt}}=y\left(-d+c\varepsilon x_{1}+c\mu x_{2}\right).\)(1) Here the variables\(x_{1}\ \mathrm{\text{and\ }}x_{2}\)represent the densities of prey \(y\) that of predator.\(r_{1}\ \mathrm{\text{and\ }}r_{2}\) are the intrinsic growth rate of
prey. \(\text{α\ }\mathrm{\text{and\ }}\beta\) are parameters
representing the competitive effects between two prey.\(\text{ε\ }\mathrm{\text{and\ }}\mu\) are coefficients of decrease of
prey species due to predation. \(c\) is the equal conversion rate of the
predator. All the parameters are assumed to be positive. In [24],
the authors showed stability and Hopf bifurcation. They also pointed out
that the stable equilibrium bifurcates to a periodic motion with a small
amplitude when the predation rate increases and chaotic motion appears
when one of two prey is superior than the other. Finally, they remarked
that predator mediated coexistence is possible by the close relationship
between preferences of a predator and competitive capacities of two
prey. However, studies in [24] only considers the effect of direct
killing on prey populations and ignore the fear effect in the model
equations. In the real world , the intraspecific competition among
predator exists. Taking the cost of fear on reproduction term only and
intraspecific competition and unequal conversion rate of predator,
system (1) becomes
\begin{equation}
\frac{dx_{1}}{\text{dt}}=x_{1}\left(\frac{r_{1}}{1+k_{1}y}-x_{1}-\alpha x_{2}-\varepsilon y\right),\nonumber \\
\end{equation}\begin{equation}
\frac{dx_{2}}{\text{dt}}=x_{2}\left(\frac{r_{2}}{1+k_{2}y}-\text{βx}_{1}-x_{2}-\mu y\right),\nonumber \\
\end{equation}\(\frac{\text{dy}}{\text{dt}}=y\left(-d+c_{1}\varepsilon x_{1}+c_{2}\mu x_{2}-hy\right)\)(2)
where \(k_{i},\ i=1,\ 2\) represents the level of fear and \(h\)denotes the intraspecific competition within the predator population.\(c_{i},\ i=1,\ 2\) is the conversion efficiency of the predator.
Justification for considering the fear term can be found in [27].
The rest of the paper is organized as follows. In Sec. 3, we state
results on positivity and boundedness of the solutions of the system. In
Sec. 4, existence and stability of different equilibrium points are
discussed. Furthermore, persistence criterion is developed in the same
section. Hopf bifurcation around the positive equilibrium point and the
nature of the limit cycle emerging through Hopf bifurcation are derived
in Sec. 5. Numerical simulations are performed in Sec. 6. A brief
discussion concludes in Sec. 7.