The mechanism of fine grains migration is the basis and key to understand the seepage erosion process of gravel soil, which is significance to study the disaster-caused mesomechanism of rain erosion process on sandy gravel slopes. However, the migration mode and critical condition for the transition of migration state are unclear. The migration types of fine grains differ from different physical and hydraulic conditions. To reveal the overall movement characteristics of fine grains and the critical conditions for the change of their motion state during the process of gravel soil erosion, this study adopted the visual cylindrical infiltration test and discrete element numerical simulation to analyze the influencing factors and mechanism of the fine grains migration, deposition, and self-dredging. The results show that: 1) the migration of fine grains is significantly affected by the gradation and hydraulic gradient but not by the initial porosity. The gradation has a greater effect than the hydraulic gradient. 2) The overall motion of fine grains under hydraulic forces can be divided into two states: deposition and self-dredging. The fine grains of gravel soil with unstable internal structure are in the self-dredging state, and that with stable and transitional internal structure will change from deposition to self-dredging with the increase of hydraulic gradient. 3) The motion state of fine grains has an obvious boundary under the combined effects of grain size ratio and hydraulic gradient, and the critical condition for fine grains transition from deposition to self-dredging is obtained to be i = 3.4 - 0.12\texte^\left( D_15/d_85 \right)/1.5 .