The HDPE/ nano-magnesium hydroxide (MH) non-halogen flame retardant composite is modified by nano-montmorillonit (OMMT), the influences of the mass of OMMT on the flame retardancy, mechanical character and rheology of composite are studied. The results show that the flame retardancy of HDPE modified by OMMT alone is deficient. Adding OMMT into HDPE/MH could bring synergistic effect in improving the flame retardancy, but the mass of MH should be above the 60% of HDPE’s mass. When the ratio of HDPE/MH/OMMT is 100/60/6, the flame retardancy and mechanical properties of composite are good. The melt of HDPE/MH/OMMT is pseudoplastic, the mass of OMMT over 6% of HDPE increases the viscosity in low shear rate, the rheology becomes worse. At the same time, the increase of OMMT causes the increase of pseudoplastic character of the melt.