Aiming at the inconvenience of many people pressing elevator buttons, designed a smart DSP voice recognition controlling system for elevators. With TMS320C6713 as the system operation control center, TLV320AIC23B as the analog end of voice input and output and EP2C5Q208C8 FPGA as the system I/O expanding port, the system was integrated to achieve non-specific people, isolated word and small vocabulary embedded speech recognition elevator controlling. Test results show that the system architecture is logically arranged and speech recognition accuracy of the elevator control reaches 80%.