In this work, we focus on lightweight and accurate face alignment. For that purpose, we propose an algorithm design that promotes a most recently published face alignment method in terms of model size and computing cost while maintaining high accuracy of face alignment. Specifically, we construct a lightweight two-stage neural network. The first stage estimates boundary heatmaps on the facial region, which are then used to guide the facial landmark position prediction in the second stage. For the first stage, we compress an HourglassNet-based structure by reducing the numbers of feature channels and convolutional kernels and optimizing the structure of Hourglass block by ShuffleNet modules. For the second stage, we compress the subnet by utilizing DeLighT, a recently published lightweight version of Transformer. Experimental results on several standard facial landmark detection datasets show that the proposed algorithm achieves sharp advances in model compactness and computing efficiency while keeping a state-of-the-art level of accuracy in facial landmark detection.