Three-dimensional (3D) point clouds (PCs) and meshes have increasingly become available and indispensable for diversified applications in work and life. In addition, 3D visual data contain information from any viewpoint when needed, introducing new challenges and opportunities. As in the cases of 2D images and videos, computationally modeling saliency and quality for 3D PCs and meshes are important for widespread, economical adaption and optimization. This paper aims to provide a comprehensive overview of the related signal presentation and existing saliency and quality models, with major perspectives from the ultimate users (i.e., humans or machines), modeling methodology (with handcrafted features or machine learning), and modeling scope (generic or utility-oriented models). Possible future research directions are also discussed.