The sine cosine algorithm (SCA) is a metaheuristic algorithm proposed in recent years that does not resort to nature-related metaphors but explores and exploits the search space with the help of two simple mathematical functions of sine and cosine. SCA has fewer parameters and a simple structure and is widely used in various fields. However, it tends to fall into local optimality because it does not have a well-balanced exploitation and exploration phase. Therefore, in this paper, a new, improved SCA algorithm (QCSCA) is proposed to improve the performance of the algorithm by introducing a quick move mechanism and a crisscross mechanism to SCA and adaptively improving one of the parameters. To verify the effectiveness of QCSCA, comparison experiments with some conventional metaheuristic algorithms, advanced metaheuristic algorithms, and SCA variants are conducted on IEEE CEC2017 and CEC2013. The experimental results show a significant improvement in the convergence speed and the ability to jump out of the local optimum of the QCSCA. The scalability of the algorithm is verified in the benchmark function. In addition, QCSCA is applied to 14 real-world datasets from the UCI machine learning database for selecting a subset of near-optimal features, and the experimental results show that QCSCA is still very competitive in feature selection (FS) compared to similar algorithms. Our experimental results and analysis show that QCSCA is an effective method for solving global optimization problems and FS problems.