It has been showed by Byde that it is possible to attach a Delaunay-type end to a compact nondegenerate manifold of positive constant scalar curvature, provided it is locally conformally flat in a neighborhood of the attaching point. The resulting manifold is noncompact with the same constant scalar curvature. The main goal of this paper is to generalize this result. We will construct a one-parameter family of solutions to the positive singular Yamabe problem for any compact non-degenerate manifold with Weyl tensor vanishing to sufficiently high order at the singular point. If the dimension is at most 5, no condition on the Weyl tensor is needed. We will use perturbation techniques and gluing methods.