Sports tracking applications are increasingly available on the market, and research has recently picked up this topic. Tracking a user’s running track and providing feedback on the performance are among the key features of such applications. However, little attention has been paid to the accuracy of the applications’ localization measurements. In evaluating the nine currently most popular running applications, we found tremendous differences in the GPS measurements. Besides this finding, our study contributes to the scientific knowledge base by qualifying the findings of previous studies concerning accuracy with smartphones’ GPS components.