Empirically investigating the human role on the code smell effect

Abstract

Context: Code smell is a term commonly used to describe potential problems in the design of object-oriented systems. The theory about code smells, initially presented in nineties, is focused on characterization of different types of smells and heuristics for their detection and removal. Since then, empirical studies have evaluated the impact of the practical adoption of the code smell concept on software development. We call this issue the code smell effect. The findings of empirical studies on the code smell effect have presented inconsistencies with respect to what is expected by the theory. This phenomenon is not well understood. Objective: Enhancing the understanding of the code smell effect, characterizing problems hindering the practical adoption of the smell concept and investigating their origin. More specifically, we focus on the major confounding factor affecting observations on the code smell effect, which is the human role on smell detection. The human role is mostly disregarded by the theory and it strongly affects the experiments on the subject. Method: We explore the problem empirically, from two perspectives. First, we synthesize the current knowledge on the code smell effect from a set of primary studies published in the literature. To do this, we performed a systematic review based on thematic synthesis. Second, we propose the investigation of factors affecting the human evaluation of smells. To do this, we performed a family of controlled experiments. In it, we investigate an extensive number of factors affecting the human perception of smells. The factors are the use of design comprehension tool support; developer's experience, knowledge and training; and software size. From the thematic synthesis and the family of controlled experiments, we identify challenges for the area and present them as a research agenda. Results: We find out that human evaluation of smell should not be trusted. The studies indicate that the agreement on smell detection is low. They also show that demographic attributes, such as developers' experience, impact the agreement on smell detection. We also concluded that, nowadays, there are not evidences supporting the use of code smells for practical evaluation of design quality. Our evidences are divergent findings correlating code smells and software attributes, such as changes on source code. In some cases, the studies converge showing that code smells do not impact some attributes. This happens, for example, with studies correlating smells and effort on maintenance. Conclusion: This thesis points out to the gap between the theory and the experimental findings about the code smell effect. Our evidences indicate that the main source for this gap is the low number of studies focusing on the human role on the smell effect. The researches on the subject should focus on better understanding this issue. This not seems the path followed by the area, which has directed its effort on researches focused on tool assessments, such as automatic detection or smell removal. In order to face the problem, we propose a research agenda, indicating that the area needs to: i) investigate the nature of smells, grouping them according to their characteristics; ii) explore human factors affecting smell detection, and their relationships; iii) explore cognitive aspects affecting human perception of smells; and iv) outline attributes of the experiments in order to classify the context where findings can be grouped. These challenges indicate some paths to be followed by the area in order to systematize the use of code smell in the practice of the software development. Despite difficulties, we believe that this is possible and our thesis contributes for this journey.

Contexto: Code smell ´e um termo comumente utilizado para descrever potenciais problemas em projetos de software orientados a objetos. A teoria relacionada a code smells, inicialmente apresentada nos anos noventa, tem foco na caracteriza¸c˜ao de diferentes tipos de smells e nas estrat´egias para sua detec¸c˜ao e remo¸c˜ao. Desde ent˜ao, estudos emp´ıricos tˆem avaliado o impacto da ado¸c˜ao do conceito de code smell no desenvolvimento de software, ou seja, o “efeito code smell”. Os resultados destes estudos tˆem apresentado inconsistˆencias com rela¸c˜ao aos efeitos propostos pela teoria. As causas deste fenˆomeno n˜ao s˜ao bem compreendidas. Objetivo: Melhorar a compreens˜ao sobre o efeito code smell, identificando problemas que tˆem dificultado uso do conceito no desenvolvimento de software. Mais especificamente, nos concentramos no papel humano na detec¸c˜ao de smells. O fator humano ´e uma vari´avel pouco considerada pela teoria, mas afeta significativamente os resultados dos estudos emp´ıricos sobre o tema. M´etodo: N´os exploramos o problema de forma emp´ırica, a partir de duas perspectivas. Primeiro, n´os sintetizamos o conhecimento relacionado ao efeito code smell a partir de um conjunto de estudos prim´arios existente na literatura. Para isso, n´os realizamos uma revis˜ao sistem´atica, com base no m´etodo de s´ıntese tem´atica. Segundo, n´os propusemos a investiga¸c˜ao de fatores que afetam a percep¸c˜ao humana sobre smells. Neste caso, n´os realizamos uma fam´ılia de experimentos controlados, investigando um extensivo n´umero de fatores que afetam a percep¸c˜ao humana sobre smells. Os fatores s˜ao: o uso de ferramentas de suporte `a compreens˜ao do projeto de software; a experiˆencia; o conhecimento te´orico sobre code smells e o treinamento dos desenvolvedores; e o tamanho do software. A partir da s´ıntese tem´atica e da fam´ılia de experimentos, n´os identificamos alguns desafios para a ´area, apresentandoos como uma agenda de pesquisa. Resultados: N´os percebemos que a avalia¸c˜ao humana de smells n˜ao ´e confi´avel. Os estudos indicam que o grau de concordˆancia relacionado `a avalia¸c˜ao humana ´e baixo. Al´em disso, os estudos mostram que atributos demogr´aficos, como experiˆencia do desenvolvedor, afetam o grau de concordˆancia na detec¸c˜ao de smells. Tamb´em conclu´ımos que n˜ao h´a evidˆencias que indiquem o uso do conceito de code smells como ferramenta para a avalia¸c˜ao da qualidade dos projetos de software. Estas evidˆencias s˜ao as inconsistˆencias nos resultados correlacionando os smells e atributos de software, como mudan¸cas no c´odigo. Em alguns casos, os estudos concordam que os smells n˜ao tem correla¸c˜ao com certos atributos. Isso ocorre, por exemplo, com estudos correlacionando smells e esfor¸co na manuten¸c˜ao de software. Conclus˜ao: Esta tese evidencia uma lacuna existente entre a teoria e os estudos emp´ıricos relacionados ao efeito code smell. Nossas evidˆencias apontam o baixo n´umero de estudos abordando o papel humano na detec¸c˜ao de smells como principal causa desta lacuna. Pesquisas na ´area devem se concentrar em entender este aspecto. Este n˜ao parece ser o caminho seguido pela ´area, que tem se concentrado principalmente no desenvolvimento de ferramentas de suporte, como ferramentas para detec¸c˜ao autom´atica e remo¸c˜ao de smells. Para auxiliar neste sentido, n´os propomos uma agenda de pesquisa indicando pontos importantes a serem considerados: i) investiga¸c˜ao da natureza dos smells, agrupando-os de acordo com suas caracter´ısticas; ii) explora¸c˜ao de fatores humanos afetando a detec¸c˜ao de smells e seus inter-relacionamentos; iii) explora¸c˜ao de aspectos cognitivos afetando a percep¸c˜ao humana de smells; e iv) identifica¸c˜ao de atributos relacionados aos estudos emp´ıricos de forma a delinear o contexto onde resultados podem ser agrupados. Estes desafios indicam alguns caminhos que a ´area pode seguir para sistematizar o uso do conceito de code smell no desenvolvimento de software. Apesar das dificuldades, n´os acreditamos que isso ´e poss´ıvel e nossa tese visa a contribuir neste sentido.