A Systematic Survey of Security-oriented Static Analysis Tools  |  2021 - Under Review

Authors: Sofia Reis and Rui Abreu

ACM Computing Surveys (CSUR) - Journal, Core Q1
Abstract: Over the past decades, a vast amount of static analysis tools has been studied, designed, and produced in academia and industry. Static analysis is a technique capable of examining entire codebases against coding rules before executing the source code. These tools have the potential of addressing issues early on the software development lifecycle by pinpointing software defects such as security vulnerabilities. Detecting this type of issues early on decreases the amount of money wasted on maintenance and makes the software safer by default. Despite these techniques being around for a few decades, there are still several research opportunities and problems to be solved. With this systematic survey, we aim to solve one of them: unstructured knowledge. The knowledge regarding these tools is very spread out on the internet and academic papers, which turn the understanding and adoption very difficult. In this systematic literature review, we organize and describe the current state of security-oriented static analysis tools (SoSATs) by providing a a structured overview of previous approaches, including techniques, programming languages and weaknesses spectra, performance, availability, and popularity. This work is a contribution to both industry and academia: industry, by providing a complete description of the tools; and academia, by providing a set of open research opportunities in the field.

  Website (WIP)  

Fixing Vulnerabilities Potentially Hinders Maintainability  |  2021 - To Appear

Authors: Sofia Reis, Rui Abreu and Luís Cruz

Empirical Software Engineer (EMSE'21) - Journal, Core Q1
Abstract: Security is a requirement of utmost importance to produce high-quality software. However, there is still a considerable amount of vulnerabilities being discovered and fixed almost weekly. We hypothesize that developers affect the maintainability of their codebases when patching vulnerabilities. This paper evaluates the impact of patches to improve security on the maintainability of open-source software. Maintainability is measured based on the Better Code Hub’s model of 10 guidelines on a dataset, including 1300 security-related commits. Results show evidence of a trade-off between security and maintainability for 41.90% of the cases, i.e., developers may hinder software maintainability. Our analysis shows that 38.29% of patches increased software complexity and 37.87% of patches increased the percentage of LOCs per unit. The implications of our study are that changes to codebases while patching vulnerabilities need to be performed with extra care; tools for patch risk assessment should be integrate into the CI/CD pipeline; computer science curricula needs to be updated; and, more secure programming languages are necessary.

  April 2021   Replication Package

Demystifying the Combination of Dynamic Slicing and Spectrum-based Fault Localization  |  2019

Authors: Sofia Reis, Rui Abreu and Marcelo d'Amorim

International Joint Conference on Artificial Intelligence (IJCAI'19) - Conference (Main Track), 17.8% (850/4752), Core A*
Abstract: Several approaches have been proposed to reduce debugging costs through automated software fault diagnosis. Dynamic Slicing (DS) and Spectrum- based Fault Localization (SFL) are popular fault diagnosis techniques and normally seen as complementary. This paper reports on a comprehensive study to reassess the effects of combining DS with SFL. With this combination, components that are of- ten involved in failing but seldom in passing test runs could be located and their suspiciousness reduced. Results show that the DS-SFL combination, coined as Tandem-FL, improves the diagnostic accuracy up to 73.7% (13.4% on average). Furthermore, results indicate that the risk of missing faulty statements, which is a DS’s key limitation, is not high — DS misses faulty statements in 9% of the 260 cases. To sum up, we found that the DS-SFL combination was practical and effective and encourage new SFL techniques to be evaluated against that optimization.

May 2019    PDF   Tool

Leveraging Known Vulnerabilities to Modernize Static Analysis Tools  |  2018

Authors: Sofia Reis and Rui Abreu

PhD Open Days 2018 (Lisbon, PT) @ Instituto Superior Técnico - Poster

March 2018    Poster  

A Database of Existing Vulnerabilities to Enable Controlled Testing Studies  |  2017

Authors: Sofia Reis and Rui Abreu

International Journal of Secure Software Engineering (IJSSE) 8(3) - Journal Paper (Special Invitation)
Abstract: From holding worldwide companies' information hostage to keeping several distributed systems down for hours, the last years were marked by several security attacks which are the result of complex software and its fast production. There are already tools which can be used to help companies detect vulnerabilities responsible for such attacks. However, their reliability is still not the best and well discriminated. In software testing, researchers tend to use hand-seeded test cases or mutations due to the challenges involved in the extraction or reproduction of real test cases which might not be suitable for testing techniques, since both approaches can create samples that inadvertently differ from the real vulnerabilities and thus might lead to misleading assessments of the tools' capabilities. The lack of databases of real security vulnerabilities is an issue since it hampers the tools' evaluation and categorization. To study these tools, the researchers created a database of 682 real test cases which is the outcome of mining 248 repositories for 16 different vulnerability patterns.

November 2017   10.4018/IJSSE.2017070101

SECBENCH: A Database of Real Security Vulnerabilities  |  2017

Authors: Sofia Reis and Rui Abreu

International Workshop on Secure Software Engineering in DevOps and Agile Development (SecSE 2017) co-located with the 22nd European Symposium on Research in Computer Security (ESORICS 2017) - Workshop Proceedings
Abstract: Currently, to satisfy the high number of system requirements, complex software is created which turns its development costintensive and more susceptible to security vulnerabilities. In software security testing, empirical studies typically use artificial faulty programs because of the challenges involved in the extraction or reproduction of real security vulnerabilities. Thus, researchers tend to use hand-seeded faults or mutations to overcome these issues which might not be suitable for software testing techniques since the two approaches can create samples that inadvertently differ from the real vulnerabilities and thus might lead to misleading assessments of the capabilities of the tools. Although there are databases targeting security vulnerabilities test cases, one database contains only real vulnerabilities, the other ones are a mix of real and artificial or even only artificial samples. Secbench is a database of real security vulnerabilities mined from Github which hosts millions of open-source projects carrying a considerable number of security vulnerabilities. We mined 248 projects - accounting to almost 2M commits - for 16 different vulnerability patterns, yielding a Database with 682 real security vulnerabilities.

September 2017   urn:nbn:de:0074-1977-5

Using Github to Create a Dataset of Natural Occurring Vulnerabilities  |  2017

Authors: Sofia Reis and Rui Abreu

28th International Workshop on Principles Of Diagnosis (Bréscia, IT) - Poster

September 2017    Poster   Abstract

Assessing software vulnerabilities using Naturally Occurring Defects  |  2017

MSc Thesis at Faculty of Engineering of the University of Porto - Technical Report

July 2017   PDF  Score: 19/20 - top 5%