Open-Source Assessment Use Case: TETRA™ Detected Significant Improvements in the Quality of the Odoo ERP System

Open-code projects are becoming more and more popular every year, with business efficiency being the key driver. Such an approach results in a more profitable and efficient project because of increased productivity, reduced time to market and development costs, and accelerated innovations. Yet, all this is possible only if an open-source code is of good quality, which may become quite a challenge.

What Is the Issue with Open-Source?

Open-source code implies that anyone can access and edit it. On the one hand, it opens new horizons: thousands of developers reviewing the code, fixing bugs, and bringing their expertise to the table. On the other hand, just a couple of engineers with bad intentions or sloppy coding practices can drop the quality.

So, before incorporating any open-source code into your project, you must assess its quality. It will not be superfluous to think about such a check, even if you are already using an open-code product. Finding and eliminating known weaknesses will significantly increase work efficiency, reduce the number of problems, and avoid possible issues in the future. After all, to harness the power of open-source code, its quality should be assessed across a variety of criteria.

Why Is Intetics Using TETRA™ to Assess Open-Source Products?

TETRA™ (Technical Debt Reduction Platform) performs an in-depth audit and assessment of the product in order to measure technical debt, evaluate product efficiency, and rate compliance with industry standards. It analyzes your product across 8 dimensions:

• Code quality
• UI&UX, documentation
• Security
• Performance
• Business logic
• Architecture quality
• Data quality
• Open-source code use

To demonstrate TETRA™ capabilities and how it facilitates the analysis, we’ve checked the most popular and diversified open-source products from different domains and with different functionality. This goal provides the scope of special requirements in the preparation stage.

What Product Was Chosen for Analysis?

The product to analyze had to be sufficiently significant and well-known while allowing us to use all the TETRA™ tools — from the UX/UI component and business logic to the source code quality, security, performance, etc.

Out of 25 open-code products, the Odoo ERP system was chosen as a starting point. Here are some reasons why:

• A large, complex system. Odoo’s solution includes CRM, e-commerce, billing, accounting, manufacturing, warehouse, project management, and inventory management.
• Worldwide product. Now Odoo ERP System is one of the most rapidly developing open-source CRM systems, with 5+ million users and 1,000+ new employees hired last year.
• Easy customizing system. Odoo’s components are its framework, about 30 core applications (also called official modules), and thousands of community modules.
• The scope of disadvantages. Odoo ERP System is one of the most popular ERPs and is used by thousands of businesses/organizations worldwide. But it has its own set of disadvantages figured out by clients, and TETRA™ should be able to describe potential improvements.

Odoo ERP System Assessment: Step by Step

The project was analyzed across the 5 most critical dimensions that the Technical Debt Reduction Platform (TETRA™) covers:

1. Source Code Quality
2. Usability, UI & Documentation
3. Security
4. Performance
5. Open-Source Code Use.

1. Source Code Quality Assessment

The source code quality is the main artifact of any software product, which often decides the final project performance. Poor code, even if it was supposed to reduce the development time or costs, leads to a bad project.

As a result, developers spend more of their workweek fixing code. Moreover, sloppy code increases the chance of critical errors and security risks. Sometimes, when attempting to improve a bad piece, an engineer uses corrections, which are another injection of poor code.

In terms of Odoo source code evaluation, the main goal was to find poor source code and vulnerabilities, as well as to determine the rate of technical debt. TETRA™ detected:

1) 2 blockers, 28 major, and 2 minor bugs. Blocker issues that can be found in the Mail addon and Tools module are related to the coding rules: the first rule raises an issue in which an operator is used on incompatible types. The second rule is when a special method is defined with an unexpected number of parameters.

2) 2 bugs related to the security section:

• 1 Blocker connected to Odoo’s Tools module, in which access was enabled to external entities in XML parsing. When parsing the XML file, the content of the external entities is retrieved from external storage, such as the file system or network. This may lead, if no restrictions are put in place, to arbitrary file disclosures or server-side request forgery (SSRF) vulnerabilities.
• 1 Critical issue related to the default temporary file name and extension was found in the Deploy module. Creating temporary files using insecure methods exposes the application to race conditions on filenames: a malicious user can try to create a file with a predictable name before the application does. A successful attack can result in other files being accessed, modified, corrupted, or deleted.

3) The Code Smell section contains over 2.8k found issues.

Leaving it as-is means that, at best, it will make it harder for maintainers to make changes to the code. At worst, they’ll be so confused by the state of the code that they’ll introduce additional errors when making changes.

Conclusion

• The percentage of duplicated code is up to 6%. Almost all the duplicated code is contained in the addons rather than in the application.
• The average complexity score for the function is 2.7. At the same time, 75% of the most complex functions are contained in addons, and only 25% are in the application itself.
• From the security and code quality standpoints, there are blocker and critical issues that were included in the resulting report.
• The technical debt estimation showed that the approximate time cost is evaluated at 70 days.

2. Usability Assessment

An appealing interface is an essential part of product design. To facilitate user experience, it should be intuitive and bug-free. Since they are an inevitable part of the design process, accurate usability testing is a must. Its goal is to ensure users are satisfied with the design and layout: in other words, it’s easy to learn new functionality, look for information, and accomplish specific tasks.

With a wide variety of interface assessment metrics and methods, you should choose based on your product specifics. Here are fundamental metrics and methods in terms of our usability testing strategy:

1. Checklist. It’s the first step of usability testing. Perform tasks that represent the most common user goals and/or the most important conversion goals. Generally, the user tasks are goal-oriented and take several minutes.
2. List of found defects.
3. Ease of Use evaluation is a central usability concept. Usability comprises all user experience (UX) ease-related elements with which users can discover content and do more with a design/product. We use the following assessments: Learnability, Memorability, Efficiency, Error tolerability, and Likeability.
4. Documentation testing ensures complex products are intuitive enough. It gained momentum since the solutions are getting more sophisticated, while users focus on simple navigation when choosing a digital product.

Intetics performed a detailed review of the Odoo ERP System in accordance with the TETRA™ approach. The final grade calculated on the basis of these 4 used group metrics is estimated as Good.

Conclusion

During usability testing, no critical or blocker issues were found. The Odoo application has good quality of Ease of Use, and there are full and good quality FAQs and Documentation on the site. There are only a few minor issues (related to the Checklist and the List of found defects) that have little to no impact on the user experience.

3. Security Assessment

The security testing goal was to illuminate risks by leveraging weaknesses within the environment that lead to the obtainment of unauthorized access and/or the retrieval of sensitive information. The following objectives were identified:

• Identify if a remote attacker could penetrate the applications.
• Identify breaches of any sensitive data and information security risks.
• Empirically verify the resistance of applications to misuse and exploitation.
• Report the findings.

We used manual and automated testing in our evaluation. Manual security testing is based on OWASP methods, approaches, and application security assessment standards. During the assessment, each finding is classified according to its severity, reflecting the risk each vulnerability may pose on the business processes implemented by the application.

The following vulnerabilities were found, as shown in Table 1:

Critical severity vulnerabilities detected on:

• Login, Reset Password, and Logout page.
• URL redirectors are common website functionality to forward an incoming request to an alternate resource. This is often done to 1) allow resources to be moved within the directory structure, 2) to avoid breaking functionality for users that request the resource at its previous location, and 3) to implement load balancing, leveraging abbreviated URLs or recording outgoing links. URL redirectors do not necessarily represent a direct security vulnerability but can be abused by attackers deceiving social engineer victims into believing that they are navigating to a site other than the true destination.
• The Path Traversal attack technique allows a hacker access to files, directories, and commands that potentially reside outside the web document root directory. An attacker may manipulate a URL so that the website will execute or reveal the contents of arbitrary files anywhere on the web server. Any device that exposes an HTTP-based interface is potentially vulnerable to Path Traversal.
• SQL injection implies a code injection technique that might destroy your database and is one of the most common web hacking techniques. Malicious code is embedded in SQL statements via web page input.

Several high-severity vulnerabilities were located: one on the Project page and others on most application pages.

The shortcomings identified during the assessment were used to formulate recommendations and mitigation strategies for improving the overall security posture. Thanks to a better security posture understanding, TETRA™ Security Assessment allows you to easily identify, prioritize, and remediate critical application vulnerabilities. Appropriate security mechanisms, remediations, and recommendations will help to defend against cyber-attacks and ensure effective business continuity while using Odoo ERP System.

Conclusion

The TETRA™ assessment approach, together with industry-standard tools, allows us to analyze the overall security posture of the Odoo ERP system and conclude: the app is at risk, and the vulnerabilities need remediation.

4. Performance Assessment

To satisfy modern user needs and keep a user’s interest in the market, an app should be stable, fast, and reliable. Performance testing assesses how a system performs under a particular load. It allows to:

• Eliminate performance bottlenecks in the software application.
• Measure some specific parameters like response time, speed, stability, resource usage, etc.
• Determine whether the application satisfies performance requirements: for instance, what is a page load time under a certain workload, what is the maxim number of concurrent users the application can support, and other important indicators for a business.

During Performance testing we:

• Investigated the Odoo application for a maximum number of users (the upper limit at which the application is still stable). One of the main goals was to assess the system behavior when the load is close to the maximum: the test server didn’t produce a lot of errors affecting user experience. When the number of users was set to 120–130, the percentage of errors grew quickly, and the application became hard to use. Therefore, the acceptable number of users is 100. Test “users” were constantly navigating through the Odoo pages for 1 hour.
• Measured the average response time of some pages. The “5 sec” response time requirement for 80 concurrent virtual users was met: in other words, a user can conveniently work without errors or delays. However, the main test run with 100 users showed that the average response time for most of the requests exceeded the acceptable limit (5 sec). Obviously, we need to keep in mind that results could be affected by VM config, network bandwidth, and even Odoo settings during installation. On the graph below, you can see requests showed response times exceeding 5 sec (everything above the magenta dashed line).

• Checked system usage (CPU, memory, etc.) for a given user load. We analyzed errors during test execution and saw how the error rate depends on the number of concurrent users. It was discovered that the app is well suited for use when the load doesn’t exceed 100 virtual users on the current Virtual Machine. In this case, the percentage of error-generating requests compared to total requests is about 0.1%. With a larger number of users, the error rate increases quickly. For example, when running 150 virtual users, server error responses took about 30%, which is an unacceptable rate.

Conclusion

Based on the test data analysis, the application performance level was estimated as Medium.

The application is quite stress-resistant because it was stable, the related pages were still working, and errors didn’t break a user journey, even though the response time was mostly 5+ sec.

Error-free work for up to 100 active users (with the test server in use) could be a limitation for a large company, so in order to improve the product, it’d be useful to consider performance optimization — to decrease response time under high load and to increase the maximum number of users that could work simultaneously without bugs.

5. Open-Source Code Use Assessment

Increased use of open-source code is an essential part of product-tailoring projects. However, you need to ensure every component is well scrutinized before integrating it into your project. Unlicensed or outdated open-source code, as well as libraries of unknown origin, might conceal vulnerabilities, such as licensing, maintenance, and security issues accompanied by legal and financial risks.

We have defined three key bottlenecks related to the use of open-source code:

1. Use of unlicensed software
2. Use of libraries without information
3. Use of outdated open-source components causing vulnerability risks

All third-party open-source components used in Odoo ERP System were analyzed according to these bottlenecks. Relevance and loyalty research has been done as well.

The project uses the following list of licenses:

• Apache 2.0
• MIT
• BSD
• LGPL
• LGPL-3
• GNU LGPL
• OEEL-1
• HPND

As a result, no critical issues were found, and the general state of the application is Perfect.

Conclusion

All components are licensed correctly and in actual states. However, we recommend updating 4 components to up-to-date versions (6,3% of all used in project components) which are sufficiently outdated:

• Chardet (Python)
• Jinja2 (Python)
• XlsxWriter (Python)
• Select2 (JS)

Assess the Quality of Your Product with TETRA™

Does your application have hidden vulnerabilities? The best way to find out them is through comprehensive quality assessment administered via TETRA™. TETRA™ aims to help designers, developers, PMs, and clients understand how much technical debt their project has.

You will receive an unbiased report that dives deep into your code quality; not only will various areas of your application receive a grade, but the exact issues will be pinpointed. If you are using libraries without information and unlicensed software, or if updates are required, it will be brought to your attention immediately. You’ll also see the number of bugs and vulnerabilities, as well as general usability.

With this information, you’ll know where to make improvements — thereby providing a better user experience and improving your business reputation.

Related Material: Open Source Assessment Use Case: TETRA™ Detected Significant Improvements in the Quality of the Redmine Cross-Platform Application