General Performance Test

This Wiki page describes how the performance of a codeBeamer instance can be measured.

Requirements

Install the proper codeBeamer version from the following folder: You must login to see this link. Register now, if you have no user account yet.

Do not start post-install.

Because requests are sent from several accounts parallel to codeBeamer the number of required floating ALM licenses is min. 46.

JMeter must be installed

Most Linux distribution include JMeter however it can be also downloaded from http://jmeter.apache.org/download_jmeter.cgi.

Running the performance tests on the same host where also codeBeamer is running does not significantly impact the results and has the advantage that network speed does not impact the test results.

Currently JMeter tests run with jmeter 3.3.x

JMeter 3.3 requires Java 8 or higher.

Performance Monitoring

Please refer to Performance Monitoring how to monitor your codeBeamer instance.

Download Test scripts and database dump

Test data: general-performance.zip

Import database dump

To import database from the attached zip copy dmp file to oracle dpdump directory.

Run the following Oracle scripts

CREATE TABLESPACE perf_test_tablespace DATAFILE '<oracle instance's tablespace folder>/perf_test_datafile01.dbf' SIZE 100M AUTOEXTEND ON NEXT 100M MAXSIZE 1000M;
CREATE USER C##PERF_TEST IDENTIFIED BY PERF_TEST DEFAULT TABLESPACE perf_test_tablespace QUOTA UNLIMITED ON perf_test_tablespace;
GRANT CREATE SESSION TO C##PERF_TEST;
GRANT CREATE TYPE TO C##PERF_TEST ;
GRANT CREATE TABLE TO C##PERF_TEST ;
GRANT CREATE CLUSTER TO C##PERF_TEST ;
GRANT CREATE TRIGGER TO C##PERF_TEST ;
GRANT CREATE OPERATOR TO C##PERF_TEST ;
GRANT CREATE SEQUENCE TO C##PERF_TEST ;
GRANT CREATE INDEXTYPE TO C##PERF_TEST ;
GRANT CREATE PROCEDURE TO C##PERF_TEST ;
GRANT CREATE VIEW TO C##PERF_TEST ;
GRANT EXECUTE ON CTXSYS.CTX_DDL TO C##PERF_TEST ;
GRANT EXECUTE ON DBMS_LOB TO C##PERF_TEST ;
GRANT IMPORT FULL DATABASE TO C##PERF_TEST;
GRANT READ, WRITE ON DIRECTORY data_pump_dir TO C##PERF_TEST;

To import file use Oracle impdp:

impdp C##PERF_TEST/PERF_TEST directory=data_pump_dir schemas=C##PERF_TEST REMAP_SCHEMA=C##PERF_TEST:C##PERF_TEST REMAP_TABLESPACE=perf_test_tablespace:perf_test_tablespace dumpfile=PERF_TEST_DUMP01.DMP logfile=data_pump_dir:perf_imp.log

After import run the following script on the database:

BEGIN
	DBMS_STATS.GATHER_SCHEMA_STATS (
		ownname => 'C##PERF_TEST',
		estimate_percent => 100,
		method_opt => 'FOR ALL COLUMNS SIZE AUTO',
		degree => 1,
		granularity => 'ALL',
		cascade => TRUE,
		options => 'GATHER'
	) ;
END ;

Setup Oracle 12

We measured our performance results using Adaptive Query Optimization ( OPTIMIZER_ADAPTIVE_FEATURES ) setting is disabled. Please make sure this setting is available and disabled on dedicated Oracle 12 server before starting Performance tests.

Setup codeBeamer

Change the database in general.xml (change localhost if necessary):

<database JDBC_ConnectionURL="jdbc:oracle:thin:@localhost:1521:orcl" JDBC_Driver="oracle.jdbc.driver.OracleDriver" JDBC_Password="PERF_TEST" JDBC_Password_Storage="plain" JDBC_Username="C##PERF_TEST"/>

Set the installation tag:

<installation setup="false"></installation>

Change document store to database (store-into-db):

<document access-log="true" auto_restore_from_trash="30s" display-plain-text-as-html="false" log-expiration="0" relocatable="false" storage-path="" store-into-db="true"/>

After these changes codeBeamer can be started.

After codeBeamer start setup the license. Default System Administrator: (bond / 007)

Executing Performance Tests

The tests/scripts require that jmeter is in your PATH.

The performance tests can be executed with the following command (zip contains this jmx file)

On Windows use: general-perf.bat

general-perf.bat -n -t general-performance.jmx -Jprotocol=http -Jserver.host=<codeBeamer url> -Jport=<codeBeamer_port> -Jproject.admin.ct=4 -Jdev.ct=4 -Jtester.ct=4 -Jtestlead.ct=4 -Jrepeat=4

On *nix use:general-perf.sh

./general-perf.sh -n -t general-performance.jmx -Jprotocol=http -Jserver.host=<codeBeamer url> -Jport=<codeBeamer_port> -Jproject.admin.ct=4 -Jdev.ct=4 -Jtester.ct=4 -Jtestlead.ct=4 -Jrepeat=4

You don't need to define result file location since it is in the bat/sh. Test result is generated into: general-performance_<Y-m-dd_H-M-S>.jtl file

Performance Verification Data

Please provide Intland the files below:

• general-performance_<Y-m-dd_H-M-S>.jtl
• cb.txt (compressed)

Test Cases

The test do the following:

(Default: 4 users parallel 4 turns)

The table below shows configurable parameters and their default values:

Performance Test Results

The performance test results will be stored into a files with extension jtl.

Test hardware:

• Hardware: IBM System x3690 X5 Server and 256 GB Memory, 20 CPUs x Intel(R) Xeon(R) CPU E7- 2870 @ 2.40GHz
• codeBeamer and Oracle were running on two separated virtual machines
• Running on Oracle Linux Server release 7.3
• Database: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0

codeBeamer 9.3.0 Results - 20181001

Most important page load time: