Manual installation

Metadata database initialization

Knowage requires a database schema to store its own metadata (definition of analyses, datasets and so on). For initializing such a schema, the user must execute the creation scripts provided for the DBMS in use. The package which includes the DDL will contain the following SQL files:

Listing 12 Scripts for metadata schema
     XXX_create.sql
     XXX_create_quartz_schema.sql

where XXX represents the DBMS type (as instance ORA stands for Oracle). The corresponding SQL files for deleting tables are also provided.

Using Tomcat

Dependencies

You must add required libraries into TOMCAT_HOME/lib folder:

  • the JDBC connector for the metadata database with its dependencies (if any);
  • the JDBC connector for the business data database with its dependencies (if any);
  • the commonj library with its dependencies:

Important

Enterprise Edition only

To enable the Import/Export capability, please also add the JDBC connector for HyperSQLDB , taking care of using version 1.8.0.2 .

File system resources

Create the folder TOMCAT_HOME/resources. Such a folder will contain some useful static resources and the indexes for the search engine used by Knowage.

Connection to metadata database

To define connection towards metadata database, edit the TOMCAT_HOME/conf/server.xml and add the information related to the metadata database inside the GlobalNamingResources tag. Specify: username, password, driver class name, JDBC URL and validation query (any valid query to be executed to validate connections). Connection’s name must the jdbc/knowage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
 <Resource auth="Container"
     driverClassName="JDBC driver"
     name="jdbc/knowage"
     password="password"
     type="javax.sql.DataSource"
     url="JDBC URL"
     username="username"
     validationQuery="validation query"
     maxTotal="50"
     maxIdle="50"
     minIdle="10"
     validationInterval="34000"
     removeAbandoned="true"
     removeAbandonedTimeout="3600"
     logAbandoned="true"
     testOnBorrow="true"
     testWhileIdle="true"
     timeBetweenEvictionRunsMillis="10000"
     minEvictableIdleTimeMillis="60000" />

Cache database connection

In some scenarios (for example when defining a cockpit document on top of a file dataset), Knowage requires a database to be used as cache. It is highly recommended to create an empty database schema for this purpose. Then, you need to configure it inside TOMCAT_HOME/conf/server.xml as you did for metadata database. Feel free to type a name of your choice, in this example we used jdbc/ds_cache:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
 <Resource auth="Container"
     driverClassName="JDBC driver"
     name="jdbc/ds_cache"
     password="password"
     type="javax.sql.DataSource"
     url="JDBC URL"
     username="user name"
     validationQuery="validation query"
     maxTotal="50"
     maxIdle="50"
     minIdle="10"
     validationInterval="34000"
     removeAbandoned="true"
     removeAbandonedTimeout="3600"
     logAbandoned="true"
     testOnBorrow="true"
     testWhileIdle="true"
     timeBetweenEvictionRunsMillis="10000"
     minEvictableIdleTimeMillis="60000" />

Connection to business data

Edit the TOMCAT_HOME/conf/server.xml and add the information related to the database containing business data to be analysed by Knowage inside the GlobalNamingResources tag, specifying username, password, driver class name, URL and validation query. Feel free to type a name of your choice, in this example we used jdbc/dwh:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
 <Resource auth="Container"
         driverClassName="JDBC driver"
         name="jdbc/dwh"
         password="password"
         type="javax.sql.DataSource"
         url="JDBC URL"
         username="username"
         validationQuery="validation query"
         maxTotal="50"
         maxIdle="50"
         minIdle="10"
         validationInterval="34000"
         removeAbandoned="true"
         removeAbandonedTimeout="3600"
         logAbandoned="true"
         testOnBorrow="true"
         testWhileIdle="true"
         timeBetweenEvictionRunsMillis="10000"
         minEvictableIdleTimeMillis="60000"
         factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" />

Environment variables definition

Edit the file TOMCAT_HOME/conf/server.xml in Tomcat and add the following constants in the GlobalNamingResources tag.

<Environment name="resource_path" type="java.lang.String" value="${catalina.home}/resources" />
<Environment name="sso_class" type="java.lang.String" value="it.eng.spagobi.services.common.JWTSsoService" />
<Environment name="service_url" type="java.lang.String" value="http://localhost:8080/knowage" />
<Environment name="hmacKey" description="HMAC key" type="java.lang.String" value="PUT ANY RANDOM STRING HERE" />
<Environment name="password_encryption_secret" description="File for security encryption location" type="java.lang.String" value="complete_file_path_with_file_name" />

Such environment variables have the following meaning:

  • resource_path: resources folder path,
  • sso_class:SSO connector class name,
  • service_url:backend services address, typically set to http://localhost:8080/knowage,
  • hmacKey: secret key to generate JWT tokens used by the default security mechanism. You must change it, and do not distribute it. You can put any random alphanumeric string in it, and you can change it everytime you want, you just need to restart Tomcat to apply the change,
  • password_encryption_secret: the complete path of a file to contain the password encryption secret. The file must contain random text of any length. This is a security configuration, so don’t use short strings. For example, you can create a file and write text into it. Do not distribute it for any reason, create at least a backup copy of the file. After the first start of Knowage, it will no longer be possible to change the secret key. In case you lost this secret, look at the paragraph below to see how to update the passwords of existing users.

Important

Again we stress the point that the HMAC key must be a random string. Please DO NOT copy and paste it from this documentation, since this will compromise the security of the application.

Below you can see an example of configuration of the above variables in the server.xml file

<Environment name="resource_path" type="java.lang.String" value="${catalina.home}/resources"/>
<Environment name="sso_class" type="java.lang.String" value="it.eng.spagobi.services.common.JWTSsoService"/>
<Environment name="service_url" type="java.lang.String" value="http://mydomain.com/knowage"/>
<Environment name="hmacKey" description="HMAC key" type="java.lang.String" value="a random string"/>
<Environment name="password_encryption_secret" description="File for security encryption location" type="java.lang.String" value="${catalina.home}/conf/knowage.secret"/>

Changing the secret key for password encryption

The password encryption secret key must be set during the installation and cannot be changed anymore, otherwise Knowage will no longer be able to authenticate already defined users. In case the secret key is lost you must create a new one, configure it into Knowage as described above and update passwords of existing users direclty into Knowage metadata database (SBI_USER table). For this reason Knowage provides you a tool to get new encrypted values. This tool is a Java class that is shipped with the knowage-utils library; it accepts 2 input parameters:

  • the complete path of the password encryption secret file;
  • the password value in plaintext.

Below is an example of invoking the tool by command line using ‘mypassword’ as the plaintext password to be encrypted (of course TOMCAT_HOME must be replaced by the actual Tomcat base folder path).

1
java -cp "<TOMCAT_HOME>/webapps/knowage/WEB-INF/lib/knowage-utils-7.2.0.jar" it.eng.spagobi.security.utils.PasswordEncryptionToolMain <TOMCAT_HOME>/conf/knowage.secret mypassword

This procedure must be repeated for all already existing users.

Applications deploy

To deploy Knowage you have to copy all the WAR files inside the TOMCAT_HOME/webapps folder. Once the first start is ended each WAR file will be unzipped. It is also possible to unzip the WAR files manually using the unzip utility.

Thread pool definition

You must configure TOMCAT_HOME/conf/server.xml file and add the settings related to the pool of thread editing the GlobalNamingResources tag, as shown follow.

1
     <Resource auth="Container" factory="de.myfoo.commonj.work.FooWorkManagerFactory" maxThreads="5" name="wm/SpagoWorkManager" type="commonj.work.WorkManager"/>

Advanced memory settings

It is recommended to increase the memory dimension used by the application server. This can be done by adjusting some properties. The memory required by each application server depends on many factors: number of users, type of analyses, amount of handled data, etc. The minimum requirements are Xms1024m and Xmx2048m.

[LINUX] Insert at the beginning of the TOMCAT_HOME/bin/setenv.sh file this command:

1
     export JAVA_OPTS="$JAVA_OPTS -Xms1024m -Xmx2048m -XX:MaxPermSize=512m"

[WIN] Insert at the beginning of the TOMCAT_HOME/bin/setenv.bat file this command:

1
     set JAVA_OPTS= %JAVA_OPTS% -Xms1024m Xmx2048m -XX:MaxPermSize=512m

Advanced Connector settings

Important

It is highly recommend to add URIEncoding=”UTF-8” attribute to server.xml file connector tags in order to avoid special characters issues.

1
     <Connector address="0.0.0.0" port="8009" protocol="AJP/1.3" maxPostSize="2097152000" redirectPort="8443" URIEncoding="UTF-8" />

Configuration of the metadata db dialect

Important

This step is not mandatory anymore

Knowage is now able to autonomously determine following configuration. That said, the following is not mandatory anymore however a user can force it to specific value.

Verify that the right dialect has been set inside hibernate.cfg.xml files. We list all the possible dialects that can be used:

1
2
3
     <property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>,
     <property name="hibernate.dialect">org.hibernate.dialect.PostgreSQLDialect</property>
     <property name="hibernate.dialect">org.hibernate.dialect.Oracle9Dialect</property>

You have to configure these following Hibernate configuration files and set the chosen dialect:

1
2
3
4
5
6
7
8
     knowagekpiengine/WEB-INF/classes/hibernate.cfg.xml
     knowagegeoreportengine/WEB-INF/classes/hibernate.cfg.xml
     knowage/WEB-INF/classes/hsql/hibernate.cfg.xml
     knowage/WEB-INF/classes/hibernate.cfg.xml
     knowagesvgviewerengine/WEB-INF/classes/hibernate.cfg.xml
     knowagemeta/WEB-INF/classes/hibernate.cfg.xml
     knowagecockpitengine/WEB-INF/classes/hibernate.cfg.xml
     knowagedataminingengine/WEB-INF/classes/hibernate.cfg.xml

Important

Context update

The modification of these files will be effective as soon as the web application is reloaded or the application server is restarted.

Modification of the Quartz configuration

Important

This step is not mandatory anymore

Knowage is now able to autonomously determine following configuration. That said, the following is not mandatory anymore however a user can force it to specific value.

The scheduler is configured in knowage/WEB-INF/classes/quartz.properties. It is essential to enhance in this file the property org.quartz.jobStore.driverDelegateClass with the right value, according to the metadata database in use. Following the possible values:

1
2
3
4
5
6
7
      # Hsqldb delegate class
      #org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.HSQLDBDelegate
      # Mysql delegate class org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
      # Postgres delegate class
      #org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
      # Oracle delegate class
      #org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.oracle.OracleDelegate

Clustering

When Knowage is installed in cluster with several nodes, it is necessary to activate the Cluster modality, adding these parameters to the knowage/WEB-INF/classes/quartz.properties file of every involved machines:

1
2
3
4
      org.quartz.jobStore.isClustered = true
      org.quartz.jobStore.clusterCheckinInterval = 20000
      org.quartz.scheduler.instanceId = AUTO
      org.quartz.scheduler.instanceName = RHECMClusteredSchedule

Logging

It is necessary to set up a folder where Knowage and its analytical engines can store their respective log files. From now on, we will call LOG_DIR such folder and LOG_DIR_PATH the path that leads to it. This path is configured in file log4j.properties located inside the \WEB-INF\classes\ available in each web application. Shortly, to configure the Knowage log folder the user must execute the following steps:

  • create the LOG_DIR folder on all cluster nodes on which it is intended to deploy Knowage Server and/or one of its analytical engines. The LOG_DIR_PATH string must be the same for every node;
  • [LINUX] verify that Knowage has write permissions on this folder; set the property log4j.appender.knowage.File inside the WEB-INF/classes/log4j.properties to LOG_DIR_PATH/knowage.log;
  • set the property log4j.appender.knowageXXXXXEngine.File inside the WEB-INF/classes/log4j.properties file of each engine to LOG_DIR_PATH/knwoageXXXXXEngine.log;
  • only for the Birt Engine, to set the property logDirectory inside the WEB-INF/classes/BirtLogConfig.properties file of the knowagebirtreportengine application to LOG_DIR_PATH.

Enable Java Security Manager

In Knowage, a user can create datasets, LOVs, etc.. with script languages like JavaScript. That introduces a security concern where a malicious user can execute code that can break the entire system. Java allows a system administrator to enable a Security Manager and to create a sandbox to limit privileges around the code that execute the script.

The Security Manager can be enabled with the following steps:

  • Write a Security Policy for the Security Manager;
  • Enable the Security Manager in the JVM.

The Security Policy is a text file read by a Security Manager that specifies all the privileges that a JVM can give to Java code: Tomcat has already a default policy in the file TOMCAT_HOME/conf/catalina.policy but is too much strict for Knowage code that needs to write multiple logs, make network connection and execute external applications. Knowage is already secured and can use a more relaxed policy like:

Listing 13 Complete path of the script
grant {
        permission java.security.AllPermission;
};

This policy can be saved to TOMCAT_HOME/conf/knowage-default.policy.

To enable the Security Manager a system administrator have to add some options to the Java JVM:

[LINUX] Insert at the end of the TOMCAT_HOME/bin/setenv.sh file this command:

1
export JAVA_OPTS="$JAVA_OPTS -Djava.security.manager -Djava.security.policy=$CATALINA_HOME/conf/knowage-default.policy"

[WIN] Insert at the end of the TOMCAT_HOME/bin/setenv.bat file this command:

1
set JAVA_OPTS= %JAVA_OPTS% -Djava.security.manager -Djava.security.policy=%CATALINA_HOME%\conf\knowage-default.policy

Installation of Chromium Cockpit Export script

Important

Enterprise Edition only

Chromium Cockpit Export script is only available for Enterprise Edition.

Extract archive knowage-cockpit-export-installation.zip to /opt/knowage:

Listing 14 Complete path of the script
/opt/knowage/cockpit-export/cockpit-export.js

For alternatives path you have to fix internal.nodejs.chromium.export.path in Knowage Configuration Management.