Manual installation¶
Metadata database initialization¶
Knowage requires a database schema to store its own metadata (definition of analyses, datasets and so on). For initializing such a schema, the user must execute the creation scripts provided for the DBMS in use. The package which includes the DDL will contain the following SQL files:
XXX_create.sql
XXX_create_quartz_schema.sql
where XXX represents the DBMS type (as instance ORA stands for Oracle). The corresponding SQL files for deleting tables are also provided.
Using Tomcat¶
Dependencies¶
You must add required libraries into TOMCAT_HOME/lib
folder:
- the JDBC connector for the metadata database with its dependencies (if any);
- the JDBC connector for the business data database with its dependencies (if any);
- the commonj library with its dependencies:
Important
Enterprise Edition only
To enable the Import/Export capability, please also add the JDBC connector for HyperSQLDB , taking care of using version 1.8.0.2 .
File system resources¶
Create the folder TOMCAT_HOME/resources
. Such a folder will contain some useful static resources and the indexes for the search engine used by Knowage.
Connection to metadata database¶
To define connection towards metadata database, edit the TOMCAT_HOME/conf/server.xml
and add the information related to the metadata database inside the GlobalNamingResources
tag. Specify: username, password, driver class name, JDBC URL and validation query (any valid query to be executed to validate connections). Connection’s name must the jdbc/knowage
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | <Resource auth="Container"
driverClassName="JDBC driver"
name="jdbc/knowage"
password="password"
type="javax.sql.DataSource"
url="JDBC URL"
username="username"
validationQuery="validation query"
maxTotal="50"
maxIdle="50"
minIdle="10"
validationInterval="34000"
removeAbandoned="true"
removeAbandonedTimeout="3600"
logAbandoned="true"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="10000"
minEvictableIdleTimeMillis="60000" />
|
Cache database connection¶
In some scenarios (for example when defining a cockpit document on top of a file dataset), Knowage requires a database to be used as cache. It is highly recommended to create an empty database schema for this purpose. Then, you need to configure it inside TOMCAT_HOME/conf/server.xml
as you did for metadata database. Feel free to type a name of your choice, in this example we used jdbc/ds_cache
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | <Resource auth="Container"
driverClassName="JDBC driver"
name="jdbc/ds_cache"
password="password"
type="javax.sql.DataSource"
url="JDBC URL"
username="user name"
validationQuery="validation query"
maxTotal="50"
maxIdle="50"
minIdle="10"
validationInterval="34000"
removeAbandoned="true"
removeAbandonedTimeout="3600"
logAbandoned="true"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="10000"
minEvictableIdleTimeMillis="60000" />
|
Connection to business data¶
Edit the TOMCAT_HOME/conf/server.xml
and add the information related to the database containing business data to be analysed by Knowage inside the GlobalNamingResources
tag, specifying username, password, driver class name, URL and validation query. Feel free to type a name of your choice, in this example we used jdbc/dwh
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | <Resource auth="Container"
driverClassName="JDBC driver"
name="jdbc/dwh"
password="password"
type="javax.sql.DataSource"
url="JDBC URL"
username="username"
validationQuery="validation query"
maxTotal="50"
maxIdle="50"
minIdle="10"
validationInterval="34000"
removeAbandoned="true"
removeAbandonedTimeout="3600"
logAbandoned="true"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="10000"
minEvictableIdleTimeMillis="60000"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" />
|
Environment variables definition¶
Edit the file TOMCAT_HOME/conf/server.xml
in Tomcat and add the following constants in the GlobalNamingResources
tag.
<Environment name="resource_path" type="java.lang.String" value="${catalina.home}/resources" />
<Environment name="sso_class" type="java.lang.String" value="it.eng.spagobi.services.common.JWTSsoService" />
<Environment name="service_url" type="java.lang.String" value="http://localhost:8080/knowage" />
<Environment name="hmacKey" description="HMAC key" type="java.lang.String" value="PUT ANY RANDOM STRING HERE" />
<Environment name="password_encryption_secret" description="File for security encryption location" type="java.lang.String" value="complete_file_path_with_file_name" />
Such environment variables have the following meaning:
resource_path
: resources folder path,sso_class
:SSO connector class name,service_url
:backend services address, typically set tohttp://localhost:8080/knowage
,hmacKey
: secret key to generate JWT tokens used by the default security mechanism. You must change it, and do not distribute it. You can put any random alphanumeric string in it, and you can change it everytime you want, you just need to restart Tomcat to apply the change,password_encryption_secret
: the complete path of a file to contain the password encryption secret. The file must contain random text of any length. This is a security configuration, so don’t use short strings. For example, you can create a file and write text into it. Do not distribute it for any reason, create at least a backup copy of the file. After the first start of Knowage, it will no longer be possible to change the secret key. In case you lost this secret, look at the paragraph below to see how to update the passwords of existing users.
Important
Again we stress the point that the HMAC key must be a random string. Please DO NOT copy and paste it from this documentation, since this will compromise the security of the application.
Below you can see an example of configuration of the above variables in the server.xml file
<Environment name="resource_path" type="java.lang.String" value="${catalina.home}/resources"/>
<Environment name="sso_class" type="java.lang.String" value="it.eng.spagobi.services.common.JWTSsoService"/>
<Environment name="service_url" type="java.lang.String" value="http://mydomain.com/knowage"/>
<Environment name="hmacKey" description="HMAC key" type="java.lang.String" value="a random string"/>
<Environment name="password_encryption_secret" description="File for security encryption location" type="java.lang.String" value="${catalina.home}/conf/knowage.secret"/>
Changing the secret key for password encryption¶
The password encryption secret key must be set during the installation and cannot be changed anymore, otherwise Knowage will no longer be able to authenticate already defined users. In case the secret key is lost you must create a new one, configure it into Knowage as described above and update passwords of existing users direclty into Knowage metadata database (SBI_USER table). For this reason Knowage provides you a tool to get new encrypted values.
This tool is a Java class that is shipped with the knowage-utils
library; it accepts 2 input parameters:
- the complete path of the password encryption secret file;
- the password value in plaintext.
Below is an example of invoking the tool by command line using ‘mypassword’ as the plaintext password to be encrypted (of course TOMCAT_HOME must be replaced by the actual Tomcat base folder path).
1 | java -cp "<TOMCAT_HOME>/webapps/knowage/WEB-INF/lib/knowage-utils-7.2.0.jar" it.eng.spagobi.security.utils.PasswordEncryptionToolMain <TOMCAT_HOME>/conf/knowage.secret mypassword
|
This procedure must be repeated for all already existing users.
Recommended configuration¶
Edit TOMCAT_HOME/conf/setenv.sh
(Linux) or TOMCAT_HOME/conf/setenv.bat
(Windows) file in Tomcat by adding the following JVM arguments:
1 2 3 4 5 6 7 8 9 10 11 | export JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=UTF-8"
# We add -Duser.timezone=UTC to solve error when establishing connection to Oracle metadata database:
# java.sql.SQLException: ORA-00604: error occurred at recursive SQL level 1
# ORA-01882: timezone region not found
export JAVA_OPTS="$JAVA_OPTS -Duser.timezone=UTC"
export JAVA_OPTS="$JAVA_OPTS -Djava.awt.headless=true"
export JAVA_OPTS="$JAVA_OPTS -Djava.security.manager -Djava.security.policy=$CATALINA_HOME/conf/catalina-relaxed.policy"
|
Applications deploy¶
To deploy Knowage you have to copy all the WAR files inside the TOMCAT_HOME/webapps
folder.
Once the first start is ended each WAR file will be unzipped. It is also possible to unzip the WAR files manually using the unzip utility.
Thread pool definition¶
You must configure TOMCAT_HOME/conf/server.xml
file and add the settings related to the pool of thread editing the GlobalNamingResources
tag, as shown follow.
1 | <Resource auth="Container" factory="de.myfoo.commonj.work.FooWorkManagerFactory" maxThreads="5" name="wm/SpagoWorkManager" type="commonj.work.WorkManager"/>
|
Advanced memory settings¶
It is recommended to increase the memory dimension used by the application server. This can be done by adjusting some properties. The memory required by each application server depends on many factors: number of users, type of analyses, amount of handled data, etc. The minimum requirements are Xms1024m
and Xmx2048m
.
[LINUX] Insert at the beginning of the TOMCAT_HOME/bin/setenv.sh
file this command:
1 | export JAVA_OPTS="$JAVA_OPTS -Xms1024m -Xmx2048m -XX:MaxPermSize=512m"
|
[WIN] Insert at the beginning of the TOMCAT_HOME/bin/setenv.bat
file this command:
1 | set JAVA_OPTS= %JAVA_OPTS% -Xms1024m Xmx2048m -XX:MaxPermSize=512m
|
Advanced Connector settings¶
Important
It is highly recommend to add URIEncoding=”UTF-8” attribute to server.xml file connector tags in order to avoid special characters issues.
1 | <Connector address="0.0.0.0" port="8009" protocol="AJP/1.3" maxPostSize="2097152000" redirectPort="8443" URIEncoding="UTF-8" />
|
Datasource link within the applications¶
You would set up ResourceLink for JNDI datasource. To do so, you have to configure each knowage*/META-INF/context.xml
and set the ResourceLink for each JNDI data source previously created. Inside the released packages two links are already defined:
- one for the
jdbc/knowage
resource, which the user must keep - the other for the
jdbc/foodmart
, which should be renamed withjdbc/dwh
.
1 2 3 4 5 6 7 8 9 10 11 | <Context docBase="knowage-ee" path="/knowage" reloadable="true">
<ResourceLink global="jdbc/dwh" name="jdbc/dwh" type="javax.sql.DataSource"/>
<ResourceLink global="jdbc/knowage" name="jdbc/knowage" type="javax.sql.DataSource"/>
<ResourceLink global="jdbc/ds_cache" name="jdbc/ds_cache" type="javax.sql.DataSource"/>
<ResourceLink global="resource_path" name="resource_path" type="java.lang.String" />
<ResourceLink global="sso_class" name="sso_class" type="java.lang.String" />
<ResourceLink name="hmacKey" global="hmacKey" type="java.lang.String"/>
<ResourceLink global="service_url" name="service_url" type="java.lang.String"/>
<ResourceLink global="wm/SpagoWorkManager" name="wm/SpagoWorkManager" type="commonj.work.WorkManager" />
<ResourceLink global="password_encryption_secret" name="password_encryption_secret" type="java.lang.String"/>
</Context>
|
Due to security reasons, configuration file containing some content to be used for encrypting must be created in Knowage environment. File must not be empty, but there’s no minimum length. File can be put everywhere; path and name must be properly configured for each knowage*/META-INF/context.xml
as shown below.
Important
Context update
The modification of these files will be effective as soon as the web application is reloaded or the application server is restarted.
Configuration of the metadata db dialect¶
Important
This step is not mandatory anymore
Knowage is now able to autonomously determine following configuration. That said, the following is not mandatory anymore however a user can force it to specific value.
Verify that the right dialect has been set inside hibernate.cfg.xml
files. We list all the possible dialects that can be used:
1 2 3 | <property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>,
<property name="hibernate.dialect">org.hibernate.dialect.PostgreSQLDialect</property>
<property name="hibernate.dialect">org.hibernate.dialect.Oracle9Dialect</property>
|
You have to configure these following Hibernate configuration files and set the chosen dialect:
1 2 3 4 5 6 7 8 | knowagekpiengine/WEB-INF/classes/hibernate.cfg.xml
knowagegeoreportengine/WEB-INF/classes/hibernate.cfg.xml
knowage/WEB-INF/classes/hsql/hibernate.cfg.xml
knowage/WEB-INF/classes/hibernate.cfg.xml
knowagesvgviewerengine/WEB-INF/classes/hibernate.cfg.xml
knowagemeta/WEB-INF/classes/hibernate.cfg.xml
knowagecockpitengine/WEB-INF/classes/hibernate.cfg.xml
knowagedataminingengine/WEB-INF/classes/hibernate.cfg.xml
|
Important
Context update
The modification of these files will be effective as soon as the web application is reloaded or the application server is restarted.
Modification of the Quartz configuration¶
Important
This step is not mandatory anymore
Knowage is now able to autonomously determine following configuration. That said, the following is not mandatory anymore however a user can force it to specific value.
The scheduler is configured in knowage/WEB-INF/classes/quartz.properties
. It is essential to enhance in this file the property org.quartz.jobStore.driverDelegateClass
with the right value, according to the metadata database in use. Following the possible values:
1 2 3 4 5 6 7 | # Hsqldb delegate class
#org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.HSQLDBDelegate
# Mysql delegate class org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
# Postgres delegate class
#org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
# Oracle delegate class
#org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.oracle.OracleDelegate
|
Clustering¶
When Knowage is installed in cluster with several nodes, it is necessary to activate the Cluster modality, adding these parameters to the knowage/WEB-INF/classes/quartz.properties
file of every involved machines:
1 2 3 4 | org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 20000
org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.instanceName = RHECMClusteredSchedule
|
Logging¶
It is necessary to set up a folder where Knowage and its analytical engines can store their respective log files. From now on, we will call LOG_DIR
such folder and LOG_DIR_PATH
the path that leads to it. This path is configured in file log4j.properties
located inside the \WEB-INF\classes\
available in each web application.
Shortly, to configure the Knowage log folder the user must execute the following steps:
- create the
LOG_DIR
folder on all cluster nodes on which it is intended to deploy Knowage Server and/or one of its analytical engines. TheLOG_DIR_PATH
string must be the same for every node; - [LINUX] verify that Knowage has write permissions on this folder; set the property
log4j.appender.knowage.File
inside theWEB-INF/classes/log4j.properties
toLOG_DIR_PATH/knowage.log
; - set the property
log4j.appender.knowageXXXXXEngine.File
inside theWEB-INF/classes/log4j.properties
file of each engine toLOG_DIR_PATH/knwoageXXXXXEngine.log
; - only for the Birt Engine, to set the property
logDirectory
inside theWEB-INF/classes/BirtLogConfig.properties
file of the knowagebirtreportengine application toLOG_DIR_PATH
.
Enable Java Security Manager¶
In Knowage, a user can create datasets, LOVs, etc.. with script languages like JavaScript. That introduces a security concern where a malicious user can execute code that can break the entire system. Java allows a system administrator to enable a Security Manager and to create a sandbox to limit privileges around the code that execute the script.
The Security Manager can be enabled with the following steps:
- Write a Security Policy for the Security Manager;
- Enable the Security Manager in the JVM.
The Security Policy is a text file read by a Security Manager that specifies all the privileges that a JVM can give to Java code: Tomcat has already a default policy in the file TOMCAT_HOME/conf/catalina.policy
but is too much strict for Knowage code that needs to write multiple logs, make network connection and execute external applications. Knowage is already secured and can use a more relaxed policy like:
grant {
permission java.security.AllPermission;
};
This policy can be saved to TOMCAT_HOME/conf/knowage-default.policy
.
To enable the Security Manager a system administrator have to add some options to the Java JVM:
[LINUX] Insert at the end of the TOMCAT_HOME/bin/setenv.sh
file this command:
1 | export JAVA_OPTS="$JAVA_OPTS -Djava.security.manager -Djava.security.policy=$CATALINA_HOME/conf/knowage-default.policy"
|
[WIN] Insert at the end of the TOMCAT_HOME/bin/setenv.bat
file this command:
1 | set JAVA_OPTS= %JAVA_OPTS% -Djava.security.manager -Djava.security.policy=%CATALINA_HOME%\conf\knowage-default.policy
|
Installation of Chromium Cockpit Export script¶
Important
Enterprise Edition only
Chromium Cockpit Export script is only available for Enterprise Edition.
Extract archive knowage-cockpit-export-installation.zip
to /opt/knowage
:
/opt/knowage/cockpit-export/cockpit-export.js
For alternatives path you have to fix internal.nodejs.chromium.export.path
in Knowage Configuration Management.