Apache Zeppelin Configuration
Zeppelin Properties
Zeppelin can be configured via several sources.
Sources descending by priority:
- environment variables can be defined conf/zeppelin-env.sh
(conf\zeppelin-env.cmd
for Windows).
- system properties
- configuration file can be defined in conf/zeppelin-site.xml
Mouse hover on each property and click then you can get a link for that.
zeppelin-env.sh | zeppelin-site.xml | Default value | Description |
---|---|---|---|
ZEPPELIN_ADDR |
zeppelin.server.addr |
127.0.0.1 | Zeppelin server binding address |
ZEPPELIN_PORT |
zeppelin.server.port |
8080 | Zeppelin server port Note: Please make sure you're not using the same port with Zeppelin web application development port (default: 9000). |
ZEPPELIN_SSL_PORT |
zeppelin.server.ssl.port |
8443 | Zeppelin Server ssl port (used when ssl environment/property is set to true) |
ZEPPELIN_JMX_ENABLE |
zeppelin.jmx.enable |
false | Enable JMX by defining "true" |
ZEPPELIN_JMX_PORT |
zeppelin.jmx.port |
9996 | Port number which JMX uses |
ZEPPELIN_MEM |
N/A | -Xmx1024m -XX:MaxMetaspaceSize=512m | JVM mem options |
ZEPPELIN_INTP_MEM |
N/A | ZEPPELIN_MEM | JVM mem options for interpreter process |
ZEPPELIN_JAVA_OPTS |
N/A | JVM options | |
ZEPPELIN_ALLOWED_ORIGINS |
zeppelin.server.allowed.origins |
* | Enables a way to specify a ',' separated list of allowed origins for REST and websockets. e.g. http://localhost:8080 |
ZEPPELIN_CREDENTIALS_PERSIST |
zeppelin.credentials.persist |
true | Persist credentials on a JSON file (credentials.json) |
ZEPPELIN_CREDENTIALS_ENCRYPT_KEY |
zeppelin.credentials.encryptKey |
If provided, encrypt passwords on the credentials.json file (passwords will be stored as plain-text otherwise | |
ZEPPELIN_SERVER_CONTEXT_PATH |
zeppelin.server.context.path |
/ | Context path of the web application |
ZEPPELIN_NOTEBOOK_COLLABORATIVE_MODE_ENABLE |
zeppelin.notebook.collaborative.mode.enable |
true | Enable basic opportunity for collaborative editing. Does not change the logic of operation if the note is used by one person. |
ZEPPELIN_SSL |
zeppelin.ssl |
false | |
ZEPPELIN_SSL_CLIENT_AUTH |
zeppelin.ssl.client.auth |
false | |
ZEPPELIN_SSL_KEYSTORE_PATH |
zeppelin.ssl.keystore.path |
keystore | |
ZEPPELIN_SSL_KEYSTORE_TYPE |
zeppelin.ssl.keystore.type |
JKS | |
ZEPPELIN_SSL_KEYSTORE_PASSWORD |
zeppelin.ssl.keystore.password |
||
ZEPPELIN_SSL_KEY_MANAGER_PASSWORD |
zeppelin.ssl.key.manager.password |
||
ZEPPELIN_SSL_TRUSTSTORE_PATH |
zeppelin.ssl.truststore.path |
||
ZEPPELIN_SSL_TRUSTSTORE_TYPE |
zeppelin.ssl.truststore.type |
||
ZEPPELIN_SSL_TRUSTSTORE_PASSWORD |
zeppelin.ssl.truststore.password |
||
ZEPPELIN_SSL_PEM_KEY |
zeppelin.ssl.pem.key |
This directive points to the PEM-encoded private key file for the server. | |
ZEPPELIN_SSL_PEM_KEY_PASSWORD |
zeppelin.ssl.pem.key.password |
Password of the PEM-encoded private key. | |
ZEPPELIN_SSL_PEM_CERT |
zeppelin.ssl.pem.cert |
This directive points to a file with certificate data in PEM format. | |
ZEPPELIN_SSL_PEM_CA |
zeppelin.ssl.pem.ca |
This directive sets the all-in-one file where you can assemble the Certificates of Certification Authorities (CA) whose clients you deal with. These are used for Client Authentication. Such a file is simply the concatenation of the various PEM-encoded Certificate files. | |
ZEPPELIN_NOTEBOOK_HOMESCREEN |
zeppelin.notebook.homescreen |
Display note IDs on the Apache Zeppelin homescreen e.g. 2A94M5J1Z |
|
ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE |
zeppelin.notebook.homescreen.hide |
false | Hide the note ID set by ZEPPELIN_NOTEBOOK_HOMESCREEN on the Apache Zeppelin homescreen. For the further information, please read Customize your Zeppelin homepage. |
ZEPPELIN_WAR_TEMPDIR |
zeppelin.war.tempdir |
webapps | Location of the jetty temporary directory |
ZEPPELIN_NOTEBOOK_DIR |
zeppelin.notebook.dir |
notebook | The root directory where notebook directories are saved |
ZEPPELIN_NOTEBOOK_S3_BUCKET |
zeppelin.notebook.s3.bucket |
zeppelin | S3 Bucket where notebook files will be saved |
ZEPPELIN_NOTEBOOK_S3_USER |
zeppelin.notebook.s3.user |
user | User name of an S3 bucket e.g. bucket/user/notebook/2A94M5J1Z/note.json |
ZEPPELIN_NOTEBOOK_S3_ENDPOINT |
zeppelin.notebook.s3.endpoint |
s3.amazonaws.com | Endpoint for the bucket |
N/A | zeppelin.notebook.s3.timeout |
120000 | Bucket endpoint request timeout in msec |
ZEPPELIN_NOTEBOOK_S3_KMS_KEY_ID |
zeppelin.notebook.s3.kmsKeyID |
AWS KMS Key ID to use for encrypting data in S3 (optional) | |
ZEPPELIN_NOTEBOOK_S3_EMP |
zeppelin.notebook.s3.encryptionMaterialsProvider |
Class name of a custom S3 encryption materials provider implementation to use for encrypting data in S3 (optional) | |
ZEPPELIN_NOTEBOOK_S3_SSE |
zeppelin.notebook.s3.sse |
false | Save notebooks to S3 with server-side encryption enabled |
ZEPPELIN_NOTEBOOK_S3_CANNED_ACL |
zeppelin.notebook.s3.cannedAcl |
Save notebooks to S3 with the given [Canned ACL](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/CannedAccessControlList.html) which determines the S3 permissions. | |
ZEPPELIN_NOTEBOOK_S3_PATH_STYLE_ACCESS |
zeppelin.notebook.s3.pathStyleAccess |
false | Access S3 bucket using path style |
ZEPPELIN_NOTEBOOK_S3_SIGNEROVERRIDE |
zeppelin.notebook.s3.signerOverride |
Optional override to control which signature algorithm should be used to sign AWS requests | |
ZEPPELIN_NOTEBOOK_AZURE_CONNECTION_STRING |
zeppelin.notebook.azure.connectionString |
The Azure storage account connection string e.g. DefaultEndpointsProtocol=https; |
|
ZEPPELIN_NOTEBOOK_AZURE_SHARE |
zeppelin.notebook.azure.share |
zeppelin | Azure Share where the notebook files will be saved |
ZEPPELIN_NOTEBOOK_AZURE_USER |
zeppelin.notebook.azure.user |
user | Optional user name of an Azure file share e.g. share/user/notebook/2A94M5J1Z/note.json |
ZEPPELIN_NOTEBOOK_STORAGE |
zeppelin.notebook.storage |
org.apache.zeppelin.notebook.repo.GitNotebookRepo | Comma separated list of notebook storage locations |
ZEPPELIN_NOTEBOOK_ONE_WAY_SYNC |
zeppelin.notebook.one.way.sync |
false | If there are multiple notebook storage locations, should we treat the first one as the only source of truth? |
ZEPPELIN_NOTEBOOK_PUBLIC |
zeppelin.notebook.public |
true | Make notebook public (set only owners ) by default when created/imported. If set to false will add user to readers and writers as well, making it private and invisible to other users unless permissions are granted. |
ZEPPELIN_INTERPRETER_DIR |
zeppelin.interpreter.dir |
interpreter | Interpreter directory |
ZEPPELIN_INTERPRETER_DEP_MVNREPO |
zeppelin.interpreter.dep.mvnRepo |
https://repo1.maven.org/maven2/,https://repo2.maven.org/maven2/ | Remote principal repository for interpreter's additional dependency loading |
ZEPPELIN_INTERPRETER_OUTPUT_LIMIT |
zeppelin.interpreter.output.limit |
102400 | Output message from interpreter exceeding the limit will be truncated |
ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT |
zeppelin.interpreter.connect.timeout |
600s | Interpreter process connect timeout. Default time unit is msec |
ZEPPELIN_DEP_LOCALREPO |
zeppelin.dep.localrepo |
local-repo | Local repository for dependency loader. ex)visualiztion modules of npm. |
ZEPPELIN_HELIUM_NODE_INSTALLER_URL |
zeppelin.helium.node.installer.url |
https://nodejs.org/dist/ | Remote Node installer url for Helium dependency loader |
ZEPPELIN_HELIUM_NPM_INSTALLER_URL |
zeppelin.helium.npm.installer.url |
http://registry.npmjs.org/ | Remote Npm installer url for Helium dependency loader |
ZEPPELIN_HELIUM_YARNPKG_INSTALLER_URL |
zeppelin.helium.yarnpkg.installer.url |
https://github.com/yarnpkg/yarn/releases/download/ | Remote Yarn package installer url for Helium dependency loader |
ZEPPELIN_WEBSOCKET_MAX_TEXT_MESSAGE_SIZE |
zeppelin.websocket.max.text.message.size |
1024000 | Size(in characters) of the maximum text message that can be received by websocket. |
ZEPPELIN_SERVER_DEFAULT_DIR_ALLOWED |
zeppelin.server.default.dir.allowed |
false | Enable directory listings on server. |
ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL |
zeppelin.notebook.git.remote.url |
GitHub's repository URL. It could be either the HTTP URL or the SSH URL. For example git@github.com:apache/zeppelin.git | |
ZEPPELIN_NOTEBOOK_GIT_REMOTE_USERNAME |
zeppelin.notebook.git.remote.username |
token | GitHub username. By default it is `token` to use GitHub's API |
ZEPPELIN_NOTEBOOK_GIT_REMOTE_ACCESS_TOKEN |
zeppelin.notebook.git.remote.access-token |
token | GitHub access token to use GitHub's API. If username/password combination is used and not GitHub API, then this value is the password |
ZEPPELIN_NOTEBOOK_GIT_REMOTE_ORIGIN |
zeppelin.notebook.git.remote.origin |
token | GitHub remote name. Default is `origin` |
ZEPPELIN_RUN_MODE |
zeppelin.run.mode |
auto | Run mode. 'auto|local|k8s'. 'auto' autodetect environment. 'local' runs interpreter as a local process. k8s runs interpreter on Kubernetes cluster |
ZEPPELIN_K8S_PORTFORWARD |
zeppelin.k8s.portforward |
false | Port forward to interpreter rpc port. Set 'true' only on local development when zeppelin.k8s.mode 'on'. Don't use 'true' on production environment |
ZEPPELIN_K8S_CONTAINER_IMAGE |
zeppelin.k8s.container.image |
apache/zeppelin:0.11.1 | Docker image for interpreters |
ZEPPELIN_K8S_SPARK_CONTAINER_IMAGE |
zeppelin.k8s.spark.container.image |
apache/spark:latest | Docker image for Spark executors |
ZEPPELIN_K8S_TEMPLATE_DIR |
zeppelin.k8s.template.dir |
k8s | Kubernetes yaml spec files |
ZEPPELIN_K8S_SERVICE_NAME |
zeppelin.k8s.service.name |
zeppelin-server | Name of the Zeppelin server service resources |
ZEPPELIN_K8S_TIMEOUT_DURING_PENDING |
zeppelin.k8s.timeout.during.pending |
true | Value to enable/disable timeout handling when starting Interpreter Pods. Caution: This can lead to an infinity loop |
ZEPPELIN_METRIC_ENABLE_PROMETHEUS |
zeppelin.metric.enable.prometheus |
false | Value to enable/disable Prometheus metric endpoint on /metric |
ZEPPELIN_NOTEBOOK_CRON_ENABLE |
zeppelin.notebook.cron.enable |
false | Value to enable/disable Cron support in Notes |
ZEPPELIN_NOTEBOOK_CRON_FOLDERS |
zeppelin.notebook.cron.folders |
comma-separated list of folder, where cron is allowed | |
ZEPPELIN_NOTE_CACHE_THRESHOLD |
zeppelin.note.cache.threshold |
50 | Threshold for the number of notes in the cache before an eviction occurs. |
ZEPPELIN_NOTEBOOK_VERSIONED_MODE_ENABLE |
zeppelin.notebook.versioned.mode.enable |
true | Value to enable/disable version control support in Notes. |
SSL Configuration
Enabling SSL requires a few configuration changes. First, you need to create certificates and then update necessary configurations to enable server side SSL and/or client side certificate authentication.
Creating and configuring the Certificates
Information how about to generate certificates and a keystore can be found here.
A condensed example can be found in the top answer to this StackOverflow post.
The keystore holds the private key and certificate on the server end. The trustore holds the trusted client certificates. Be sure that the path and password for these two stores are correctly configured in the password fields below. They can be obfuscated using the Jetty password tool. After Maven pulls in all the dependency to build Zeppelin, one of the Jetty jars contain the Password tool. Invoke this command from the Zeppelin home build directory with the appropriate version, user, and password.
java -cp ./zeppelin-server/target/lib/jetty-all-server-<version>.jar \
org.eclipse.jetty.util.security.Password <user> <password>
If you are using a self-signed, a certificate signed by an untrusted CA, or if client authentication is enabled, then the client must have a browser create exceptions for both the normal HTTPS port and WebSocket port. This can by done by trying to establish an HTTPS connection to both ports in a browser (e.g. if the ports are 443 and 8443, then visit https://127.0.0.1:443 and https://127.0.0.1:8443). This step can be skipped if the server certificate is signed by a trusted CA and client auth is disabled.
Configuring server side SSL
The following properties needs to be updated in the zeppelin-site.xml
in order to enable server side SSL.
<property>
<name>zeppelin.server.ssl.port</name>
<value>8443</value>
<description>Server ssl port. (used when ssl property is set to true)</description>
</property>
<property>
<name>zeppelin.ssl</name>
<value>true</value>
<description>Should SSL be used by the servers?</description>
</property>
<property>
<name>zeppelin.ssl.keystore.path</name>
<value>keystore</value>
<description>Path to keystore relative to Zeppelin configuration directory</description>
</property>
<property>
<name>zeppelin.ssl.keystore.type</name>
<value>JKS</value>
<description>The format of the given keystore (e.g. JKS or PKCS12)</description>
</property>
<property>
<name>zeppelin.ssl.keystore.password</name>
<value>change me</value>
<description>Keystore password. Can be obfuscated by the Jetty Password tool</description>
</property>
<property>
<name>zeppelin.ssl.key.manager.password</name>
<value>change me</value>
<description>Key Manager password. Defaults to keystore password. Can be obfuscated.</description>
</property>
Enabling client side certificate authentication
The following properties needs to be updated in the zeppelin-site.xml
in order to enable client side certificate authentication.
<property>
<name>zeppelin.server.ssl.port</name>
<value>8443</value>
<description>Server ssl port. (used when ssl property is set to true)</description>
</property>
<property>
<name>zeppelin.ssl.client.auth</name>
<value>true</value>
<description>Should client authentication be used for SSL connections?</description>
</property>
<property>
<name>zeppelin.ssl.truststore.path</name>
<value>truststore</value>
<description>Path to truststore relative to Zeppelin configuration directory. Defaults to the keystore path</description>
</property>
<property>
<name>zeppelin.ssl.truststore.type</name>
<value>JKS</value>
<description>The format of the given truststore (e.g. JKS or PKCS12). Defaults to the same type as the keystore type</description>
</property>
<property>
<name>zeppelin.ssl.truststore.password</name>
<value>change me</value>
<description>Truststore password. Can be obfuscated by the Jetty Password tool. Defaults to the keystore password</description>
</property>
Storing user credentials
In order to avoid having to re-enter credentials every time you restart/redeploy Zeppelin, you can store the user credentials. Zeppelin supports this via the ZEPPELINCREDENTIALSPERSIST configuration.
Please notice that passwords will be stored in plain text by default. To encrypt the passwords, use the ZEPPELINCREDENTIALSENCRYPT_KEY config variable. This will encrypt passwords using the AES-128 algorithm.
You can generate an appropriate encryption key any way you'd like - for instance, by using the openssl tool:
openssl enc -aes-128-cbc -k secret -P -md sha1
Important: storing your encryption key in a configuration file is not advised. Depending on your environment security needs, you may want to consider utilizing a credentials server, storing the ZEPPELINCREDENTIALSENCRYPT_KEY as an OS env variable, or any other approach that would not colocate the encryption key and the encrypted content (the credentials.json file).
Obfuscating Passwords using the Jetty Password Tool
Security best practices advise to not use plain text passwords and Jetty provides a password tool to help obfuscating the passwords used to access the KeyStore and TrustStore.
The Password tool documentation can be found here.
After using the tool:
java -cp $ZEPPELIN_HOME/zeppelin-server/target/lib/jetty-util-9.2.15.v20160210.jar \
org.eclipse.jetty.util.security.Password \
password
2016-12-15 10:46:47.931:INFO::main: Logging initialized @101ms
password
OBF:1v2j1uum1xtv1zej1zer1xtn1uvk1v1v
MD5:5f4dcc3b5aa765d61d8327deb882cf99
update your configuration with the obfuscated password :
<property>
<name>zeppelin.ssl.keystore.password</name>
<value>OBF:1v2j1uum1xtv1zej1zer1xtn1uvk1v1v</value>
<description>Keystore password. Can be obfuscated by the Jetty Password tool</description>
</property>
Create GitHub Access Token
When using GitHub to track notebooks, one can use GitHub's API for authentication. To create an access token, please use the following link https://github.com/settings/tokens.
The value of the access token generated is set in the zeppelin.notebook.git.remote.access-token
property.
Note: After updating these configurations, Zeppelin server needs to be restarted.