Shell interpreter for Apache Zeppelin

Overview

Shell interpreter uses Apache Commons Exec to execute external processes. In Zeppelin notebook, you can use %sh in the beginning of a paragraph to invoke system shell and run commands.

Note : Currently each command runs as the user Zeppelin server is running as.

Configuration

At the "Interpreters" menu in Zeppelin dropdown menu, you can set the property value for Shell interpreter.

Example

The following example demonstrates the basic usage of Shell in a Zeppelin notebook.

If you need further information about Zeppelin Interpreter Setting for using Shell interpreter, please read What is interpreter setting? section first.

Kerberos refresh interval

For changing the default behavior of when to renew Kerberos ticket following changes can be made in conf/zeppelin-env.sh.

# Change Kerberos refresh interval (default value is 1d). Allowed postfix are ms, s, m, min, h, and d.
export LAUNCH_KERBEROS_REFRESH_INTERVAL=4h
# Change kinit number retries (default value is 5), which means if the kinit command fails for 5 retries consecutively it will close the interpreter. 
export KINIT_FAIL_THRESHOLD=10

Object Interpolation

The shell interpreter also supports interpolation of ZeppelinContext objects into the paragraph text. The following example shows one use of this facility:

In Scala cell:

z.put("dataFileName", "members-list-003.parquet")
    // ...
val members = spark.read.parquet(z.get("dataFileName"))
    // ...

In later Shell cell:

%sh
rm -rf {dataFileName}

Object interpolation is disabled by default, and can be enabled (for the Shell interpreter) by setting the value of the property zeppelin.shell.interpolation to true (see Configuration above). More details of this feature can be found in Zeppelin-Context