Geode/Gemfire OQL Interpreter for Apache Zeppelin

This interpreter supports the Geode Object Query Language (OQL). With the OQL-based querying language:

• You can query on any arbitrary object
• You can navigate object collections
• You can invoke methods and access the behavior of objects
• Data mapping is supported
• You are not required to declare types. Since you do not need type definitions, you can work across multiple languages
• You are not constrained by a schema

This Video Tutorial illustrates some of the features provided by the Geode Interpreter.

Create Interpreter

By default Zeppelin creates one Geode/OQL instance. You can remove it or create more instances.

Multiple Geode instances can be created, each configured to the same or different backend Geode cluster. But over time a Notebook can have only one Geode interpreter instance bound. That means you can not connect to different Geode clusters in the same Notebook. This is a known Zeppelin limitation.

To create new Geode instance open the Interprter section and click the +Create button. Pick a Name of your choice and from the Interpreter drop-down select geode. Then follow the configuration instructions and Save the new instance.

Note: The Name of the instance is used only to distinct the instances while binding them to the Notebook. The Name is irrelevant inside the Notebook. In the Notebook you must use %geode.oql tag.

Bind to Notebook

In the Notebook click on the settings icon in the top right corner. The select/deselect the interpreters to be bound with the Notebook.

Configuration

You can modify the configuration of the Geode from the Interpreter section. The Geode interpreter express the following properties:

How to use

Tip 1: Use (CTRL + .) for OQL auto-completion.

Tip 2: Alawys start the paragraphs with the full %geode.oql prefix tag! The short notation: %geode would still be able run the OQL queries but the syntax highlighting and the auto-completions will be disabled.

Create / Destroy Regions

The OQL sepecification does not support Geode Regions mutation operations. To creaate/destroy regions one should use the GFSH shell tool instead. To wokr this it assumes that the GFSH is colocated with Zeppelin server.

%sh
source /etc/geode/conf/geode-env.sh
gfsh << EOF

 connect --locator=ambari.localdomain[10334]

 destroy region --name=/regionEmployee
 destroy region --name=/regionCompany
 create region --name=regionEmployee --type=REPLICATE
 create region --name=regionCompany --type=REPLICATE

 exit;
EOF

Above snippet re-creates two regions: regionEmployee and regionCompany. Note that you have to explicetely specify the locator host and port. The values should match those you have used in the Geode Interpreter configuration. Comprehensive list of GFSH Commands by Functional Area.

Basic OQL

%geode.oql 
SELECT count(*) FROM /regionEmploee

OQL IN and SET filters:

%geode.oql
SELECT * FROM /regionEmployee 
WHERE companyId IN SET(2) OR lastName IN SET('Tzolov13', 'Tzolov73')

OQL JOIN operations

%geode.oql
SELECT e.employeeId, e.firstName, e.lastName, c.id as companyId, c.companyName, c.address
FROM /regionEmployee e, /regionCompany c 
WHERE e.companyId = c.id

By default the QOL responses contain only the region entry values. To access the keys, query the EntrySet instead:

%geode.oql
SELECT e.key, e.value.companyId, e.value.email 
FROM /regionEmployee.entrySet e

Following query will return the EntrySet value as a Blob:

%geode.oql
SELECT e.key, e.value FROM /regionEmployee.entrySet e

Note: You can have multiple queries in the same paragraph but only the result from the first is displayed. [1], [2].

GFSH Commands From The Shell

Use the Shell Interpreter (%sh) to run OQL commands form the command line:

%sh
source /etc/geode/conf/geode-env.sh
gfsh -e "connect" -e "list members"

Apply Zeppelin Dynamic Forms

You can leverage Zepplein Dynamic Form inside your OQL queries. You can use both the text input and select form parametrization features

%geode.oql
SELECT * FROM /regionEmployee e WHERE e.employeeId > ${Id}

Geode REST API

To list the defined regions you can use the Geode REST API:

http://<geode server hostname>phd1.localdomain:8484/gemfire-api/v1/

{
  "regions" : [{
    "name" : "regionEmployee",
    "type" : "REPLICATE",
    "key-constraint" : null,
    "value-constraint" : null
  }, {
    "name" : "regionCompany",
    "type" : "REPLICATE",
    "key-constraint" : null,
    "value-constraint" : null
  }]
}

To enable Geode REST API with JSON support add the following properties to geode.server.properties.file and restart:

http-service-port=8484
start-dev-rest-api=true

Auto-completion

The Geode Interpreter provides a basic auto-completion functionality. On (Ctrl+.) it list the most relevant suggesntions in a pop-up window.