|
- Apache Livy
What is Apache Livy? Apache Livy is a service that enables easy interaction with a Spark cluster over a REST interface It enables easy submission of Spark jobs or snippets of Spark code, synchronous or asynchronous result retrieval, as well as Spark Context management, all via a simple REST interface or an RPC client library
- Livy - Getting Started - The Apache Software Foundation
1 Install Livy Download Livy packages from here 2 Run Livy To run the Livy server, you will also need an Apache Spark installation You can get Spark releases at https: spark apache org downloads html Livy requires at least Spark 1 6 and supports Scala 2 11 and 2 12 builds of Spark To run Livy with local sessions, first export these
- Apache Livy Examples - The Apache Software Foundation
Here’s a step-by-step example of interacting with Livy in Python with the Requests library By default Livy runs on port 8998 (which can be changed with the livy server port config option) We’ll start off with a Spark session that takes Scala code:
- Livy Docs - REST API - The Apache Software Foundation
If superuser support is configured, Livy supports the doAs query parameter to specify the user to impersonate The doAs query parameter can be used on any supported REST endpoint described above to perform the action as the specified user
- Livy Docs - The Apache Software Foundation
Apache Livy is an effort undergoing Incubation at The Apache Software Foundation (ASF), sponsored by the Incubator Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects
- Livy Docs - Programmatic API - The Apache Software Foundation
Livy provides a programmatic Java Scala and Python API that allows applications to run code inside Spark without having to maintain a local Spark context Here shows how to use the Java API Add the Livy client dependency to your application’s POM:
- Livy - History - The Apache Software Foundation
Livy 0 7 0 now requires Java 8, Scala 2 11 and Spark >= 2 2 0 With 0 7 0, JDBC ODBC feature now becomes GA Added support for all current versions of Spark (2 2 x to 2 4 x)
- Livy 0. 8. 0-incubating JavaDoc - livy. apache. org
Package org apache livy Livy programmatic Java API See: Description Interface Summary ; Interface Description; Job<T> Interface for a Spark remote job JobContext: Holds runtime information about the job execution context JobHandle<T> A handle to a submitted job JobHandle Listener<T>
|
|
|