Skip to main content

PySpark

  1. Turn on OTF.decryption.configuration in HDFSIDPConfig.properties.

  2. Run a task from PK Protect UI. (Here verify /tmp/dataguise folder been created in HDFS).

  3. Run the pyspark shell with the following commands:
    pyspark --jars /home/Hithen/DgDecrypter.jar,/home/Hithen/gson-2.2.2.jar,/home/Hithen/guava-13.0.1.jar,DgDEcrypter.jar

  4. Create a permanent hive function using the following command:
    sqlContext.sql("create function dgSpark AS 'com.dataguise.decrypterlib.hive.udf.DgDecrypter6' using jar 'hdfs:///tmp/dataguise/DgDecrypter.jar'")

  5. This is a one-time set up, you can also use existing functions created through Hive/Beeline.

  6. Run queries.

  7. Results

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.