使用 Snowpark Scala 时分析查询和排除故障

本主题提供了一些有关在使用 Snowpark 库时分析查询和故障排除的指引。

在 Snowpark 中查看查询的执行计划

To inspect the evaluation plan of a DataFrame, call the explain method of the DataFrame. This prints the SQL statements used to evaluate the DataFrame. If there is only one SQL statement, the method also prints the logical plan for the statement.

----------DATAFRAME EXECUTION PLAN----------
Query List:
0.
SELECT
  "_1" AS "col %",
  "_2" AS "col *"
FROM
  (
    SELECT
      *
    FROM
      (
        VALUES
          (1 :: int, 2 :: int),
          (3 :: int, 4 :: int) AS SN_TEMP_OBJECT_639016133("_1", "_2")
      )
  )
Logical Execution Plan:
 GlobalStats:
    partitionsTotal=0
    partitionsAssigned=0
    bytesAssigned=0
Operations:
1:0     ->Result  SN_TEMP_OBJECT_639016133.COLUMN1, SN_TEMP_OBJECT_639016133.COLUMN2
1:1          ->ValuesClause  (1, 2), (3, 4)

--------------------------------------------

After the execution of a DataFrame has been triggered, you can check on the progress of the query in the Query History page in Snowsight.

In the Query Tag column, you can find the name of the function and the line number in your code that triggered this query.

Snowpark request in the History page

故障排除

更改日志记录设置

By default, the Snowpark library logs INFO level messages to stdout. To change the logging settings, create a simplelogger.properties file, and configure the logger properties in that file. For example, to set the log level to DEBUG:

# simplelogger.properties file (a text file)
# Set the default log level for the SimpleLogger to DEBUG.
org.slf4j.simpleLogger.defaultLogLevel=debug

Put this file in your classpath. If you are using a Maven directory layout, put the file in the src/main/resources/ directory.

java.lang. OutOfMemoryError 异常

If a java.lang.OutOfMemoryError exception is thrown, increase the maximum heap size for the JVM.

If you are using the Scala REPL and you need to increase the maximum heap size, edit the run.sh shell script (provided in the archive file) and add the -J-Xmxmaximum_size flag to the scala command. The following example increases the maximum heap size to 4 GB:

scala -J-Xmx4G ...