RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [0_3_10000000530 AS shardId#597175, worklistShardItemId#597315L, product#597301, plant#597296, availabilityDateTime#597303 AS periodStart#601913, nextAvailabilityDateTime#597304 AS periodEnd#601914, 0 AS version#601912, null AS qty#601915] +- Exchange hashpartitioning(worklistShardItemId#597315L, 37), REPARTITION_BY_NUM, [plan_id=357724] +- Project [plant#597296, product#597301, availabilityDateTime#597303, nextAvailabilityDateTime#597304, worklistShardItemId#597315L] +- Filter ((isnotnull(availabilityDateTime#597303) AND isnotnull(nextAvailabilityDateTime#597304)) AND (nextAvailabilityDateTime#597304 > availabilityDateTime#597303)) +- FileScan parquet [plant#597296,product#597301,availabilityDateTime#597303,nextAvailabilityDateTime#597304,worklistShardItemId#597315L,shardId#597336,sequenceId#597314] Batched: true, DataFilters: [isnotnull(availabilityDateTime#597303), isnotnull(nextAvailabilityDateTime#597304), (nextAvailab..., Fo...

  • Storage Level: Disk Memory Serialized 2x Replicated
  • Cached Partitions: 18
  • Total Partitions: 37
  • Memory Size: 4.7 MiB
  • Disk Size: 0.0 B

Data Distribution on 3 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
10.96.20.101:7079 1746.3 KiB (267.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.25.83:7079 1212.2 KiB (1848.6 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.13.76:7079 1878.6 KiB (1156.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

18 Partitions

Page:
Block Name ▾ Storage Level Size in Memory Size on Disk Executors
rdd_61060_8 Memory Serialized 2x Replicated 269.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61060_6 Memory Serialized 2x Replicated 266.4 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_61060_5 Memory Serialized 2x Replicated 262.2 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_4 Memory Serialized 2x Replicated 273.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_33 Memory Serialized 2x Replicated 272.3 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61060_31 Memory Serialized 2x Replicated 266.6 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61060_26 Memory Serialized 2x Replicated 262.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_25 Memory Serialized 2x Replicated 279.1 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_61060_23 Memory Serialized 2x Replicated 270.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_22 Memory Serialized 2x Replicated 267.2 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_2 Memory Serialized 2x Replicated 261.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_19 Memory Serialized 2x Replicated 267.3 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_61060_18 Memory Serialized 2x Replicated 266.2 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_61060_17 Memory Serialized 2x Replicated 265.3 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_61060_16 Memory Serialized 2x Replicated 272.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_15 Memory Serialized 2x Replicated 270.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_14 Memory Serialized 2x Replicated 273.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61060_1 Memory Serialized 2x Replicated 271.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
Page: