RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [0_1_10000000529 AS shardId#15507, worklistShardItemId#15608L, product#15594, plant#15589, availabilityDateTime#15596 AS periodStart#18473, nextAvailabilityDateTime#15597 AS periodEnd#18474, 0 AS version#18472, null AS qty#18475] +- Exchange hashpartitioning(worklistShardItemId#15608L, 37), REPARTITION_BY_NUM, [plan_id=322876] +- Project [plant#15589, product#15594, availabilityDateTime#15596, nextAvailabilityDateTime#15597, worklistShardItemId#15608L] +- Filter ((isnotnull(availabilityDateTime#15596) AND isnotnull(nextAvailabilityDateTime#15597)) AND (nextAvailabilityDateTime#15597 > availabilityDateTime#15596)) +- FileScan parquet [plant#15589,product#15594,availabilityDateTime#15596,nextAvailabilityDateTime#15597,worklistShardItemId#15608L,shardId#15629,sequenceId#15607] Batched: true, DataFilters: [isnotnull(availabilityDateTime#15596), isnotnull(nextAvailabilityDateTime#15597), (nextAvailabil..., Format: Parquet, Location: Pr...

  • Storage Level: Disk Memory Serialized 2x Replicated
  • Cached Partitions: 31
  • Total Partitions: 37
  • Memory Size: 7.0 MiB
  • Disk Size: 113.4 KiB

Data Distribution on 3 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
10.96.20.101:7079 2.9 MiB (267.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.25.83:7079 2.1 MiB (1848.6 MiB Remaining) 0.0 B (0.0 B Remaining) 113.4 KiB
10.96.13.76:7079 2.1 MiB (1156.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

31 Partitions

Page:
Block Name Storage Level ▴ Size in Memory Size on Disk Executors
rdd_63417_27 Disk Memory Serialized 2x Replicated 113.4 KiB 113.4 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_0 Memory Serialized 2x Replicated 235.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_3 Memory Serialized 2x Replicated 231.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_4 Memory Serialized 2x Replicated 228.8 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63417_2 Memory Serialized 2x Replicated 235.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_5 Memory Serialized 2x Replicated 236.7 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_6 Memory Serialized 2x Replicated 235.6 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63417_7 Memory Serialized 2x Replicated 240.1 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63417_8 Memory Serialized 2x Replicated 234.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_9 Memory Serialized 2x Replicated 236.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_10 Memory Serialized 2x Replicated 236.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_11 Memory Serialized 2x Replicated 234.5 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63417_13 Memory Serialized 2x Replicated 234.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_14 Memory Serialized 2x Replicated 227.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_15 Memory Serialized 2x Replicated 235.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_17 Memory Serialized 2x Replicated 231.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_20 Memory Serialized 2x Replicated 232.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_19 Memory Serialized 2x Replicated 228.8 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63417_21 Memory Serialized 2x Replicated 225.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_22 Memory Serialized 2x Replicated 231.6 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_23 Memory Serialized 2x Replicated 240.4 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_25 Memory Serialized 2x Replicated 233.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_26 Memory Serialized 2x Replicated 239.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_28 Memory Serialized 2x Replicated 230.8 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63417_29 Memory Serialized 2x Replicated 231.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_30 Memory Serialized 2x Replicated 229.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_34 Memory Serialized 2x Replicated 234.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_32 Memory Serialized 2x Replicated 233.4 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_33 Memory Serialized 2x Replicated 238.1 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63417_35 Memory Serialized 2x Replicated 234.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63417_36 Memory Serialized 2x Replicated 234.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
Page: