RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [0_4_10000000530 AS shardId#619080, worklistShardItemId#619307L, product#619293, plant#619288, availabilityDateTime#619295 AS periodStart#623054, nextAvailabilityDateTime#619296 AS periodEnd#623055, 0 AS version#623053, null AS qty#623056] +- Exchange hashpartitioning(worklistShardItemId#619307L, 37), REPARTITION_BY_NUM, [plan_id=357448] +- Project [plant#619288, product#619293, availabilityDateTime#619295, nextAvailabilityDateTime#619296, worklistShardItemId#619307L] +- Filter ((isnotnull(availabilityDateTime#619295) AND isnotnull(nextAvailabilityDateTime#619296)) AND (nextAvailabilityDateTime#619296 > availabilityDateTime#619295)) +- FileScan parquet [plant#619288,product#619293,availabilityDateTime#619295,nextAvailabilityDateTime#619296,worklistShardItemId#619307L,shardId#619328,sequenceId#619306] Batched: true, DataFilters: [isnotnull(availabilityDateTime#619295), isnotnull(nextAvailabilityDateTime#619296), (nextAvailab..., Fo...

  • Storage Level: Disk Memory Serialized 2x Replicated
  • Cached Partitions: 37
  • Total Partitions: 37
  • Memory Size: 8.4 MiB
  • Disk Size: 1329.9 KiB

Data Distribution on 3 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
10.96.20.101:7079 4.5 MiB (267.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.25.83:7079 948.4 KiB (1848.6 MiB Remaining) 0.0 B (0.0 B Remaining) 1329.9 KiB
10.96.13.76:7079 3.0 MiB (1156.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

37 Partitions

Page:
Block Name Storage Level Size in Memory ▴ Size on Disk Executors
rdd_61949_28 Disk Memory Serialized 2x Replicated 130.1 KiB 130.1 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_14 Disk Memory Serialized 2x Replicated 130.2 KiB 130.2 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_25 Disk Memory Serialized 2x Replicated 132.0 KiB 132.0 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_61949_6 Disk Memory Serialized 2x Replicated 132.2 KiB 132.2 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_35 Disk Memory Serialized 2x Replicated 133.0 KiB 133.0 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_4 Disk Memory Serialized 2x Replicated 133.2 KiB 133.2 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_32 Disk Memory Serialized 2x Replicated 133.5 KiB 133.5 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_5 Disk Memory Serialized 2x Replicated 133.6 KiB 133.6 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_33 Disk Memory Serialized 2x Replicated 135.5 KiB 135.5 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_8 Disk Memory Serialized 2x Replicated 136.7 KiB 136.7 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_13 Memory Serialized 2x Replicated 257.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_23 Memory Serialized 2x Replicated 259.1 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_3 Memory Serialized 2x Replicated 262.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_10 Memory Serialized 2x Replicated 266.1 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_16 Memory Serialized 2x Replicated 266.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_19 Memory Serialized 2x Replicated 266.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_20 Memory Serialized 2x Replicated 266.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_27 Memory Serialized 2x Replicated 267.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_22 Memory Serialized 2x Replicated 267.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_21 Memory Serialized 2x Replicated 268.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_15 Memory Serialized 2x Replicated 268.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_0 Memory Serialized 2x Replicated 269.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_29 Memory Serialized 2x Replicated 269.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_1 Memory Serialized 2x Replicated 270.1 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_2 Memory Serialized 2x Replicated 270.2 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_34 Memory Serialized 2x Replicated 271.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_36 Memory Serialized 2x Replicated 271.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_17 Memory Serialized 2x Replicated 272.1 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_7 Memory Serialized 2x Replicated 272.8 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_61949_31 Memory Serialized 2x Replicated 273.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_26 Memory Serialized 2x Replicated 273.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_11 Memory Serialized 2x Replicated 274.2 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_61949_9 Memory Serialized 2x Replicated 276.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_30 Memory Serialized 2x Replicated 277.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_24 Memory Serialized 2x Replicated 278.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_61949_18 Memory Serialized 2x Replicated 280.8 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_61949_12 Memory Serialized 2x Replicated 281.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
Page: