RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [0_6_10000000529 AS shardId#21982, worklistShardItemId#22050L, product#22036, plant#22031, availabilityDateTime#22038 AS periodStart#28370, nextAvailabilityDateTime#22039 AS periodEnd#28371, 0 AS version#28369, null AS qty#28372] +- Exchange hashpartitioning(worklistShardItemId#22050L, 37), REPARTITION_BY_NUM, [plan_id=323308] +- Project [plant#22031, product#22036, availabilityDateTime#22038, nextAvailabilityDateTime#22039, worklistShardItemId#22050L] +- Filter ((isnotnull(availabilityDateTime#22038) AND isnotnull(nextAvailabilityDateTime#22039)) AND (nextAvailabilityDateTime#22039 > availabilityDateTime#22038)) +- FileScan parquet [plant#22031,product#22036,availabilityDateTime#22038,nextAvailabilityDateTime#22039,worklistShardItemId#22050L,shardId#22071,sequenceId#22049] Batched: true, DataFilters: [isnotnull(availabilityDateTime#22038), isnotnull(nextAvailabilityDateTime#22039), (nextAvailabil..., Format: Parquet, Location: Pr...

  • Storage Level: Disk Memory Serialized 2x Replicated
  • Cached Partitions: 14
  • Total Partitions: 37
  • Memory Size: 3.3 MiB
  • Disk Size: 0.0 B

Data Distribution on 3 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
10.96.20.101:7079 1064.7 KiB (267.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.25.83:7079 1429.5 KiB (1848.6 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.13.76:7079 836.7 KiB (1156.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

14 Partitions

Page:
Block Name Storage Level ▴ Size in Memory Size on Disk Executors
rdd_63670_0 Memory Serialized 2x Replicated 235.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63670_2 Memory Serialized 2x Replicated 234.4 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63670_3 Memory Serialized 2x Replicated 235.8 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63670_7 Memory Serialized 2x Replicated 239.6 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63670_15 Memory Serialized 2x Replicated 234.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63670_17 Memory Serialized 2x Replicated 237.1 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_63670_19 Memory Serialized 2x Replicated 245.2 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63670_20 Memory Serialized 2x Replicated 236.7 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63670_21 Memory Serialized 2x Replicated 243.9 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63670_26 Memory Serialized 2x Replicated 236.1 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63670_30 Memory Serialized 2x Replicated 240.5 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63670_33 Memory Serialized 2x Replicated 236.0 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_63670_35 Memory Serialized 2x Replicated 240.0 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_63670_36 Memory Serialized 2x Replicated 234.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
Page: