RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [0_2_10000000528 AS shardId#835583, worklistShardItemId#835810L, product#835796, plant#835791, availabilityDateTime#835798 AS periodStart#838792, nextAvailabilityDateTime#835799 AS periodEnd#838793, 0 AS version#838791, null AS qty#838794] +- Exchange hashpartitioning(worklistShardItemId#835810L, 37), REPARTITION_BY_NUM, [plan_id=443920] +- Project [plant#835791, product#835796, availabilityDateTime#835798, nextAvailabilityDateTime#835799, worklistShardItemId#835810L] +- Filter ((isnotnull(availabilityDateTime#835798) AND isnotnull(nextAvailabilityDateTime#835799)) AND (nextAvailabilityDateTime#835799 > availabilityDateTime#835798)) +- FileScan parquet [plant#835791,product#835796,availabilityDateTime#835798,nextAvailabilityDateTime#835799,worklistShardItemId#835810L,shardId#835831,sequenceId#835809] Batched: true, DataFilters: [isnotnull(availabilityDateTime#835798), isnotnull(nextAvailabilityDateTime#835799), (nextAvailab..., Fo...

  • Storage Level: Disk Memory Serialized 2x Replicated
  • Cached Partitions: 31
  • Total Partitions: 37
  • Memory Size: 6.7 MiB
  • Disk Size: 603.7 KiB

Data Distribution on 3 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
10.96.20.101:7079 2.8 MiB (267.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.25.83:7079 958.0 KiB (1848.6 MiB Remaining) 0.0 B (0.0 B Remaining) 603.7 KiB
10.96.13.76:7079 2.9 MiB (1156.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

31 Partitions

Page:
Block Name Storage Level Size in Memory ▴ Size on Disk Executors
rdd_71009_9 Disk Memory Serialized 2x Replicated 118.2 KiB 118.2 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_71009_26 Disk Memory Serialized 2x Replicated 119.1 KiB 119.1 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_71009_2 Disk Memory Serialized 2x Replicated 121.9 KiB 121.9 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_71009_23 Disk Memory Serialized 2x Replicated 122.0 KiB 122.0 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_71009_6 Disk Memory Serialized 2x Replicated 122.5 KiB 122.5 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_71009_8 Memory Serialized 2x Replicated 232.9 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_71009_35 Memory Serialized 2x Replicated 235.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_1 Memory Serialized 2x Replicated 235.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_71009_30 Memory Serialized 2x Replicated 235.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_18 Memory Serialized 2x Replicated 236.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_33 Memory Serialized 2x Replicated 237.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_3 Memory Serialized 2x Replicated 238.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_36 Memory Serialized 2x Replicated 239.9 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_71009_31 Memory Serialized 2x Replicated 240.4 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_71009_22 Memory Serialized 2x Replicated 240.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_19 Memory Serialized 2x Replicated 240.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_20 Memory Serialized 2x Replicated 240.6 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_71009_5 Memory Serialized 2x Replicated 241.0 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_71009_25 Memory Serialized 2x Replicated 242.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_16 Memory Serialized 2x Replicated 242.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_27 Memory Serialized 2x Replicated 242.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_71009_10 Memory Serialized 2x Replicated 242.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_71009_13 Memory Serialized 2x Replicated 243.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_0 Memory Serialized 2x Replicated 243.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_11 Memory Serialized 2x Replicated 244.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_12 Memory Serialized 2x Replicated 245.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_15 Memory Serialized 2x Replicated 245.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_17 Memory Serialized 2x Replicated 245.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_14 Memory Serialized 2x Replicated 245.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_32 Memory Serialized 2x Replicated 248.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_71009_24 Memory Serialized 2x Replicated 252.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
Page: