RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [0_3_10000000528 AS shardId#691879, worklistShardItemId#693006L, product#692992, plant#692987, availabilityDateTime#692994 AS periodStart#698894, nextAvailabilityDateTime#692995 AS periodEnd#698895, 0 AS version#698893, null AS qty#698896] +- Exchange hashpartitioning(worklistShardItemId#693006L, 37), REPARTITION_BY_NUM, [plan_id=443947] +- Project [plant#692987, product#692992, availabilityDateTime#692994, nextAvailabilityDateTime#692995, worklistShardItemId#693006L] +- Filter ((isnotnull(availabilityDateTime#692994) AND isnotnull(nextAvailabilityDateTime#692995)) AND (nextAvailabilityDateTime#692995 > availabilityDateTime#692994)) +- FileScan parquet [plant#692987,product#692992,availabilityDateTime#692994,nextAvailabilityDateTime#692995,worklistShardItemId#693006L,shardId#693027,sequenceId#693005] Batched: true, DataFilters: [isnotnull(availabilityDateTime#692994), isnotnull(nextAvailabilityDateTime#692995), (nextAvailab..., Fo...

  • Storage Level: Disk Memory Serialized 2x Replicated
  • Cached Partitions: 37
  • Total Partitions: 37
  • Memory Size: 8.2 MiB
  • Disk Size: 740.4 KiB

Data Distribution on 3 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
10.96.20.101:7079 4.2 MiB (267.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.25.83:7079 2.2 MiB (1848.6 MiB Remaining) 0.0 B (0.0 B Remaining) 740.4 KiB
10.96.13.76:7079 1853.5 KiB (1156.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

37 Partitions

Page:
Block Name ▾ Storage Level Size in Memory Size on Disk Executors
rdd_65288_9 Memory Serialized 2x Replicated 247.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_8 Memory Serialized 2x Replicated 244.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_7 Memory Serialized 2x Replicated 242.2 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_6 Memory Serialized 2x Replicated 247.9 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_65288_5 Disk Memory Serialized 2x Replicated 121.9 KiB 121.9 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_4 Memory Serialized 2x Replicated 236.6 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_36 Disk Memory Serialized 2x Replicated 123.0 KiB 123.0 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_65288_35 Memory Serialized 2x Replicated 245.6 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_34 Memory Serialized 2x Replicated 244.0 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_33 Memory Serialized 2x Replicated 250.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_32 Memory Serialized 2x Replicated 241.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_31 Memory Serialized 2x Replicated 239.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_30 Memory Serialized 2x Replicated 243.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_3 Memory Serialized 2x Replicated 243.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_29 Memory Serialized 2x Replicated 240.3 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_28 Memory Serialized 2x Replicated 243.3 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_27 Disk Memory Serialized 2x Replicated 122.7 KiB 122.7 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_26 Memory Serialized 2x Replicated 246.7 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_25 Memory Serialized 2x Replicated 249.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_24 Memory Serialized 2x Replicated 244.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_23 Disk Memory Serialized 2x Replicated 127.4 KiB 127.4 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_22 Memory Serialized 2x Replicated 252.2 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_21 Memory Serialized 2x Replicated 242.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_20 Disk Memory Serialized 2x Replicated 123.2 KiB 123.2 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_2 Memory Serialized 2x Replicated 251.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_19 Memory Serialized 2x Replicated 243.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_18 Memory Serialized 2x Replicated 252.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_17 Memory Serialized 2x Replicated 249.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_16 Disk Memory Serialized 2x Replicated 122.2 KiB 122.2 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_15 Memory Serialized 2x Replicated 254.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_14 Memory Serialized 2x Replicated 242.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_13 Memory Serialized 2x Replicated 244.3 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_12 Memory Serialized 2x Replicated 252.9 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_11 Memory Serialized 2x Replicated 249.1 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_10 Memory Serialized 2x Replicated 248.6 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_65288_1 Memory Serialized 2x Replicated 247.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_65288_0 Memory Serialized 2x Replicated 244.5 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
Page: