RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [0_4_10000000529 AS shardId#992137, worklistShardItemId#992381L, product#992367, plant#992362, availabilityDateTime#992369 AS periodStart#995451, nextAvailabilityDateTime#992370 AS periodEnd#995452, 0 AS version#995450, null AS qty#995453] +- Exchange hashpartitioning(worklistShardItemId#992381L, 37), REPARTITION_BY_NUM, [plan_id=319900] +- Project [plant#992362, product#992367, availabilityDateTime#992369, nextAvailabilityDateTime#992370, worklistShardItemId#992381L] +- Filter ((isnotnull(availabilityDateTime#992369) AND isnotnull(nextAvailabilityDateTime#992370)) AND (nextAvailabilityDateTime#992370 > availabilityDateTime#992369)) +- FileScan parquet [plant#992362,product#992367,availabilityDateTime#992369,nextAvailabilityDateTime#992370,worklistShardItemId#992381L,shardId#992402,sequenceId#992380] Batched: true, DataFilters: [isnotnull(availabilityDateTime#992369), isnotnull(nextAvailabilityDateTime#992370), (nextAvailab..., Fo...

  • Storage Level: Disk Memory Serialized 2x Replicated
  • Cached Partitions: 37
  • Total Partitions: 37
  • Memory Size: 8.5 MiB
  • Disk Size: 1058.7 KiB

Data Distribution on 3 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
10.96.20.101:7079 3.0 MiB (267.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B
10.96.25.83:7079 1590.2 KiB (1848.6 MiB Remaining) 0.0 B (0.0 B Remaining) 1058.7 KiB
10.96.13.76:7079 4.0 MiB (1156.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

37 Partitions

Page:
Block Name Storage Level ▴ Size in Memory Size on Disk Executors
rdd_77086_2 Disk Memory Serialized 2x Replicated 131.8 KiB 131.8 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_11 Disk Memory Serialized 2x Replicated 134.8 KiB 134.8 KiB 10.96.20.101:7079 10.96.25.83:7079
rdd_77086_13 Disk Memory Serialized 2x Replicated 129.0 KiB 129.0 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_17 Disk Memory Serialized 2x Replicated 129.4 KiB 129.4 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_22 Disk Memory Serialized 2x Replicated 134.2 KiB 134.2 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_26 Disk Memory Serialized 2x Replicated 134.1 KiB 134.1 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_30 Disk Memory Serialized 2x Replicated 136.6 KiB 136.6 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_28 Disk Memory Serialized 2x Replicated 128.7 KiB 128.7 KiB 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_0 Memory Serialized 2x Replicated 261.0 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_1 Memory Serialized 2x Replicated 270.4 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_19 Memory Serialized 2x Replicated 270.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_3 Memory Serialized 2x Replicated 260.1 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_31 Memory Serialized 2x Replicated 262.6 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_6 Memory Serialized 2x Replicated 265.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_5 Memory Serialized 2x Replicated 266.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_4 Memory Serialized 2x Replicated 273.1 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_77086_7 Memory Serialized 2x Replicated 261.1 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_8 Memory Serialized 2x Replicated 259.5 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_9 Memory Serialized 2x Replicated 266.2 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_10 Memory Serialized 2x Replicated 266.6 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_12 Memory Serialized 2x Replicated 263.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_14 Memory Serialized 2x Replicated 269.7 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_15 Memory Serialized 2x Replicated 263.3 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_77086_16 Memory Serialized 2x Replicated 253.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_20 Memory Serialized 2x Replicated 263.4 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_18 Memory Serialized 2x Replicated 262.5 KiB 0.0 B 10.96.13.76:7079 10.96.25.83:7079
rdd_77086_21 Memory Serialized 2x Replicated 272.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_24 Memory Serialized 2x Replicated 267.8 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_23 Memory Serialized 2x Replicated 269.3 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_77086_25 Memory Serialized 2x Replicated 269.9 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_27 Memory Serialized 2x Replicated 260.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_32 Memory Serialized 2x Replicated 258.0 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
rdd_77086_33 Memory Serialized 2x Replicated 262.7 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_34 Memory Serialized 2x Replicated 262.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_29 Memory Serialized 2x Replicated 266.3 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_36 Memory Serialized 2x Replicated 265.5 KiB 0.0 B 10.96.13.76:7079 10.96.20.101:7079
rdd_77086_35 Memory Serialized 2x Replicated 265.8 KiB 0.0 B 10.96.20.101:7079 10.96.25.83:7079
Page: