== Physical Plan ==
AdaptiveSparkPlan (11)
+- == Final Plan ==
ResultQueryStage (7), Statistics(sizeInBytes=8.0 EiB)
+- ObjectHashAggregate (6)
+- ShuffleQueryStage (5), Statistics(sizeInBytes=8.2 KiB, rowCount=50)
+- Exchange (4)
+- ObjectHashAggregate (3)
+- * Project (2)
+- * Scan ExistingRDD Delta Table State #20 - hdlfs://7da5304c-1f57-47fa-b75a-0f0b4349c280.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/order-proposal-update/10000000532/stream-10000000532-3/_delta_log (1)
+- == Initial Plan ==
ObjectHashAggregate (10)
+- Exchange (9)
+- ObjectHashAggregate (8)
+- Project (2)
+- Scan ExistingRDD Delta Table State #20 - hdlfs://7da5304c-1f57-47fa-b75a-0f0b4349c280.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/order-proposal-update/10000000532/stream-10000000532-3/_delta_log (1)
(1) Scan ExistingRDD Delta Table State #20 - hdlfs://7da5304c-1f57-47fa-b75a-0f0b4349c280.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/order-proposal-update/10000000532/stream-10000000532-3/_delta_log [codegen id : 1]
Output [10]: [txn#3506237, add#3506238, remove#3506239, metaData#3506240, protocol#3506241, cdc#3506242, checkpointMetadata#3506243, sidecar#3506244, domainMetadata#3506245, commitInfo#3506246]
Arguments: [txn#3506237, add#3506238, remove#3506239, metaData#3506240, protocol#3506241, cdc#3506242, checkpointMetadata#3506243, sidecar#3506244, domainMetadata#3506245, commitInfo#3506246], Delta Table State #20 - hdlfs://7da5304c-1f57-47fa-b75a-0f0b4349c280.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/order-proposal-update/10000000532/stream-10000000532-3/_delta_log MapPartitionsRDD[148926] at $anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128, ExistingRDD, UnknownPartitioning(0)
(2) Project [codegen id : 1]
Output [6]: [txn#3506237, add#3506238, remove#3506239, metaData#3506240, protocol#3506241, domainMetadata#3506245]
Input [10]: [txn#3506237, add#3506238, remove#3506239, metaData#3506240, protocol#3506241, cdc#3506242, checkpointMetadata#3506243, sidecar#3506244, domainMetadata#3506245, commitInfo#3506246]
(3) ObjectHashAggregate
Input [6]: [txn#3506237, add#3506238, remove#3506239, metaData#3506240, protocol#3506241, domainMetadata#3506245]
Keys: []
Functions [10]: [partial_collect_set(domainMetadata#3506245, 0, 0), partial_collect_set(txn#3506237, 0, 0), partial_count(protocol#3506241), partial_sum(add#3506238.size), partial_last(metaData#3506240, true), partial_count(metaData#3506240), partial_last(protocol#3506241, true), partial_count(remove#3506239), partial_count(add#3506238), partial_count(txn#3506237)]
Aggregate Attributes [12]: [buf#3506360, buf#3506361, count#3506362L, sum#3506363L, last#3506364, valueSet#3506365, count#3506366L, last#3506367, valueSet#3506368, count#3506369L, count#3506370L, count#3506371L]
Results [12]: [buf#3506372, buf#3506373, count#3506374L, sum#3506375L, last#3506376, valueSet#3506377, count#3506378L, last#3506379, valueSet#3506380, count#3506381L, count#3506382L, count#3506383L]
(4) Exchange
Input [12]: [buf#3506372, buf#3506373, count#3506374L, sum#3506375L, last#3506376, valueSet#3506377, count#3506378L, last#3506379, valueSet#3506380, count#3506381L, count#3506382L, count#3506383L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=1004504]
(5) ShuffleQueryStage
Output [12]: [buf#3506372, buf#3506373, count#3506374L, sum#3506375L, last#3506376, valueSet#3506377, count#3506378L, last#3506379, valueSet#3506380, count#3506381L, count#3506382L, count#3506383L]
Arguments: 0
(6) ObjectHashAggregate
Input [12]: [buf#3506372, buf#3506373, count#3506374L, sum#3506375L, last#3506376, valueSet#3506377, count#3506378L, last#3506379, valueSet#3506380, count#3506381L, count#3506382L, count#3506383L]
Keys: []
Functions [10]: [collect_set(domainMetadata#3506245, 0, 0), collect_set(txn#3506237, 0, 0), count(protocol#3506241), sum(add#3506238.size), last(metaData#3506240, true), count(metaData#3506240), last(protocol#3506241, true), count(remove#3506239), count(add#3506238), count(txn#3506237)]
Aggregate Attributes [10]: [collect_set(domainMetadata#3506245, 0, 0)#3506306, collect_set(txn#3506237, 0, 0)#3506307, count(protocol#3506241)#3506308L, sum(add#3506238.size)#3506309L, last(metaData#3506240) ignore nulls#3506310, count(metaData#3506240)#3506311L, last(protocol#3506241) ignore nulls#3506312, count(remove#3506239)#3506313L, count(add#3506238)#3506314L, count(txn#3506237)#3506315L]
Results [14]: [collect_set(domainMetadata#3506245, 0, 0)#3506306 AS domainMetadata#3506291, null AS fileSizeHistogram#3506292, collect_set(txn#3506237, 0, 0)#3506307 AS setTransactions#3506293, null AS numDeletedRecordsOpt#3506294, count(protocol#3506241)#3506308L AS numOfProtocol#3506295L, coalesce(sum(add#3506238.size)#3506309L, 0) AS sizeInBytes#3506296L, last(metaData#3506240) ignore nulls#3506310 AS metadata#3506297, count(metaData#3506240)#3506311L AS numOfMetadata#3506298L, null AS deletedRecordCountsHistogramOpt#3506299, null AS numDeletionVectorsOpt#3506300, last(protocol#3506241) ignore nulls#3506312 AS protocol#3506301, count(remove#3506239)#3506313L AS numOfRemoves#3506302L, count(add#3506238)#3506314L AS numOfFiles#3506303L, count(txn#3506237)#3506315L AS numOfSetTransactions#3506304L]
(7) ResultQueryStage
Output [14]: [domainMetadata#3506291, fileSizeHistogram#3506292, setTransactions#3506293, numDeletedRecordsOpt#3506294, numOfProtocol#3506295L, sizeInBytes#3506296L, metadata#3506297, numOfMetadata#3506298L, deletedRecordCountsHistogramOpt#3506299, numDeletionVectorsOpt#3506300, protocol#3506301, numOfRemoves#3506302L, numOfFiles#3506303L, numOfSetTransactions#3506304L]
Arguments: 1
(8) ObjectHashAggregate
Input [6]: [txn#3506237, add#3506238, remove#3506239, metaData#3506240, protocol#3506241, domainMetadata#3506245]
Keys: []
Functions [10]: [partial_collect_set(domainMetadata#3506245, 0, 0), partial_collect_set(txn#3506237, 0, 0), partial_count(protocol#3506241), partial_sum(add#3506238.size), partial_last(metaData#3506240, true), partial_count(metaData#3506240), partial_last(protocol#3506241, true), partial_count(remove#3506239), partial_count(add#3506238), partial_count(txn#3506237)]
Aggregate Attributes [12]: [buf#3506360, buf#3506361, count#3506362L, sum#3506363L, last#3506364, valueSet#3506365, count#3506366L, last#3506367, valueSet#3506368, count#3506369L, count#3506370L, count#3506371L]
Results [12]: [buf#3506372, buf#3506373, count#3506374L, sum#3506375L, last#3506376, valueSet#3506377, count#3506378L, last#3506379, valueSet#3506380, count#3506381L, count#3506382L, count#3506383L]
(9) Exchange
Input [12]: [buf#3506372, buf#3506373, count#3506374L, sum#3506375L, last#3506376, valueSet#3506377, count#3506378L, last#3506379, valueSet#3506380, count#3506381L, count#3506382L, count#3506383L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=1004493]
(10) ObjectHashAggregate
Input [12]: [buf#3506372, buf#3506373, count#3506374L, sum#3506375L, last#3506376, valueSet#3506377, count#3506378L, last#3506379, valueSet#3506380, count#3506381L, count#3506382L, count#3506383L]
Keys: []
Functions [10]: [collect_set(domainMetadata#3506245, 0, 0), collect_set(txn#3506237, 0, 0), count(protocol#3506241), sum(add#3506238.size), last(metaData#3506240, true), count(metaData#3506240), last(protocol#3506241, true), count(remove#3506239), count(add#3506238), count(txn#3506237)]
Aggregate Attributes [10]: [collect_set(domainMetadata#3506245, 0, 0)#3506306, collect_set(txn#3506237, 0, 0)#3506307, count(protocol#3506241)#3506308L, sum(add#3506238.size)#3506309L, last(metaData#3506240) ignore nulls#3506310, count(metaData#3506240)#3506311L, last(protocol#3506241) ignore nulls#3506312, count(remove#3506239)#3506313L, count(add#3506238)#3506314L, count(txn#3506237)#3506315L]
Results [14]: [collect_set(domainMetadata#3506245, 0, 0)#3506306 AS domainMetadata#3506291, null AS fileSizeHistogram#3506292, collect_set(txn#3506237, 0, 0)#3506307 AS setTransactions#3506293, null AS numDeletedRecordsOpt#3506294, count(protocol#3506241)#3506308L AS numOfProtocol#3506295L, coalesce(sum(add#3506238.size)#3506309L, 0) AS sizeInBytes#3506296L, last(metaData#3506240) ignore nulls#3506310 AS metadata#3506297, count(metaData#3506240)#3506311L AS numOfMetadata#3506298L, null AS deletedRecordCountsHistogramOpt#3506299, null AS numDeletionVectorsOpt#3506300, last(protocol#3506241) ignore nulls#3506312 AS protocol#3506301, count(remove#3506239)#3506313L AS numOfRemoves#3506302L, count(add#3506238)#3506314L AS numOfFiles#3506303L, count(txn#3506237)#3506315L AS numOfSetTransactions#3506304L]
(11) AdaptiveSparkPlan
Output [14]: [domainMetadata#3506291, fileSizeHistogram#3506292, setTransactions#3506293, numDeletedRecordsOpt#3506294, numOfProtocol#3506295L, sizeInBytes#3506296L, metadata#3506297, numOfMetadata#3506298L, deletedRecordCountsHistogramOpt#3506299, numDeletionVectorsOpt#3506300, protocol#3506301, numOfRemoves#3506302L, numOfFiles#3506303L, numOfSetTransactions#3506304L]
Arguments: isFinalPlan=true