Rewrite physical expressions in execution plans #20009
Conversation
github-actions
bot
added
physical-expr
|
@askalt, could you review the PR as well? |
|
cc @Omega359 (I remember you were also interested in placeholders within physical plans). |
Yes, I would like to be able to reuse plans - I'll try and find time in the next day or so to review this though I'm not particularly familiar with this area of code. |
LLDay
force-pushed
the
feat/physical-placeholders
branch
from
7d86e73 to
eb09e71
Compare
LLDay
force-pushed
the
feat/physical-placeholders
branch
from
eb09e71 to
3046ed0
Compare
github-actions
bot
removed
optimizer
LLDay
changed the title
LLDay
force-pushed
the
feat/physical-placeholders
branch
from
3046ed0 to
4420b7f
Compare
| Ok(Some(Arc::new(Self { | ||
| projector, | ||
| input: Arc::clone(&self.input), | ||
| metrics: ExecutionPlanMetricsSet::new(), | ||
| cache: self.cache.clone(), | ||
| }))) |
There was a problem hiding this comment.
Let's use a bit shorter form that highlights what were really changed (we could apply it for most plans):
Ok(Some(Arc::new(Self {
projector,
metrics: ExecutionPlanMetricsSet::new(),
..self.clone()
})))There was a problem hiding this comment.
I applied these changes where possible (some execution plans do not implement clone).
| filter, | ||
| join_type: self.join_type, | ||
| join_schema: Arc::clone(&self.join_schema), | ||
| left_fut: Arc::clone(&self.left_fut), |
There was a problem hiding this comment.
We have two different semantics for runtime state: on the one hand metrics are recreated, when left_fut is cloned. Let's explicitly re-set a runtime state in with_physical_expressions: recreate left_fut and add a note to the trait method comment, that it is ok to not preserve the state.
There was a problem hiding this comment.
I followed the behavior of the with_new_children method. It keeps dynamic filters and left_fut, but recreates metrics. I added a note to the with_physical_expressions method.
| self.name() | ||
| ); | ||
|
|
||
| let lex_ordering = LexOrdering::new( |
There was a problem hiding this comment.
Can we add a method for LexOrdering to avoid code duplication in different sort nodes?
There was a problem hiding this comment.
Added the try_with_new_expressions method. I'm still wondering if this method should be part of the LexOrdering, because it is of little use.
| fn physical_expressions<'a>( | ||
| &'a self, | ||
| ) -> Option<Box<dyn Iterator<Item = Arc<dyn PhysicalExpr>> + 'a>> { | ||
| let left_sort_iter = self |
There was a problem hiding this comment.
It seems we also need to return self.on (to ensure that query like select * from t1 join t2 on t1.a + $1 = t2.a works).
| /// Returns a new [`DataSource`] with its physical expressions replaced by the provided | ||
| /// `exprs`. | ||
| /// | ||
| /// # Constraints |
There was a problem hiding this comment.
Let's also add notes about properties (for example, DataSource::EquivalenceProperties) and state here.
|
By some reasons, I cannot resolve threads in this PR -- if you can, let's resolve all threads from the previous review iteration. |
LLDay
force-pushed
the
feat/physical-placeholders
branch
from
4420b7f to
e4dd613
Compare
Introduces `physical_expressions` and `with_physical_expressions` methods to the `ExecutionPlan` trait. These methods provide an interface to inspect and replace physical expressions within execution plan nodes. The commit also implements these methods for various ExecutionPlans and introduces a new physical expression invariant check to ensure consistency between expression retrieval and replacement.
LLDay
force-pushed
the
feat/physical-placeholders
branch
from
e4dd613 to
2186826
Compare
3 participants
Rationale for this change
This PR covers a part of #14342.
This PR introduces methods for replacing physical expressions in execution plan nodes. This is an intermediate step in the development of physical-level placeholders. Later, these methods will be used to resolve placeholders in physical plans.
What changes are included in this PR?
Expression Rewriting Interface: Added
physical_expressionsandwith_physical_expressionsto theExecutionPlantrait to allow inspection and replacement of expressions within plan nodes.Are these changes tested?
physical_expressionsandwith_physical_expressionsmethods are verified using thecheck_physical_expressionsinvariant.Are there any user-facing changes?
Yes, a user can rewrite physical expressions in some execution plans.