-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
3.1.4
What's Wrong?
We encountered an stream load error for wide tables after upgrading our cluster to version 3.1.4:
org.apache.doris.spark.exception.StreamLoadException: stream load failed, status: 431, reason: Request Header Fields Too Large
Even though changing FE config jetty_server_max_http_header_size to an extremely large number (such as 1048576 for 1MB header), the problem still exists.
We found that the properties for configuring jetty server in org.apache.doris.httpv2.HttpServer are out-dated after spring boot 2.3 been released.
doris/fe/fe-core/src/main/java/org/apache/doris/httpv2/HttpServer.java
Lines 160 to 169 in 6384827
| properties.put("server.jetty.acceptors", this.acceptors); | |
| properties.put("server.jetty.max-http-post-size", this.maxHttpPostSize); | |
| properties.put("server.jetty.selectors", this.selectors); | |
| properties.put("server.jetty.threadPool.maxThreads", this.maxThreads); | |
| properties.put("server.jetty.threadPool.minThreads", this.minThreads); | |
| properties.put("server.max-http-header-size", this.maxHttpHeaderSize); | |
| // Worker thread pool is not set by default, set according to your needs | |
| if (this.workers > 0) { | |
| properties.put("server.jetty.workers", this.workers); | |
| } |
ref: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.3.0-M4-Configuration-Changelog
What You Expected?
update jetty property names to fix the problem
How to Reproduce?
stream load into wide tables whose column list is very long (max header size is 8k by default)
Anything Else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct