[BUG] Prometheus monitor history data query fails due to 'app' parameter inconsistency in some storages

[Saramanda9988]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When querying historical metrics data for a Prometheus-type monitor, the query returns empty results even though the data has been successfully stored. Consequently, historical data charts display "No Data."

The issue stems from an inconsistency in the app parameter value between the storage and query phases for Prometheus monitors.

Expected Behavior

No response

Steps To Reproduce

No response

Environment

HertzBeat version(s):1.8.0

Debug logs

No response

Anything else?

At Storage (Monitor Creation/Modification): In MonitorServiceImpl.java:

if (CommonConstants.PROMETHEUS.equals(monitor.getApp())) {
    appDefine.setApp(CommonConstants.PROMETHEUS_APP_PREFIX + monitor.getName());
}

For Prometheus monitors, AppDefine.app is set to prometheus_{monitorName} (e.g., prometheus_my_monitor). This value is used to construct table names in storage engines.

At Storage Execution (using TdEngine as an example):

• String app = metricsData.getApp() (retrieves the prefixed name from AppDefine).
• Table name becomes app + "_" + metrics + "_" + instance + "_v2", resulting in prometheus_my_monitor_metrics_instance_v2.

At Query (Frontend to Controller):

• The frontend passes the app parameter from Monitor.getApp(), which is simply prometheus.
• MetricsDataController.java: Extracts app = names[0] (resulting in prometheus).

When the query reaches the storage layer (e.g., TdEngineDataStorage.java:360), getTable(app, metrics, instance) uses the raw prometheus string instead of the prefixed prometheus_my_monitor, failing to locate the correct table.

The following storages construct table names using the app parameter and are affected:
TDengine (TdEngineDataStorage.java)
IoTDB (IotDbDataStorage.java)
InfluxDB (InfluxdbDataStorage.java)
QuestDB (QuestdbDataStorage.java)

[Saramanda9988]

I’ve also investigated why GreptimeDB and VictoriaMetrics are not affected by this issue:

GreptimeDB: The table name is derived solely from metrics, not app. The app parameter is only used as a label filter in PromQL
VictoriaMetrics: It uses a label-based query model where app only affects label filtering logic, not the underlying storage structure.

These implementations follow a label-based (PromQL-compatible) model, whereas TDengine, IoTDB, and others use a physical table-based model constructed from the app name.

We may need to align other data sources with this label-based design to ensure consistency. This would require careful regression testing and potential data migration.

[Duansg]

@Saramanda9988 hi, thank you for reproducing and troubleshooting this issue.

Given that we will be releasing a new version soon, can this issue be addressed with minimal changes, such as:

Before querying the instrument panel, front-end compatibility checks are performed, for apps of the Prometheus type, dynamic concatenation is executed.

[Saramanda9988]

@Duansg hi, thanks for your suggestion.

I agree that we shouldn't make major changes for now. I mainly raised this issue to seek the community's opinion on the best long-term approach.

Using dynamic concatenation as a minimal change is fine, but it will require the monitor.name to be passed to the tsdb for querying. Maybe we would need to modify the signature of the method used for querying historical data?


Currently, these methods do not pass the monitor.name since GreptimeDB and VictoriaMetrics don't require it.

Is this ok for now? I'm eager to hear the community's thoughts!

[Duansg]

Hi, thank you for your response. In the long term, it is indeed necessary to standardize the consistency or near-consistency of write rules at the TSDB storage layer. This will enhance user convenience when setting thresholds. We are currently investigating this issue.

I believe temporarily using an additional parameter for pass-through should be fine. You can annotate it with a note like "deprecated," and we'll deprecate this parameter at the appropriate stage. Passing through a single parameter maximizes the benefit of resolving the current chart issues.

@Saramanda9988 @tomsun28 @zqr10159 what do you think?

[prachisingh342006]

I’d like to work on this issue. I’ll start by checking how the app parameter is handled across different storages and propose a fix.

[zqr10159]

I’d like to work on this issue. I’ll start by checking how the app parameter is handled across different storages and propose a fix.

Hello, this issue has already been claimed.