Merge branch 'main' into pre-commit-ci-update-config

Author: e-belfer

.github/ISSUE_TEMPLATE/existing_data_updates.md

@@ -2,7 +2,7 @@
 name: Integrate New Year of Data
 about: Check-list for integrating a new year of data
 title: ""
-labels: new-data
+labels: data-update
 assignees: ""
 ---
 

.github/workflows/deploy-pudl.yml

@@ -0,0 +1,12 @@
+---
+name: deploy-pudl
+on:
+  workflow_dispatch:
+
+jobs:
+  deploy_pudl:
+    name: Deploy PUDL build
+    runs-on: ubuntu-latest
+    steps:
+      - name: stub
+        run: echo 'stub'

dbt/seeds/etl_full_row_counts.csv

@@ -46,8 +46,8 @@ archivers themselves.
 
 **1.2)** Obtain production DOIs from
 `Zenodo <https://zenodo.org/communities/catalyst-cooperative/>`__ and then update the
-dictionary of in :mod:`pudl.workspace.datastore` to refer to the new raw input
-archives.
+record in :mod:`pudl.package_data.settings.zenodo_dois.yml` to refer to the new raw
+input archives.
 
 **1.3)** In :py:const:`pudl.metadata.sources.SOURCES`, update the ``working_partitions``
 to reflect the years, months, or quarters of data that are available for each dataset
@@ -92,10 +92,10 @@ the years (e.g. ``boiler_fuel``). However ``page_name`` does not necessarily cor
 directly to PUDL database table names because we don't load the data from all pages, and
 some pages result in more than one database table after normalization.
 
-**2.A.1)** If you're adding a new year, add a column for the new year of data to
+**2.A.1)** If you're adding a new year, add a row for the new year of data to
   each of the aforementioned files. If there are any changes to prior years, make
   sure to address those too. If you are updating early release data with final
-  release data, replace the values in the appropriate year column. **The easiest way
+  release data, replace the values in the appropriate year row. **The easiest way
   to correct the values for these files is to test extraction in Dagster as
   described in the next step, then use the error messages to narrow down what should
   be updated.** Exhaustively examining each file manually to compare it with its

docs/dev/existing_data_updates.rst

@@ -48,12 +48,28 @@ EPA CEMS
 * Updated EPA CEMS hourly emissions data through December 2025. See :issue:`4986`
   and :pr:`4990`.
 
+EIA-860M
+~~~~~~~~
+
+* Updated EIA-860M with monthly data through December 2025. See :issue:`4983` and
+  :pr:`4993`.
+
+EIA-923
+~~~~~~~
+
+* Updated EIA-923 with monthly data through November 2025. See :issue:`4984` and
+  :pr:`4993`.
+
 EIA 930
 ~~~~~~~
 
 * Updated EIA 930 data through December 2025. See :issue:`4985`
   and :pr:`4995`.
 
+EIA 176
+~~~~~~~
+
+* Updated EIA 176 data through 2024. See :issue:`5000` and :pr:`5005`.
 
 Documentation
 ~~~~~~~~~~~~~

docs/release_notes.rst

@@ -73,7 +73,7 @@
             ),
             "source_format": "Comma Separated Value (.csv)",
         },
-        "working_partitions": {"years": sorted(set(range(1997, 2024)))},
+        "working_partitions": {"years": sorted(set(range(1997, 2025)))},
         "contributors": [CONTRIBUTORS["catalyst-cooperative"]],
         "keywords": sorted(
             set(
@@ -207,7 +207,7 @@
         "working_partitions": {
             "year_months": [
                 str(q).lower()
-                for q in pd.period_range(start="2015-07", end="2025-11", freq="M")
+                for q in pd.period_range(start="2015-07", end="2025-12", freq="M")
             ],
         },
         "keywords": sorted(

src/pudl/metadata/sources.py

@@ -26,3 +26,4 @@ year_index,operating_state,operator_id_eia,operator_name,operator_status,report_
 2021,report_state,id,company_name,status,last_update
 2022,report_state,id,company_name,status,last_update
 2023,report_state,id,company_name,status,last_update
+2024,report_state,id,company_name,status,last_update

src/pudl/package_data/eia176/column_maps/company_list.csv

@@ -26,3 +26,4 @@ year_index,operating_state,reference_state,operator_id_eia,operator_name,report_
 2021,report_state,ref_state,company_id,company_name,year,survey_line,survey_line_description,volume_mcf,reference_company_or__line_description,mode_of__transportation
 2022,report_state,ref_state,company_id,company_name,year,survey_line,survey_line_description,volume_mcf,reference_company_or__line_description,mode_of__transportation
 2023,report_state,ref_state,company_id,company_name,year,survey_line,survey_line_description,volume_mcf,reference_company_or__line_description,mode_of__transportation
+2024,report_state,ref_state,company_id,company_name,year,survey_line,survey_line_description,volume_mcf,reference_company_or__line_description,mode_of__transportation

src/pudl/package_data/eia176/column_maps/continuation_text_lines.csv

@@ -26,3 +26,4 @@ year_index,operator_name,operating_state,value,variable_name,form_line_numbers,o
 2021,company,area,value,item,itemsort,id,line,atype
 2022,company,area,value,item,itemsort,id,line,atype
 2023,company,area,value,item,itemsort,id,line,atype
+2024,company,area,value,item,itemsort,id,line,atype

src/pudl/package_data/eia176/column_maps/custom.csv

@@ -26,3 +26,4 @@ year_index,report_year,operating_state,operator_id_eia,operator_name,is_distribu
 2021,year,state,company,company_name,cooperative,residential_total_volume_mcf,residential_sales_volume_mcf,residential_sales_price_dollars,residential_transported_price_dollars,residential_sales_revenue_dollars,residential_total_customers,residential_sales_customers,commercial_total_volume_mcf,commercial_sales_volume_mcf,commercial_sales_price_dollars,commercial_transported_price_dollars,commercial_sales_revenue_dollars,commercial_total_customers,commercial_sales_customers,industrial_sales_price_dollars,industrial_transported_price_dollars,industrial_total_customers,electric_total_volume_mcf,electric_sales_volume_mcf,electric_sales_price_dollars,electric_transported_price_dollars,electric_sales_revenue_dollars,electric_total_customers,electric_sales_customers,vehicle_fuel_sales_price_dollars,vehicle_fuel_transported_price_dollars,vehicle_fuel_total_customers,intrastate,producer,gatherer,privately_owned,losses_from_leaks_etc_within_state_mcf,industrial_total_volume_mcf,industrial_sales_volume_mcf,industrial_sales_revenue_dollars,industrial_sales_customers,electric_transported_volume_mcf,electric_transported_revenue_dollars,electric_transported_customers,storage_operator,industrial_transported_volume_mcf,industrial_transported_customers,investor_owned,commercial_transported_volume_mcf,commercial_transported_revenue_dollars,commercial_transported_customers,municipally_owned,interstate,lng_operator,cng_public,industrial_transported_revenue_dollars,vehicle_fuel_sales_revenue_dollars,vehicle_fuel_sales_customers,vehicle_fuel_total_volume_mcf,vehicle_fuel_sales_volume_mcf,vehicle_fuel_transported_volume_mcf,vehicle_fuel_transported_revenue_dollars,vehicle_fuel_transported_customers,other_ownership,,lng_public,other_descr,residential_transported_volume_mcf,residential_transported_customers,residential_transported_revenue_dollars,sng_operator,lng_marine_terminal
 2022,year,state,company,company_name,cooperative,residential_total_volume_mcf,residential_sales_volume_mcf,residential_sales_price_dollars,residential_transported_price_dollars,residential_sales_revenue_dollars,residential_total_customers,residential_sales_customers,commercial_total_volume_mcf,commercial_sales_volume_mcf,commercial_sales_price_dollars,commercial_transported_price_dollars,commercial_sales_revenue_dollars,commercial_total_customers,commercial_sales_customers,industrial_sales_price_dollars,industrial_transported_price_dollars,industrial_total_customers,electric_total_volume_mcf,electric_sales_volume_mcf,electric_sales_price_dollars,electric_transported_price_dollars,electric_sales_revenue_dollars,electric_total_customers,electric_sales_customers,vehicle_fuel_sales_price_dollars,vehicle_fuel_transported_price_dollars,vehicle_fuel_total_customers,intrastate,producer,gatherer,privately_owned,losses_from_leaks_etc_within_state_mcf,industrial_total_volume_mcf,industrial_sales_volume_mcf,industrial_sales_revenue_dollars,industrial_sales_customers,electric_transported_volume_mcf,electric_transported_revenue_dollars,electric_transported_customers,storage_operator,industrial_transported_volume_mcf,industrial_transported_customers,investor_owned,commercial_transported_volume_mcf,commercial_transported_revenue_dollars,commercial_transported_customers,municipally_owned,interstate,lng_operator,cng_public,industrial_transported_revenue_dollars,vehicle_fuel_sales_revenue_dollars,vehicle_fuel_sales_customers,vehicle_fuel_total_volume_mcf,vehicle_fuel_sales_volume_mcf,vehicle_fuel_transported_volume_mcf,vehicle_fuel_transported_revenue_dollars,vehicle_fuel_transported_customers,other_ownership,,lng_public,other_descr,residential_transported_volume_mcf,residential_transported_customers,residential_transported_revenue_dollars,sng_operator,lng_marine_terminal
 2023,year,state,company,company_name,cooperative,residential_total_volume_mcf,residential_sales_volume_mcf,residential_sales_price_dollars,residential_transported_price_dollars,residential_sales_revenue_dollars,residential_total_customers,residential_sales_customers,commercial_total_volume_mcf,commercial_sales_volume_mcf,commercial_sales_price_dollars,commercial_transported_price_dollars,commercial_sales_revenue_dollars,commercial_total_customers,commercial_sales_customers,industrial_sales_price_dollars,industrial_transported_price_dollars,industrial_total_customers,electric_total_volume_mcf,electric_sales_volume_mcf,electric_sales_price_dollars,electric_transported_price_dollars,electric_sales_revenue_dollars,electric_total_customers,electric_sales_customers,vehicle_fuel_sales_price_dollars,vehicle_fuel_transported_price_dollars,vehicle_fuel_total_customers,intrastate,producer,gatherer,privately_owned,losses_from_leaks_etc_within_state_mcf,industrial_total_volume_mcf,industrial_sales_volume_mcf,industrial_sales_revenue_dollars,industrial_sales_customers,electric_transported_volume_mcf,electric_transported_revenue_dollars,electric_transported_customers,storage_operator,industrial_transported_volume_mcf,industrial_transported_customers,investor_owned,commercial_transported_volume_mcf,commercial_transported_revenue_dollars,commercial_transported_customers,municipally_owned,interstate,lng_operator,cng_public,industrial_transported_revenue_dollars,vehicle_fuel_sales_revenue_dollars,vehicle_fuel_sales_customers,vehicle_fuel_total_volume_mcf,vehicle_fuel_sales_volume_mcf,vehicle_fuel_transported_volume_mcf,vehicle_fuel_transported_revenue_dollars,vehicle_fuel_transported_customers,other_ownership,,lng_public,other_descr,residential_transported_volume_mcf,residential_transported_customers,residential_transported_revenue_dollars,sng_operator,lng_marine_terminal
+2024,year,state,company,company_name,cooperative,residential_total_volume_mcf,residential_sales_volume_mcf,residential_sales_price_dollars,residential_transported_price_dollars,residential_sales_revenue_dollars,residential_total_customers,residential_sales_customers,commercial_total_volume_mcf,commercial_sales_volume_mcf,commercial_sales_price_dollars,commercial_transported_price_dollars,commercial_sales_revenue_dollars,commercial_total_customers,commercial_sales_customers,industrial_sales_price_dollars,industrial_transported_price_dollars,industrial_total_customers,electric_total_volume_mcf,electric_sales_volume_mcf,electric_sales_price_dollars,electric_transported_price_dollars,electric_sales_revenue_dollars,electric_total_customers,electric_sales_customers,vehicle_fuel_sales_price_dollars,vehicle_fuel_transported_price_dollars,vehicle_fuel_total_customers,intrastate,producer,gatherer,privately_owned,losses_from_leaks_etc_within_state_mcf,industrial_total_volume_mcf,industrial_sales_volume_mcf,industrial_sales_revenue_dollars,industrial_sales_customers,electric_transported_volume_mcf,electric_transported_revenue_dollars,electric_transported_customers,storage_operator,industrial_transported_volume_mcf,industrial_transported_customers,investor_owned,commercial_transported_volume_mcf,commercial_transported_revenue_dollars,commercial_transported_customers,municipally_owned,interstate,lng_operator,cng_public,industrial_transported_revenue_dollars,vehicle_fuel_sales_revenue_dollars,vehicle_fuel_sales_customers,vehicle_fuel_total_volume_mcf,vehicle_fuel_sales_volume_mcf,vehicle_fuel_transported_volume_mcf,vehicle_fuel_transported_revenue_dollars,vehicle_fuel_transported_customers,other_ownership,,lng_public,other_descr,residential_transported_volume_mcf,residential_transported_customers,residential_transported_revenue_dollars,sng_operator,lng_marine_terminal