build(lint): consolidate linters under pre-commit

[MichaelYochpaz]

This PR migrates all linting tools to pre-commit for unified local and CI execution.

This allows developers to run hatch run lint:precommit (or pre-commit run) to check for lint issues locally and auto-fix them locally, or even install git pre-commit hooks that will auto-fix them before making a commit.

Notes:

• Super-Linter which had Markdown linting (already covered by markdownlint) and editorconfig validation (now covered by editorconfig-checker) was removed.
• The .markdownlint-config.yaml config file was renamed to .markdownlint.yaml to follow the official naming convention, which is auto-used by the CLI tool used (reference). Beforehand the name was a "made up" name that was referenced by using --config .markdownlint-config.yaml.
• markdownlint required Node.js to work (we install it in the CI beforehand). The pre-commit implementation here makes so that it's required on the CI, but for local runs the markdownlint check is skipped (with a warning) if Node.js isn't installed.
• There were some unrelated files that were slightly modified. Those are files that were picked up by the pre-commit and fixed. If not updated (or more accurately, fixed to follow or linting rules), they would change whenever pre-commit is ran.

[tiran]

FWIW, I'm not a fan of pre-commit. I prefer to have linters and tests in the same automation tool, so I can run all checks in parallel from the same tool with a single command. I'm still maintaining my own tox.ini for Fromager and run ruff, linters, doc builds, and unit tests in parallel with tox p.

[MichaelYochpaz]

I prefer to have linters and tests in the same automation tool, so I can run all checks in parallel from the same tool with a single command.

@tiran This is what pre-commit does... You don't have to install the hooks, you can run pre-commit run and it will run all configured tests and linters with a single command.

[LalatenduMohanty]

Looks like some extra texts from AI agents?

[MichaelYochpaz]

It's intentional, it's just comments that help visually notice different sections (there are more on top of each pre-commit
It explains why this is the first one, in case someone wants to move it.

I can remove it if it seems redundant, it's not that important.

Maybe looking at it here instead of the diff view will help.

[LalatenduMohanty]

@MichaelYochpaz It is a nit pick, can you ask your AI agent to add only must have comments. We need comments only when the section is hard to understand, so that a comment will help with understanding if someone-going through it. Just not this instance of comments, but I saw some other comments which we do not actually need. Specially it is easier to understand why the code block is there with the help of a AI agent.

[LalatenduMohanty]

Pre-commit is a significant change in developer workflow, we can skip the pre-commit failures in the local workflow incase we want to focus on the code chnages initially i.e. git commit --no-verify -m "wip: work in progress". However I am not sure if we want to adopt this workflow. I need to use this workflow and findout more.

Developer Workflow WITHOUT Pre-commit:
┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐
│  Write   │ ─► │  Commit  │ ─► │   Push   │ ─► │ CI Fails │ 😞
│  Code    │    │ (no check)│   │          │    │ (lint!)  │
└──────────┘    └──────────┘    └──────────┘    └──────────┘

Developer Workflow WITH Pre-commit:
┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐
│  Write   │ ─► │  Commit  │ ─► │   Push   │ ─► │ CI Pass  │ ✅
│  Code    │    │ BLOCKED! │    │ (clean)  │    │          │
└──────────┘    └──────────┘    └──────────┘    └──────────┘
                     │
                Fix & retry

[MichaelYochpaz]

Just to make sure it's clear - you are not required to install pre-commit as a git hook. It's opt-in and you have to install the hook manually locally by running pre-commit install. Only then it run automatically before every commit, autofix issues, and if there are still issues remain, block you from commiting with an error message.

Another way to use it (which is what I use, and is written in the README) is just by running it manually with pre-commit run (or with hatch run lint:precommit in our repo). Just like how you'd run pytest or black or any other linter manually with a command. It just allows to unify them all under a single command that makes it easy and automatic for developers and contributors.

[tiran]

We need a compelling reason why we should introduce yet another automation tool to run linters. pre-commit is a new tool that we have not used in any project. We would need to train all engineers on the new tool. If hatch is not sufficient for our needs, then I would rather go back to tox (or maybe look into nox).

Also I'm worried about 321 insertions(+), 116 deletions(-). That's 200 more lines to accomplish the same task.

• Fromager was using tox until @LalatenduMohanty switched us to hatch in PR Start using hatch instead of tox #627
• https://github.com/python-wheel-build/elfdeps/ from this organization uses tox
• internally we are using tox in several places like the internal wheel builder repo
• no internal or public project uses pre-commit, yet

[MichaelYochpaz]

@tiran We already have pre-commit used here on Fromager. @dhellmann added it to the repo (Link #1, Link #2).
You can see I'm just modifying the existing pre-commit config file and not creating a new one.

I'm just consolidating the linters under the same tool, and making it easier to run the linters we only have on the CI, locally.
Running pre-commit and having the official linters fix linting issues for you locally is much more convenient over pushing, waiting for the CI to run, reading the failures, and manually fixing them.

I don't think line counts should be a metric. I've used a lot of comments and updated the docs/develop.md document to be more informative. I can easily deflate that number if that was a concern for some reason.

What training would developers need? It's literally just a single command to run.
If anything, using tox would require a lot more learning and training from developers.