Eliminate TestCentric Engine

[CharliePoole]

We have been slowly reducing the content of the TestCentric Engine and relying on the NUnit Engine for more and more features. I've been doing this slowly in the background and recently we seem to have reached a tipping point, where it became much easier to phase out classes and interfaces. I'm now fairly sure this will be ready for the next beta release, so this issue is for the purpose of documenting the change.

This is a breaking change both in regard to previous betas and the version1 GUI. However, it's effect is limited to developers creating their own runners using our engine. I don't know of any myself but if they exist, they should switch to use of the NUnit engine. Folks who use the GUI directly should not be impacted.

@Row360 I'm expecting that this change will probably initially break a feature or two in the GUI, since some of our implementations add functionality to the NUnit engine. When I merge the final change to remove the engine, I'd like you to identify any problems that arise. We can initially just list them here. Then you and I can decide, issue by issue, whether to fix them in NUnit or compensate (if possible) in the test model.

I'm also thinking that it makes sense to change the repo back to it's original name, testcentric-gui, once we are only a gui. What do you think?

[rowo360]

It's great that we're already at this stage. And I'm prepared to do the testing, identifying issues and list+discuss them here! 👍

Regarding the repro name: I don't think the name TestCentricRunner is bad. But the term GUI is more accurate. So, switching back to TestCentric.Gui is fine. When I think about it, I find a mix of both nice: GuiRunner. That sounds similar to the NUnit ConsoleRunner (Ok, I looked it up now: NUnit.ConsoleRunner is only the folder name in my nuget folder. The repro and the exe doesn't contain the term runner). But nonetheless that's a third option. 😄

[rowo360]

@CharliePoole:
I started to do some testing and these points caught my attention during testing:

• Stoping a test run doesn't work. Allthough I pressed 'Stop' and even 'Kill' sometime afterwards, the tests continue to run. I hope it's not just my environment (I use a Thread.Sleep() in a setup method to even have a chance of executing a stop).
• A Nunit-agent*.log file is created everytime. It remains empty - but I believe I didn't activate logging somewhere.
• The ResultWriter extensions are listed properly in the extension dialog. But it's not possible to write a result file in any different format. The method TestModel.ResultsFormats only returns 'NUnit3' format. Here are some screenshots:

• I cannot load any TestCentric project. Not even the one I just saved. Here's the message box:

• TestCentric blocks .Net 4.8 test assemblies. Well, we talked about this behavior already in the past and I was curious if it'll work now. We agreed that for .Net Core test assemblies a new approach must be implemented (maybe loading test assembly only during test run and unload afterwards), but .Net 4.8 is using the ShadowCopy approach. So, there's a chance that it'll work without any new implementation, but only a small bug prevents this. I can investigate this further...
• I've been playing around with the MaxAgents settings. Not right away but when creating a new project a crash happens. "An element with same key already inserted". (TestCentricProject constructor)

Maybe we should use Settings.Set(...) instead? Only for this setting or in general when applying any setting?

These are my initial observations; I will continue testing. One question: Should other extensions also work: i.e., ProjectLoader or TestEventListener?

[CharliePoole]

Thanks for that list...

For empty log files see nunit/nunit-console#1791. TestCentric.InternalTrace has a fix, which I'll need to port.

For stopping test run, see nunit/nunit-console#1187. We have some code in our engine for this one.

I'll investigate the other items in the morning.

UPDATE Extensions are periodically broken when we change the API. So all extensions currently need to be modified to use the latest extensibility and engine interfaces.

[rowo360]

Two tiny issues:

• Changing a package related setting in the Settings dialog but no project is loaded results into a crash:
  I'll take care about it and add the null check!

• Maybe it's possible to adapt the version number in der .Net 9.0 agent description?

[CharliePoole]

@rowo360

I agree. The null check is needed in TestCentric. I'll fix the agent description in the agent.

You'll also notice that all the agents are loaded. We have lost that feature, which I added to our testcentric agents, where agent loading is not needed to determine whether the agent targets a particular runtime.

Today I'll try to create a full list of things that need fixing in the engine for our benefit.

[rowo360]

Regarding the ShadowCopy for the .Net 4.62 agent: I'm not sure, but could it be that we simply need a new alpha release from the agent? Currently this is the lastest version: NUnit.Extension.Net462PluggableAgent.4.1.0-alpha.5
Perhaps this will be possible once all the other simple issues have been resolved?

[CharliePoole]

I'm having difficulty getting all tests to pass with the later releases of the engine, particularly the 462 build. That's why I have kept us at alpha.5 for the moment.

WRT ShadowCopy, it will become unnecessary in future, possibly soon, see nunit/nunit-console#1425.

There's a lot of conflicting ideas in that discussion, so here's what I'm currently thinking...

1. When loading an assembly, construct our in-memory representation of the tests and then unload the assembly.
2. If we discover that this causes problems in some cases, make it an option in the low-level call to load, default = unload.
3. When unloading tests, unload the assembly as well if it is still loaded.

Unloading the assembly when test loading is complete is probably a breaking change, but that's not a problem if we document it. I am assuming it's OK that you can't rebuild while loading is going on but what do you think?

[rowo360]

FYI: I fixed and merged the tiny issue with the null check.

Changing a package related setting in the Settings dialog but no project is loaded results into a crash

[CharliePoole]

Thank you!

…

On Sun, Dec 21, 2025 at 11:38 PM rowo360 ***@***.***> wrote: *rowo360* left a comment (TestCentric/TestCentricRunner#1396) <#1396 (comment)> FYI: I fixed and merged the tiny issue with the null check. Changing a package related setting in the Settings dialog but no project is loaded results into a crash — Reply to this email directly, view it on GitHub <#1396 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUN7BIQD36YY6TGDGY76R34C6NW7AVCNFSM6AAAAACPOYMT6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMOBQHA2TQOJXGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[rowo360]

I also think we should reconsider and improve the loading of the test assemblies. And it would also be great if we had a single approach that will work the same for both frameworks.
A few months ago, we solved this problem in the TestCentric engine by loading the test assembly each time before an action and unloading it again at the end of the action. During the execution of the action, the test assembly remained loaded and thus the file also remained blocked. Obviously, this approach requires multiple loading of the test assembly while user interaction. But so far it worked fine. Do you think that this approach must be further improved? Is it acceptable that a file is blocked during the execution of an action? For example, if a test runs for 10min, the test assembly (and all dependencies) are blocked for this period of time?
(I recall that we were not happy about the GetTestCount(filter) method, which required another load/unload roundtrip. And we considered if the NUnit API can be improved somehow to mitigate it).

Well, that's the known approach used by TestCentric. From your lines, I gather that you consider to take one step further. Of course you will also load the test assembly when starting an action, but you want to unload it right away after loading succeeded. The action itself might require some further time (for example 10min test run), but the test assembly is getting unblocked right away. This would definitely mean that the test assembly would be blocked for a much shorter period of time. Did I get it correct?
Do you already know the behavior for dependent assemblies? How they will be getting loaded? And are those blocked or not?
From TestCentric UI view we should avoid that the AssemblyWatcher forces a reload while a test is running in this case (or introduce some smart logic) - so far this use case was not possible, since the assembly was blocked during a test run,

In my view, the former approach used by TestCentric's is already sufficient to solve this issue. In particular, if the dependencies are already blocked during a test run, then blocking additionally the test assembly would not make much difference. Or do I miss some important use case?

One last remark:
Of course, all of this can also be solved in TestCentric code. By adding load/unload statements at all appropriate locations. But since you are fortunately the lead in both projects, you are in the best position to decide where this can best be resolved. 😄