* Quality Week 2022: The First Step to a Better Documentation.
* Removed the Markdown Extension's additional development tags.
* Removed redundant stuff.
* Removed redundant stuff.
* Removed guide testing versions to avoid confusion that they would be the only supported versions.
* Addressed some Powershell comments, added a pointer to a yaml with links to other .NET sibling repos, and did a couple mini-corrections. Still got a few comments to address...
* Fixed a ';:' with Powershell's syntax.
* Addressed review comments: Redaction improvements, localization url's, improved code snippets, added info on individual subsets building configurations, changed COMPlus for DOTNET, and other misc improvements.
* Addressed remaining comments.
* Addressed some last comments regarding corerun, and the subset flags in the build scripts.
Co-authored-by: Ivan Diaz <bluehorizon186@gmail.com>
* Add test
* Add CHECKs
* Build scripts and test settings
* COMPlus -> DOTNET
* Fix filenames
* Missing >
* Missing quotes
* use %scriptPath%
* Rework properties, start two examples
* Arch demo
* Cleanup
* Remove tab
* Easier environment variables. Undo precommand changes.
* undo blank line
* clean CHECKs
* Draft of bash
* Bash, conditionals
* More variables, start on run.cmd/sh/py
* another output
* Support in cmd/bash for RunningDisasmChecks
* copy, factor, formatting
* Initial work to include FileCheck. Added SuperFileCheck.
* Able to build SuperFileCheck
* Do not DisasmCheck TypeEquality_r for now. Update some FileChecks to follow SuperFileCheck rules.
* Partially wiring up SuperFileCheck to tests
* Piping list of method names from SuperFileCheck to JitDisasm
* Handling bash a little bit
* Moving SuperFileCheck to tests/Common
* Few tweaks
* Building SuperFileCheck as part of the test build
* Tweaking a few things
* Fixed a bug
* Moving SuperFileCheck back to src\coreclr\tools. Removed checks from TypeEquality_r.
* Restore original logic in Runtime_73681
* Trying to add CI leg for disasmchecks
* Use x64 package if x86 platform detected for JIT tools package
* Remove innerloop for disasmchecks
* Trying to fix build. Only run in Windows for now.
* Update Runtime_73681.cs
Trying to fail test
* Trying to fix build
* Update Runtime_73681.cs
* Update Runtime_73681.cs
* Fixing a few issues
* Trying to run disasmchecks as part of CI
* Trying to run disasmchecks
* Trying to run disasmchecks
* Trying to run disasmchecks
* Revert a change
* Trying to run disasmchecks
* Trying to run disasmchecks
* build SuperFileCheck on non-windows
* few tweaks
* Trying to fix CI
* Including SuperFileCheck for tests
* Cleanup
* More cleanup
* Cleanup
* Changed SuperFileCheck to not publish everything. Changed SuperFileCheck's lookup for FileCheck.
* Invoking SuperFileCheck using dotnet
* Making the test pass
* Only run disasm checks for coreclr and not mono
* Using HasBatchDisasmCheck and HasBashDisasmCheck to determine to run the check
* Enabling filecheck on linux and osx
* Added more comments
* Added ARM64 specific test. Do not run SuperFileCheck if no methods were found.
* Added documentation. Changed disasm-output.
* Minor doc tweak
* Minor doc tweak
* Minor doc tweak
* Minor doc tweak
* Minor doc tweak
* Cleanup. Trying to fix linux
* Fixing test
* Add information on additional functionality
* cleanup
* Add FileCheck snippet
* Undo environment variable changes
* Feedback from Mark
* Cleanup
* Trying to fix linux test run
* Trying to fix linux test run
* A few missing changes from the original branch
* Enable OSX for disasm checks
* cleanup / comment
* Force test failure
* Update Runtime_73681.cs
* Set env vars after errorlevel check
* Reverting back on setting environment variables in test. Added new FileCheck test for mod optimization
* Force a failure by changing the register
* Ignore native binaries for superpmi collect
* Update Runtime_34937.cs
* Force the correct failure
* Update Runtime_34937.cs
* Update Runtime_34937.cs
* Adding specific OS check prefixes. Changed dump-input context amount
* Added getting fully qualified method names with wildcards for SuperFileCheck
* More tests. Fixed a few issues with generics.
* Disabling generic support
* Error if it cannot find enclosing type declaration
* Fixing build
* Remove namespac
* Bring generics back, but in a limited form
Co-authored-by: Mark Plesko <markples@microsoft.com>
- This will help simplifying the path subsets in
`eng/pipelines/common/evaluate-default-paths.yml`, which would then help
to avoid a lot of unrelated non-wasm builds on PRs
* [wasm] Modify workload to pick threading runtime packs
This change adds the 2 wasm threading runtime packs to the wasm workload. In order for a threading runtime pack to be chosen, WorkloadManifest.targets is also modified to override the runtime pack name when the following props are set:
WasmEnableThreads - full threading support and will load Microsoft.NETCore.App.Runtime.multithread.Mono.browser-wasm
WasmEnablePerfTrace - runtime only threading support and will load Microsoft.NETCore.App.Runtime.perftrace.Mono.browser-wasm
* Add error condition when both WasmEnableThreading and WasmEnablePerfTrace are true
* [wasm] Set default value for MonoWasmBuildVariant, for runtime pack nuget
* [wasm] Wasm.Build.Tests: build runtime pack nugets with different names
- The earlier approach of simply making copies of the existing runtime
pack nuget with different names doesn't work, and `dotnet workload
install` rejects it.
```
Installing pack Microsoft.NETCore.App.Runtime.Mono.multithread.browser-wasm version 7.0.0-ci...
Workload installation failed. Rolling back installed packs...
```
Instead, now we build the missing nugets from the project with different
values for `$(MonoWasmBuildVariant)`.
- this handles local builds, and incremental builds also
- To skip building the missing nugets, for example, when you have all of
them available, then set `WasmSkipMissingRuntimePackBuild=true`.
* fix comment
Co-authored-by: Steve Pfister <steve.pfister@microsoft.com>
Co-authored-by: Ankit Jain <radical@gmail.com>
* Add document describing how to disable tests
Update libraries "filtering tests" doc with information and examples for
using `SkipOnCoreClrAttribute`
* Update for feedback
Added SkipOnMonoAttribute, CollectionAttribute, an example
of using multiple SkipOnCoreClrAttribute, and how the xunit
attributes for test disabling apply to the "converted"
src/tests tree.
* Use CMakeProjectReference instead of ProjectReference
Changing this permits usage of "\runtime\dotnet.cmd build" as opposed
to forcing users to use "\runtime\dotnet.cmd msbuild".
- Move all SetScriptCommands to a target, so they are placed after scenario in the generated script.
- Add WasmXHarnessArgsCli as an alternative variable that can be used from cli.
- Fix NodeJS symbolic links on Helix.
- Copy TestEchoMiddleware and RemoteLoopMiddleware even for Scenario=NodeJS.
- Enable System.Net.WebSockets.Client.Tests on NodeJS.
* Modify existing tests
* Enable process start and kill on MacCatalyst
* Typo
* Temporarily enable ProcessTests for MacCatalyst
* Temporarily enable ProcessTests for MacCatalyst - attempt 2
* Allow running Mac Catalyst builds in App Sandbox
* Allow enabling app sandbox for the other Mac Catalyst sample
* Add missing parameter to XCode project generator
* Remove unnecessary MacCatalyst detection
* Enable App Sandbox for Mac Catalyst tests
* Create a separate test branch for App Sandbox
* Remove the restriction to enable app sandbox just for Mac Catalyst apps
* Do not throw PNSE for Mac Catalyst
* Clean-up platform-specific conditions
* Build correct Process implementation for MacCatalyst
* Try to get more information from CI for further investigation of the failing build
* Revert "Try to get more information from CI for further investigation of the failing build"
This reverts commit fc63a370d6.
* Add MacCatalyst target framework for System.Diagnostics.Process
* Add supported platform annotations for Mac Catalyst
* Fix annotations
* Remove incorrectly placed attributes
* Update attributes including the reference file
* Change platform attributes
* Generate app.entitlements in the BuildAppleAppBundles test build target
* Revert "Generate app.entitlements in the BuildAppleAppBundles test build target"
This reverts commit abbe224158.
* Enable AppSandbox when generating CMakeLists.txt for libraries tests
* Try implementing a workaround for app sandbox mode
* Fix app sandbox detection bug
* Add explanation comment
* Enable more tests for MacCatalyst
* Add apple app builder input validation
* Add Mac Catalyst w/ App Sandbox enabled to runtime-manual test pipeline
* Enable networking in App Sandbox mode
* Skip test which won't work on Mac Catalyst
* Skip some tests in app sandbox mode
* Update src/tasks/AppleAppBuilder/AppleAppBuilder.cs
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
* Add a clarification comment for a networking entitlement
* Simplify supported platform condition
* Remove temporarily enabled test
* Remove unnecessary attributes
* Replace checking env variable with checking errno in libproc
* Update docs
* Remove unnecessary changes
* Temporarily enable running System.Diagnostics.Process.Tests for this PR
This reverts commit 02d370c2cf.
* Revert "Temporarily enable running System.Diagnostics.Process.Tests for this PR"
This reverts commit dc72f0f1d0.
* Fix job suffix in runtime-manual
* Remove attributes
* Revert "Remove attributes"
This reverts commit 704e9fa73c.
* Try changing the order of attributes to please CI
* Update ApiCompat baseline
* ApiCompat step 1: remove all attributes for MaxWorkingSet setter
* Revert "Update ApiCompat baseline"
This reverts commit a2ad03207c.
* Revert "ApiCompat step 1: remove all attributes for MaxWorkingSet setter"
This reverts commit be72a3d256.
* Update ApiCompat baseline
* Update ApiCompat baseline after pulling upstream main
* Remove trailing whitespace
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
* [tests] If -debug is LLDB, add separator before corerun and args
For runtime test shell scripts, if a debugger is specified with `-debug=/usr/bin/lddb` or the like, add a separator so that `lldb` doesn't try to interpret `-p` as a PID argument.
Also update mono workflow doc
* Also support gdb as the debugger
* Remove EOL tfms and upgrade to net7.0
* Make linker tests net7 aware
Make installer tests run on net7.0
Don't attempt to use live apphost
Fix payloads
Regenerate test files
Fix package testing
* Implement support for passing a .env file to corerun to easily set environment variables for stress testing.
* Load the parsed dotenv into the process.
* Implement support for # comments
* Clean up code
* Avoid copy
* PR feedback
* Update docs and bash/batch scripts to support the dotenv files.
* Use what clang recommends to not copy
* Apply suggestions from code review
Co-authored-by: Aaron Robinson <arobins@microsoft.com>
Co-authored-by: Aaron Robinson <arobins@microsoft.com>
New tests that extend the System.Private.CoreLib API should add to the ref assemblies in src/libraries and the tests themselves should be library tests.
Updated the workflow doc: the Clr.Native subset is all that is need if we're interested in testing Mono.
Contributes to https://github.com/dotnet/runtime/issues/58266
* [tests] Use RuntimeFlavor to build ReferenceSystemPrivateCoreLib tests
When building tests that reference System.Private.CoreLib, use the one from the specified RuntimeFlavor (or fall back to coreclr if it's unset - although src/tests/build.sh sets it always)
* Update docs: We only need to build the Clr.Native subset if we're interested in testing Mono.
* flavor the pipelines to build the runtime-specific System.Private.CoreLib for the runtime tests.
* Don't build ReferenceSystemPrivateCoreLib tests with Mono
We should avoid adding new ReferenceSystemPrivateCoreLib tests
This change adds four new options to the runtime test build command:
1) test <test project path relative to src\tests> - build just that
one test.
2) dir <directory relative to src\tests> - build all tests in the
directory.
3) tree <directory relative to src\tests> - build all tests in the
given subtree.
4) all - use clean rebuild (i.e. don't apply incrementalism)
when building the tests.
Thanks
Tomas
* Update instructions for running runtime tests on Android
* Update docs/workflow/testing/mono/testing.md
Co-authored-by: Juan Hoyos <juan.hoyos@microsoft.com>
Co-authored-by: Juan Hoyos <juan.hoyos@microsoft.com>
* Build IJW assemblies against LKG SDK ref pack.
* Allow building against the live ref assemblies by passing `-cmakeargs -DCPP_CLI_LIVE_REF_ASSEMBLIES=1` to the test build script.
* Add test instructions
* Error handling
* Add support for the stdcall-mangled _CorDllMain entrypoint on x86 since MSVC uses it if it exists.
* Write out ref-pack path.
* Add warning on unmaintained testing doc page
* Update testing.md
Some example text that seems more clear to me, but only offered as a suggestion.
Feel free to adjust it or if you want to use it as-is please double check what I wrote is accurate : )
I think the useful elements are:
1. Being explicit about what workflow steps need to happen in total
2. Being explicit about which commands are covering the entire workflow and which ones are only covering a part of it
3. Show the simple "do-it-all" options first before showing more complex partial options. Glancing at the first example and blindly copying it should land in the pit of success.
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Update testing docs with x86 instructions
* Apply suggestions from code review
Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
* add more examples
* Apply suggestions from code review
Co-authored-by: Dan Moseley <danmose@microsoft.com>
* Update docs/workflow/testing/libraries/testing.md
Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
Co-authored-by: Dan Moseley <danmose@microsoft.com>
Co-authored-by: Eric StJohn <ericstj@microsoft.com>
* corerun supports user defined properties from command line.
* Update workflow help doc for corerun.
Co-authored-by: Elinor Fung <elfung@microsoft.com>
Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.
## This is done by:
1. building the test assemblies on the build machine
- the wasm part of the build is not executed on the build machine,
because it has the AOT build part
2. Zip up the test assembly+friends, *and* any bits required to run the wasm
app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom `aot-build.proj`
- which recreates all the build inputs for the `WasmBuildApp` target
using the paths for the assets on helix
- then we can run `WasmBuildApp` for the build, resulting in a wasm app
bundle.
4. Run the tests!
- We already have the bits required for building wasm apps on helix, supported
for `Wasm.Build.Tests`, which we can use here too.
## Trimming:
- Since, AOT can be so expensive, we use `EnableAggressiveTrimming=true`(EAT), but
that means that we could have issues due to trimming.
- And it can sometimes be unclear whether the build/test failures are due to trimming
or AOT.
- Because these builds+test runs are different from other builds, owing to the
"build partially on helix" step, a normal EAT build would not be the same as
- to help with testing this, we add two lanes to `runtime-staging`:
- `*_Mono_AOT`: builds AOT+EAT on helix
- `*_Mono_EAT`: builds EAT, on helix
- this is required because we want to run almost the same kinda
build: 1. build test assembly; *2. send to helix; 3. build wasm app;* 4. run tests
- This should effectively mean that we can see which errors might be due to EAT, and
which are clearly because of EAT+AOT.
Co-authored-by: Steve Pfister <steve.pfister@microsoft.com>
Co-authored-by: Marek Safar <marek.safar@gmail.com>
Co-authored-by: Mitchell Hwang <mitchell.hwang@microsoft.com>
Co-authored-by: Ankit Jain <radical@gmail.com>
We have a bunch of test assemblies that don't make sense on some platforms, e.g. Browser.
Right now we're skipping them via `[SkipOnMono("reason", TestPlatforms.Browser)]` but there's nothing that inherently ties this to Mono other than the current implementation.
The more generic `SkipOnPlatform` attribute can be used instead.
We can also use it in places where we do `[PlatformSpecific(~TestPlatforms....)]` to avoid the double inversion.
Ivan Diaz Sanchez