Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running unit tests in Business Central Online Sandbox environments #3754

Open
ejwoerlee opened this issue Nov 8, 2024 · 11 comments
Open

Running unit tests in Business Central Online Sandbox environments #3754

ejwoerlee opened this issue Nov 8, 2024 · 11 comments
Assignees

Comments

@ejwoerlee
Copy link

ejwoerlee commented Nov 8, 2024

I'm trying to run unit tests from a Sandbox environment using BCHelper scripts / Run-TestsInBCContainer in
my DevOps pipeline.

I have a Sandbox in which I am able to publish my App and the correspronding TestApp.

The Sandbox contains the TestRunner and Test libraries. In the Sandbox I can run the unit tests manually.
Also I can make a connection with Sandbox with right BcAuthContext (ClientId and ClientSecret) and publish
the App and the corresponding TestApp (containing the unit tests)

I want to run all the unit tests using a powershell script that is called by DevOps pipeline
I have read https://community.dynamics.com/blogs/post/?postid=56456fac-3563-4d98-9e35-5d61c33e1457

Scripts used to create container and cause the issue

# create files only container ?

         $artifactUrl = Get-BcArtifactUrl -type Sandbox -country "NL" -version "24.5.23489.23968" -select Closest
             
        # Define the authentication method and service principal details
        $auth = "AAD"  # Use AAD for Azure Active Directory Authentication ????
        $aadTenantId = "$TenantId"
        $clientId = "$ClientId"
        $clientSecret = "$ClientSecret"
        $adminEmail = "[email protected]"  # The Azure AD email for the admin user ?? what email do I need here?
        New-BcContainer `
            -accept_eula `
            -containerName $ContainerName `
            -artifactUrl $artifactUrl `
            -auth $auth `
            -filesOnly `
            -updateHosts `
            -authenticationEMail $adminEmail `
            -additionalParameters  @(
                "--env AAD_TENANT_ID=$aadTenantId",
                "--env AAD_CLIENT_ID=$clientId",
                "--env AAD_CLIENT_SECRET=$clientSecret",
                "--env AuthenticationEMail=$adminEmail"
            )

$bcAuthContext = New-BcAuthContext -TenantId $TenantId -ClientId $clientId -ClientSecret $ClientSecret
$bcAuthContext.upn = "[email protected]"  # Set this to a valid admin email in Azure AD manually ?? 



$containerName = "BcFilesOnly"
$testAppId  is the ID of my TestApp
$TestSuiteName  (is the name of my test suite added to the Sandbox)

$securePassword = ConvertTo-SecureString -String $ClientSecret -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential ("[email protected]", $securePassword)

Import-TestToolkitToBcContainer `
        -containerName $containerName `
        -bcAuthContext $bcAuthContext `
        -environment $environmentName `
        -includeTestRunnerOnly

  Run-TestsInBCContainer `
        -containerName $containerName `
        -bcAuthContext $bcAuthContext `
        -environment $environmentName `
        -extensionId $testAppId `
        -credential $credential `
        -detailed  `
        -XUnitResultFileName $xunitResultsFile `
        -testSuite $TestSuiteName `
        -testCodeunitRange '*' `
        -AzureDevOps "error"   

Full output of scripts

App Permissions Mock from Microsoft version 24.5.23489.24156 is already installed
App Test Runner from Microsoft version 24.5.23489.24156 is already installed
TestToolkit successfully published
[DBG]: PS E:\BC\BC_ExToAr\Base> 
[DBG]: PS E:\BC\BC_ExToAr\Base> 
Connecting to https://msweuweuas4260-z26hbip.appservices.weu.businesscentral.dynamics.com/cs?tenant=msweua5608t88481773&tid
ERROR DIALOG: S2S application cannot log in using client type: ClientService.
at Run-Tests, C:\ProgramData\BcContainerHelper\Extensions\BcFilesOnly\PsTestTool\PsTestFunctions.ps1: line 575
at <ScriptBlock>, <No file>: line 75
Cannot open page 130455. You might need to import the test toolkit to the container and/or remove the folder C:\ProgramData\BcContainerHelper\Extensions\BcFilesOnly\PsTestTool and retry. You might also have URL or Company name wrong.

Exception Script Stack Trace:
at Run-Tests, C:\ProgramData\BcContainerHelper\Extensions\BcFilesOnly\PsTestTool\PsTestFunctions.ps1: line 575
at <ScriptBlock>, <No file>: line 75

PowerShell Call Stack:
at Invoke-ScriptInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.18\ContainerHandling\Invoke-ScriptInNavContainer.ps1: line 71
at Run-TestsInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.18\AppHandling\Run-TestsInNavContainer.ps1: line 387
at <ScriptBlock>, E:\BC\BC_ExToAr\Base\build\scripts\Run-UnitTests.ps1: line 137
at <ScriptBlock>, E:\BC\BC_ExToAr\Base\build\scripts\Test.ps1: line 23
at <ScriptBlock>, <No file>: line 1

Container Free Physical Memory: 13.7Gb
Disk C: Free 124Gb from 127Gb

Services in container BcFilesOnly:
- No services found

Run-TestsInBCContainer Telemetry Correlation Id: 8ca7f918-16b0-44ea-8c7e-b4411daf4773
Cannot open page 130455. You might need to import the test toolkit to the container and/or remove the folder 
C:\ProgramData\BcContainerHelper\Extensions\BcFilesOnly\PsTestTool and retry. You might also have URL or Company name wrong. 
At C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.18\AppHandling\Run-TestsInNavContainer.ps1:527 char:17   
+                 throw $_.Exception.Message
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Cannot open pag...any name wrong.:String) [], RuntimeException
    + FullyQualifiedErrorId : Cannot open page 130455. You might need to import the test toolkit to the container and/or remove the folder C:\ProgramData\BcContainerHelper\Extensions\BcFilesOnly\PsTestTool and retry. You might also have URL or Company name wrong.
@freddydk
Copy link
Contributor

Please remember to format your question like I have done.

I assume that this is the problem:
Connecting to https://msweuweuas4260-z26hbip.appservices.weu.businesscentral.dynamics.com/cs?tenant=msweua5608t88481773&tid
ERROR DIALOG: S2S application cannot log in using client type: ClientService.

Is your clientId registered inside Business Central?
and... why do you do this: $bcAuthContext.upn = "[email protected]" # Set this to a valid admin email in Azure AD manually ??

@ejwoerlee
Copy link
Author

ejwoerlee commented Nov 11, 2024

Hi Freddty, Thanks for the reply.

Sorry, but I'm not sure what you mean by "Please remember to format your question like I have done."
Do you mean like font-size or something?

The clientId is registered within my BC SAAS Sandbox as a Microsoft Entra Application
with permission sets: D365 BUS FULL ACCES, EXTEN. MGT. -ADMIN, SYSTEM APP - ADMIN, SYSTEM APP - BASIC, TEST TOOL

The ClientId and Secret are registered in the Azure portal using App registrations with API permissions
Dynamics 365 Business Central (5) / AdminCenter.ReadWrite.All, API.ReadWrite.All, app_access, Automation.ReadWrite.All, user_impersonation

I can call l for example a list of installed extension using:

$bcAuthContext = New-BcAuthContext -TenantId $TenantId -ClientId $clientId -ClientSecret $ClientSecret

$installedExtensions = (Get-BcInstalledExtensions `
        -bcAuthContext $bcauthContext `
        -environment $environmentName )

or publish app and test app, for example:

$environmentName = 'my-sandbox-name'
Publish-PerTenantExtensionApps `
        -bcAuthContext $bcAuthContext `
        -environment $environmentName `
        -schemaSyncMode 'Force' `
        -appFiles $outputAppFile `

I presume this is the correct $clientId ?

If I do not set $bcAuthContext.upn to a value I get this error message when starting Run-TestsInBCContainer:
(Run-TestsInNavContainer.ps1 (line 226) / $credential = New-Object pscredential -ArgumentList $bcAuthContext.upn, (ConvertTo-SecureString -String $accessToken -AsPlainText -Force)

Here $bcAuthContext.upn is empty, ->

New-Object : Exception calling ".ctor" with "2" argument(s): "Cannot process argument because the value of argument "userName" is not valid. Change the value of the "userName" argument and run the operation again."

@freddydk
Copy link
Contributor

freddydk commented Nov 11, 2024

Formatting of code is done with this button:
Image

What do you specify as environmentName?

@ejwoerlee
Copy link
Author

Hi, Thanks I just found out howto format ;-)

For environmentName I use the name of my sandbox (actually launch.json/environmentName,
for example: "development-test"

@freddydk
Copy link
Contributor

I am pretty sure that the upn trick is the problem here - you are setting the upn to an account, which doesn't match the access token you retrieve from the authentication.

You could try to set the upn to $authContext.appId to see whether that works.

@ejwoerlee
Copy link
Author

ejwoerlee commented Nov 11, 2024

I assume you mean $bcAuthContext.upn = $bcAuthContext.appId
(this is the appId that I mentioned above) ? This gives the same error:

Connecting to https://msweuweuas4260-z26hbip.appservices.weu.businesscentral.dynamics.com/cs?tenant=msweua5608t88481773&tid
ERROR DIALOG: S2S application cannot log in using client type: ClientService.
at Run-Tests, C:\ProgramData\BcContainerHelper\Extensions\BcFilesOnly\PsTestTool\PsTestFunctions.ps1: line 575
at , : line 75
Cannot open page 130455. You might need to import the test toolkit to the container and/or remove the folder C:\ProgramData\BcContainerHelper\Extensions\BcFilesOnly\PsTestTool and retry. You might also have URL or Company name wrong.

Another question: if the scripts I presented are correct, do I have to set this field to a value manually?

@ejwoerlee
Copy link
Author

Freddy,

I think I found that this does not work:

 $bcAuthContext = New-BcAuthContext -TenantId $TenantId -ClientId $ClientId -ClientSecret $ClientSecret

 $UserName = $ClientId
 $SecurePassword = ConvertTo-SecureString $ClientSecret -AsPlainText -Force

 $credential = New-Object System.Management.Automation.PSCredential ($UserName, $SecurePassword)


 Run-TestsInBCContainer `
        -containerName $containerName `
        -bcAuthContext $bcAuthContext `
        -environment $environmentName `
        -credential $credential `
        -detailed  `
        -testSuite $TestSuiteName `
        -testCodeunitRange '*' `
        -AzureDevOps "error"

While this does seem to work:

    $UserName = "[email protected]"  
    $SecurePassword = ConvertTo-SecureString "MySandBoxPwd" -AsPlainText -Force    
    # Credential for running tests
    $credential = New-Object System.Management.Automation.PSCredential ($UserName, $SecurePassword)
   
    $bcauthContext = New-BcAuthContext -includeDeviceLogin // login with xyz@test.com, password MySandBoxPwd

 Run-TestsInBCContainer `
        -containerName $containerName `
        -bcAuthContext $bcAuthContext `
        -environment $environmentName `
        -credential $credential `
        -detailed  `
        -testSuite $TestSuiteName `
        -testCodeunitRange '*' `
        -AzureDevOps "error"

So is my conclusion correct that $bcAuthContext and $credential must have the same SAAS Sandbox username/password?
I wanted to use New-BcAuthContext -TenantId $TenantId -ClientId $ClientId -ClientSecret $ClientSecret to prevent updating
a refresh token
. But this does not really work with TestsInBCContainer, because an BC App (Id) doesn't have a login/password that can
be used by $credential and RunTestsInBCContainer

Also: Is the way I create the container (using AAD credentials) correct ?

# List available artifact URLs        
        $artifactUrl = Get-BcArtifactUrl -type Sandbox -country "NL" -version "24.5.23489.23968" -select Closest
             
        # Define the authentication method and service principal details
        $auth = "AAD"  # Use AAD for Azure Active Directory Authentication
        $aadTenantId = "$TenantId"
        $clientId = "$ClientId"
        $clientSecret = "$ClientSecret"
        $adminEmail = "[email protected]"  # The Azure AD email for the admin user

        New-BcContainer `
            -accept_eula `
            -containerName $ContainerName `
            -artifactUrl $artifactUrl `
            -auth $auth `
            -filesOnly `
            -updateHosts `
            -authenticationEMail $adminEmail `
            -additionalParameters  @(
                "--env AAD_TENANT_ID=$aadTenantId",
                "--env AAD_CLIENT_ID=$clientId",
                "--env AAD_CLIENT_SECRET=$clientSecret",
                "--env AuthenticationEMail=$adminEmail"
            )

@freddydk
Copy link
Contributor

You don't really need the container when running towards an online sandbox.
You definitely don't need the AAD parameters when creating a filesonly container - nor updatehosts - and you don't need to transfer containername and credentials to run-tests if using an online environment.

@ejwoerlee
Copy link
Author

So you don't need a files only container at all ?

If I try this:

Run-TestsInBCContainer `
        -bcAuthContext $bcAuthContext `
        -environment $environmentName `
        -credential $credential `
        -detailed  `
        -testSuite $TestSuiteName `
        -testCodeunitRange '*' `
        -AzureDevOps "error"  

I get:
Error: No such object: bcserver
Error: No such object: bcserver
Run-TestsInBCContainer Telemetry Correlation Id: 8164e13b-fa74-4285-8f81-9c87cb7ab0c4
Get-BcContainerSharedFolders : The property 'HostConfig' cannot be found on this object. Verify that the
property exists.
At C:\Program
Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.18\ContainerInfo\Get-NavContainerPath.ps1:37 char:26

  • ... edFolders = Get-BcContainerSharedFolders -containerName $containerNam ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidOperation: (:) [Get-BcContainerSharedFolders], RuntimeException
    • FullyQualifiedErrorId : PropertyNotFoundStrict,Get-BcContainerSharedFolders

that looks like it is searching for a containerName ?
Also https://freddysblog.com/2021/02/15/running-tests-in-business-central-online-sandbox-environments/
states:

Like when importing the test toolkit, a proxy container is needed to run the tests. A FilesOnly container will do. Output of the function should be something like:

@freddydk
Copy link
Contributor

I think that has been fixed - update BcContainerHelper to the latest version and retry...

Here are the things that has been fixed since 6.0.18:

6.0.28
Increase performance of Get-BcArtifactUrl when selecting latest artifact, by using an approximate filtering (set useApproximateVersion to false in settings to disable)

6.0.27
Issue 3538 Compile-AppWithBcCompilerFolder fails when dependency does propagateDependencies
Issue 3727 Regression - Release pipelines failing with SaaS environments due BcAuthContext
Add new overrides to Run-AlPipeline: BackupBcContainerDatabases and RestoreDatabasesInBcContainer for backup up and restoring databases - only parameter transferred is containerName
Add new parameter to Run-AlPipeline: restoreDatabases, indicating when (during the pipeline execution) a database restore is needed
Add PageScripting test runner to Run-AlPipeline, adding 4 new parameters: pageScriptingTests, pageScriptingTestResultsFile, pageScriptingTestResultsFolder and doNotRunPageScriptingTests.
Add BCPT test apps to build artifacts in Run-AlPipeline

6.0.26
As minimum, always use the generic tag version which was available when shipping BcContainerHelper
Add awareness of Windows 11 24H2
Issue #3697 Running tests fails with 401 when running on a BC25 sandbox container with AAD authentication
Issue #3701 Compile-AppWithBcCompilerFolder, ALTool returns dependencies.appId for older apps
Wait for databases to be mounted when running Restore-DatabasesInBcContainer
Include AI Toolkit when copying symbols from container
Issue 3714 Download-BcNuGetPackageToFolder - throws error when use "-select Exact"
Issue 3621 New-AadAppsForBc shows "-includeEmailAadApp is deprecated. Use -includeOtherServicesAadApp instead." but includeOtherServicesAadApp us not setting email permissions
Issue 3718 Business Central Web Client Fails to Load After Updating to Windows 24H2
Issue #3703 Missing Function Download-BcEnvironmentInstalledExtensionToFolder    
Fix for issue 1262 in AL-Go for GitHub
Fix instabilities during Publish-PerTenantExtensionApps

6.0.25
Fix issue where generateDependencyArtifact doesn't result in dependencies if useCompilerFolder is true or filesonly containers are used in Run-AlPipeline
Issue 3686 PowerShell container shortcuts for >= BC24 using wrong PowerShell version
Issue 3666 Run-AlCops + Run-AlPipeline using outdated appsource default ruleset
Fix issue where linux executables are missing in the AL Language Extension (use dotnet to execute the commands instead)
Fix issue where PowerShell sometimes hangs when trying to establish a new PSSession shortly after removing another PSSession (Noticed during end 2 end tests on AL-Go for GitHub)

6.0.24
Use pre-release altool when running alcops (to be able to get runtime version of nextmajor)

6.0.23
Issue 3573 GitHub NuGet Handling fails for User Feeds

6.0.22
Issue #3660 Error about missing 'azuredevops' property when using Compile-AppWithBcCompilerFolder

6.0.21
Main branch is now main instead of master
Issue 3613 aka.ms/getbc fallback to NAVUserPassword because token expired when setting up SSO
Issue #3617 Regression File Newtonsoft.Json.dll not found when copying item for New-BcCompilerFolder
Remove functions for supporting Alpaca and Cloud Containers - integration between alpaca and cloud containers with AL-Go for GitHub will NOT be done through BcContainerHelper
Issue #2623 New-BcContainer for BC25.1 or BC26 insider with includeTestToolkit AI Test Toolkit dependency missing
If template for applicationDependency or platformDependency is specified when creating a new BcNuGetPackage, it will create an invalid package if application or platform properties are missing in app.json
Normalize version strings in nuGetFeed to ensure that NuGet packages with version 1.2.3 are seen as being equal to version 1.2.3.0
Add Schedule parameter to Publish-PerTenantExtensionApps

6.0.20
Add trustServerCertificate is the parameter exists in various functions that might be running on the host
Issue #3594 Run-AlCops says unknown platform version
Add information about apps excluded due to not being referenced during Publish-BcContainerApp
Run-AlPipeline to support CompilerFolder and online environments for building and testing
Added 2 new overrides in Run-AlPipeline: PipelineInitialize and PipelineFinalize
Add parameter CompilerFolder to Run-TestsInBcContainer and Import-TestToolkitToBcContainer for running tests using CompilerFolder bits from the host
Replace all occurrences of [System.Runtime.InteropServices.Marshal]::PtrToStringAuto with [System.Runtime.InteropServices.Marshal]::PtrToStringBSTR as the Auto function doesn't always do what's expected under Linux
Ensure correct casing of Newtonsoft.Json.dll for Linux
Always add extensionId (when specified) to Properties section in test results xml
If environment is specified as a Web Client URL, and BcAuthContext contains username/password in Run-AlPipeline, then tests will run against this environment. PublishBcContainerApp and ImportTestToolkitToBcContainer needs to be overridden for this to work with full pipeline.
Including caching of appinfos in CompilerFolder cache (to save time when caching on GitHub Actions)

6.0.19
Issue #3560 Backup-BcContainerDatabases fails - password must be marked as read only bug
Issue #3559 ALCops can't find "Business Foundation" app and doesn't start because of it
Support for federated credentials in Azure
Issue 1093 from AL-Go repo, generateDependencyArtifact doesn't work when useCompilerFolder is true

@ejwoerlee
Copy link
Author

ejwoerlee commented Nov 12, 2024

Ok tried it, but What value do I give for -compilerFolder?
If I use for example: C:\Users\user1.vscode\extensions\ms-dynamics-smb.al-15.0.1177813, there is no symbols folder.
(using AL al-15.0.1177813)

if ($containerName) {
        Write-Host "Using Container"
        $customConfig = Get-BcContainerServerConfiguration -ContainerName $containerName
        $navversion = Get-BcContainerNavversion -containerOrImageName $containerName
        $version = [System.Version]($navversion.split('-')[0])
        $PsTestToolFolder = Join-Path $bcContainerHelperConfig.hostHelperFolder "Extensions\$containerName\PsTestTool"
    }
    elseif ($compilerFolder) {
        Write-Host "Using CompilerFolder"
        $customConfig = $null
        $symbolsFolder = Join-Path $compilerFolder "symbols"

BcContainerHelper version 6.0.28
WinRM service is not running, will not try to use WinRM sessions
BC.HelperFunctions emits usage statistics telemetry to Microsoft
Running on Windows, PowerShell 5.1.22621.4249
LatestGenericTagVersion is 1.0.2.50

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants