We are using Octopus deploy tool for CI, octopus has two steps configured and is given below
1- Octopus picks the automation package from the octopus library and deploys the file to the configured target machine
Note: Package will be extracted in the VM while deploying, And the target path to extract is configured in octopus.
2- Octopus will have a step to execute UI tests which triggers xunitconsolerunner to run tests in parallel through powershell script.
Observation: When the scripts are executed sequentially, UI tests are working fine but when run parallelly I am getting “System.NullReferenceException : Object reference not set to an instance of an object.”. When execute the same powershell script on VM through windows powershell parallel tests are working fine, opening 5 browser drivers at a time. But when run through octopus tentacle it throwing error. Strangely, scenarios in the last feature file are being executing.
I sense that it is a problem with opening multiple browsers at a time through octopus tentacle, since the last feature file is not having any parallel threads running together, it is trying to execute the last one.
Couldn’t understand if I am missing something here. Is it something because octopus tentacle can handle only one session of chrome webdriver. any suggestions would be helpful.
When I say -parallel none in the command, no issue as the browser will be opening one after the other sequentially “xunit.runner.console\xunit.console.exe iCHealth.CProvider.UISpecTests.dll -parallel none -trait “Category=V8” -html “$($automationPath)\index.html””
This is the command line I am using to run in parallel, which works fine on my local, Issue is only when I run through octopus tentacle. “xunit.runner.console\xunit.console.exe iCHealth.CProvider.UISpecTests.dll -maxthreads 5 -trait “Category=V8” -html “$($automationPath)\index.html””
In the logs I can’t see where the error System.NullReferenceException is being reported. Instead there are multiple instances of System.Exception : Driver is not initalised.
It will be difficult to diagnose the underlying cause from the custom code that is being run here, but it looks like you are running a WebDriver tests with Cucumber against Chrome?
Exception is because I am trying to put an if condition at the driver initialisation Level. Which is not happening when I run the tests from windows or when I run them sequentially. They are only happening when I run parallally through Octopus.
If you look at the log at the end One scenario is running without any exception while launching the driver. Which I have explained in my initial query(May be its not running in multi thread by the end, its leftover with only one feature file, and hence trying to execute).
Tried changing the Tentacle user account and still not helping
I just want to clarify my understanding of the code that is being run here.
You have a if statement in the C# code that is only being triggered when the code is run from Octopus and in parallel using xunit runner?
This if statement is causing the System.Exception : Driver is not initalised exceptions?
The System.Exception : Driver is not initalised exceptions are occurring on all but the last test?
If this is correct, then what logic is implemented in the if statement? It sounds like this is the root of the issue that is preventing the tests from running.
You have a if statement in the C# code that is only being triggered when the code is run from Octopus and in parallel using xunit runner? - YES
This if statement is causing the System.Exception : Driver is not initalised exceptions? -YES
The System.Exception : Driver is not initalised exceptions are occurring on all but the last test? - YES
Here is my Before Scenario where I am initializing the driver
[BeforeScenario]
public void BeforeScenario()
{
_driver = _initialiser.SetupWebDriver(Configuration.BrowserName);
_careProviderPageFactory = new CareProviderPageFactory(_driver);
_careProviderPageFactory.InitializePages();
_scenarioContext.Set(_careProviderPageFactory);
//I put this here to check if _driver is being initialised
if (_driver == null)
{
throw new Exception("Driver is not initalised");
}
_driver.Manage().Window.Maximize();
_initialiser.Open(Configuration.BrowserUrl);
//if (Configuration.PublishReports)
Logs = _extentReports.StartTest(_scenarioContext.ScenarioInfo.Tags[0] + "-" + _scenarioContext.ScenarioInfo.Title);
Logs.AssignCategory("Residents UI-Spec Test");
Logs.AddScreenCapture(Configuration.ReportDirectory);
}
And this is the initailizer class with setupwebdriver method, Am using Chrome for my automation
public IWebDriver SetupWebDriver(string browserType, int timeOut = 50)
{
try
{
browserType = browserType.ToUpper();
switch (browserType)
{
case "IE":
new DriverManager().SetUpDriver(new IEConfig());
webDriver = new InternetExplorerDriver();
objectContainer.RegisterInstanceAs(webDriver, typeof(IWebDriver));
break;
case "FIREFOX":
new DriverManager().SetUpDriver(new FirefoxConfig());
webDriver = new FirefoxDriver();
objectContainer.RegisterInstanceAs(webDriver, typeof(IWebDriver));
break;
//case "PHANTOMJS":
// new DriverManager().SetUpDriver(new PhantomConfig());
// Driver = new PhantomJSDriver();
// break;
case "CHROME":
var options = new ChromeOptions();
options.AddArgument("--disable-extensions");
new DriverManager().SetUpDriver(new ChromeConfig(), ConfigurationManager.AppSettings["ChromeDriverVersion"]);
webDriver = new ChromeDriver();
objectContainer.RegisterInstanceAs(webDriver, typeof(IWebDriver));
break;
default:
break;
}
webDriver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(timeOut);
if(webDriver == null)
{
throw new Exception("new driver is not created in initialiser");
}
return webDriver;
}
catch (Exception ex)
{
Log.WriteLog(LogLevel.Error, $"Failed to get global web driver instance : Error Message: {ex.Message}",
ex);
}
return webDriver;
}
There are a few points in that code that might lead to the Driver is not initalised exception. Given that you can’t step through the code when it is run by Octopus I would suggest adding a number of log statements inside the SetupWebDriver class to understand exactly what path the code is taking when being executed by Octopus.
In particular I would add a log message at the start to records what the value of browserType is, and assuming that it is Chrome then also adding log messages after each of the steps in the CHROME case.
I would also add log statements before each of the return statements to see at what point the value is returned.
It also looks like the _driver variable might be shared across threads? That may also cause some unpredictable results.
There are no configuration settings specific to running multiple browsers from Octopus. I can only suggest that running the multithreaded code is following a different execution path when run via Octopus, and that is leading to the exceptions you are seeing.
Unfortunately there is no simple answer I could offer to make the code you are trying to run work in the way you have described. It would seem that there is a solution, however it involves debugging custom multithreaded code which is beyond the scope of what we can offer through these support channels.
I see that you have added some additional log messages to the tests, so identifying any differences in the code paths would be a good place to start in determining how the code behaves differently in different environments.