The aim of this study is to search for a better optimization algorithm in applying unit root tests that inherit nonlinear models in the testing process. The algorithms analyzed include Broyden, Fletcher, Goldfarb and Shanno (BFGS), Gauss-Jordan, Simplex, Genetic, sequential quadratic programming and extensive grid-search. The simulation results indicate that the derivative free methods, such as Genetic and Simplex, have advantages over hill climbing methods, such as BFGS and Gauss-Jordan, in obtaining accurate critical values for the Leybourne et al. (J Time Ser Anal 19:83-97, 1998) (LNV) and Sollis (J Time Ser Anal 25:409-417, 2004) unit root tests. Besides, we extend our analysis by including exponential smooth transition type of trend function in to unit root testing which is not used in the previous literature. The same result also holds true for our newly proposed unit root test with exponential smooth transition function type of trend model. Furthermore, we realize that there is a gap in the unit root studies that the newly proposed tests are not analyzed between each other's data generating process (DGP). Hence, we investigate the power comparison of different nonlinear unit root test under various DGP including nonlinear unit root tests and find interesting results such as LNV type unit root test can manage to capture state dependent nonlinearity when the transition speed is high. Finally, we have used the Australian real interest rate parity hypothesis to empirically verify the results that we have obtained in the simulation studies.