On OSX, updating system nlopt causes R's nloptr to break

[GabrielHoffman]

On macOS Sonoma 14.7.1 with R 4.4.1, I started off with a normal working install of nloptr and lme4. I updated some system packages with brew, not expecting any effect on R packages. After updating, brew had installed nlopt 2.9.0. (The version number is confusing, but that is what brew says: https://formulae.brew.sh/formula/nlopt).

Then I noticed that lme4 was failing for simple examples. See bug report. After a lot of digging, re-installing nloptr resolved the issue.

My issue is resolved, but I want to post here since I was surprised that updating a system version of nlopt would break an R version of nloptr

Best,
Gabriel

[eddelbuettel]

This can happen (theoretically and practically) with dynamic linking but it is rare in practice as library interfaces and behavior tend to be stable. nloptr tries to straddle a balance of making installing easier and faster (by relying on a system library), other packages vendor. (And on other OSs you get static linking only....)

[eddelbuettel]

If you can distill out a minimally veriable reproducible example (ideally not involving lme4) we could augment the unit tests.

[jaganmn]

That you encountered a problem mixing CRAN and Homebrew binaries isn't particularly surprising. It's documented that if you obtain R for macOS from CRAN, then you must use Apple's native tools for package installation, linking libraries provided by (or built, using the same native tools, as documented by) CRAN at https://mac.r-project.org/bin/. Conversely, if you obtain R from Homebrew, then you ought to use tools and libraries from Homebrew. Your PATH, your ~/.R/Makevars, and files under /usr/local (in the default compiler and linker search paths) can all be contributing to contamination of your .libPaths().

https://cran.r-project.org/bin/macosx/

Binary libraries for dependencies not present here are available from http://mac.r-project.org/bin/ and corresponding sources at http://mac.r-project.org/src/.

https://cran.r-project.org/doc/manuals/r-release/R-admin.html#macOS-packages

Apple includes many Open Source libraries in macOS but increasingly without the corresponding headers (not even in Xcode nor the Command Line Tools): they are often rather old versions. If installing packages from source using them it is usually easiest to install a statically-linked up-to-date copy of the Open Source package from its sources or from https://mac.r-project.org/bin/.

https://mac.r-project.org/tools/

NOTE: In order to retain compatibility with native R we recommend using above tools. Although it is possible to compile R using tools from other package managers such as Homebrew, MacPorts or Fink, such binaries are by definition incompatible with macOS native libraries and applications. If you choose one of those package managers, make sure you compile everything using those tools including R and all packages and libraries you intend to use.

[jaganmn]

I've just installed (under macOS, from sources) nlopt versions 2.8.0, 2.9.0, and 2.9.1. Under each version, I re-installed nloptr then debugged lme4. The breakage happens in version 2.9.0 (https://github.com/stevengj/nlopt/releases/tag/v2.9.0), but I'll just show you a comparison of 2.8.0 and 2.9.1.

2.8.0:

> library(lme4)
Loading required package: Matrix
> debug(lme4::nloptwrap)
> fm01ML <- lmer(Yield ~ 1|Batch, Dyestuff, REML = FALSE)
debugging in: (function (par, fn, lower, upper, control = list(), ...) 
{
    for (n in names(defaultControl)) if (is.null(control[[n]])) 
        control[[n]] <- defaultControl[[n]]
    res <- nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, 
        opts = control, ...)
    with(res, list(par = solution, fval = objective, feval = iterations, 
        conv = if (status < 0 || status == 5) status else 0, 
        message = message))
})(fn = function (theta) 
.Call(lmer_Deviance, pp$ptr(), resp$ptr(), as.double(theta)), 
    par = 0.97876050342587, lower = 0, upper = Inf, control = list(
        print_level = 0))
debug: {
    for (n in names(defaultControl)) if (is.null(control[[n]])) 
        control[[n]] <- defaultControl[[n]]
    res <- nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, 
        opts = control, ...)
    with(res, list(par = solution, fval = objective, feval = iterations, 
        conv = if (status < 0 || status == 5) status else 0, 
        message = message))
}
Browse[1]> n
debug: for (n in names(defaultControl)) if (is.null(control[[n]])) control[[n]] <- defaultControl[[n]]
Browse[1]> n
debug: if (is.null(control[[n]])) control[[n]] <- defaultControl[[n]]
Browse[1]> f
debug: res <- nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, 
    opts = control, ...)
Browse[1]> n
debug: with(res, list(par = solution, fval = objective, feval = iterations, 
    conv = if (status < 0 || status == 5) status else 0, message = message))
Browse[1]> str(res)
List of 20
 $ x0                    : num 0.753
 $ eval_f                :function (x)  
 $ lower_bounds          : num 0
 $ upper_bounds          : num Inf
 $ num_constraints_ineq  : num 0
 $ eval_g_ineq           : NULL
 $ num_constraints_eq    : num 0
 $ eval_g_eq             : NULL
 $ options               :List of 18
  ..$ algorithm              : chr "NLOPT_LN_BOBYQA"
  ..$ stopval                : num -Inf
  ..$ ftol_rel               : num 0
  ..$ ftol_abs               : num 1e-08
  ..$ xtol_rel               : num 1e-04
  ..$ xtol_abs               : num 1e-08
  ..$ maxeval                : num 1e+05
  ..$ maxtime                : num -1
  ..$ tol_constraints_ineq   : num(0) 
  ..$ tol_constraints_eq     : num(0) 
  ..$ print_level            : num 0
  ..$ check_derivatives      : logi FALSE
  ..$ check_derivatives_tol  : num 1e-04
  ..$ check_derivatives_print: chr "all"
  ..$ print_options_doc      : logi FALSE
  ..$ population             : num 0
  ..$ vector_storage         : num 20
  ..$ ranseed                : num 0
 $ local_options         : NULL
 $ nloptr_environment    :<environment: 0x147ed5b20> 
 $ call                  : language nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, opts = control)
 $ termination_conditions: chr "xtol_abs: 1e-08\tftol_abs: 1e-08\tmaxeval: 1e+05"
 $ status                : int 4
 $ message               : chr "NLOPT_XTOL_REACHED: Optimization stopped because xtol_rel or xtol_abs (above) was reached."
 $ iterations            : int 20
 $ objective             : num 327
 $ solution              : num 0.753
 $ version               : chr "2.8.0"
 $ num.evals             : num 1
 - attr(*, "class")= chr "nloptr"

2.9.1:

> library(lme4)
Loading required package: Matrix
> debug(lme4::nloptwrap)
> fm01ML <- lmer(Yield ~ 1|Batch, Dyestuff, REML = FALSE)
debugging in: (function (par, fn, lower, upper, control = list(), ...) 
{
    for (n in names(defaultControl)) if (is.null(control[[n]])) 
        control[[n]] <- defaultControl[[n]]
    res <- nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, 
        opts = control, ...)
    with(res, list(par = solution, fval = objective, feval = iterations, 
        conv = if (status < 0 || status == 5) status else 0, 
        message = message))
})(fn = function (theta) 
.Call(lmer_Deviance, pp$ptr(), resp$ptr(), as.double(theta)), 
    par = 0.97876050342587, lower = 0, upper = Inf, control = list(
        print_level = 0))
debug: {
    for (n in names(defaultControl)) if (is.null(control[[n]])) 
        control[[n]] <- defaultControl[[n]]
    res <- nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, 
        opts = control, ...)
    with(res, list(par = solution, fval = objective, feval = iterations, 
        conv = if (status < 0 || status == 5) status else 0, 
        message = message))
}
Browse[1]> n
debug: for (n in names(defaultControl)) if (is.null(control[[n]])) control[[n]] <- defaultControl[[n]]
Browse[1]> n
debug: if (is.null(control[[n]])) control[[n]] <- defaultControl[[n]]
Browse[1]> f
debug: res <- nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, 
    opts = control, ...)
Browse[1]> n
debug: with(res, list(par = solution, fval = objective, feval = iterations, 
    conv = if (status < 0 || status == 5) status else 0, message = message))
Browse[1]> str(res)
List of 20
 $ x0                    : num 0.979
 $ eval_f                :function (x)  
 $ lower_bounds          : num 0
 $ upper_bounds          : num Inf
 $ num_constraints_ineq  : num 0
 $ eval_g_ineq           : NULL
 $ num_constraints_eq    : num 0
 $ eval_g_eq             : NULL
 $ options               :List of 18
  ..$ algorithm              : chr "NLOPT_LN_BOBYQA"
  ..$ stopval                : num -Inf
  ..$ ftol_rel               : num 0
  ..$ ftol_abs               : num 1e-08
  ..$ xtol_rel               : num 1e-04
  ..$ xtol_abs               : num 1e-08
  ..$ maxeval                : num 1e+05
  ..$ maxtime                : num -1
  ..$ tol_constraints_ineq   : num(0) 
  ..$ tol_constraints_eq     : num(0) 
  ..$ print_level            : num 0
  ..$ check_derivatives      : logi FALSE
  ..$ check_derivatives_tol  : num 1e-04
  ..$ check_derivatives_print: chr "all"
  ..$ print_options_doc      : logi FALSE
  ..$ population             : num 0
  ..$ vector_storage         : num 20
  ..$ ranseed                : num 0
 $ local_options         : NULL
 $ nloptr_environment    :<environment: 0x10e65fb20> 
 $ call                  : language nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, opts = control)
 $ termination_conditions: chr "xtol_abs: 1e-08\tftol_abs: 1e-08\tmaxeval: 1e+05"
 $ status                : int -2
 $ message               : chr "NLOPT_INVALID_ARGS: Invalid arguments (e.g. lower bounds are bigger than upper bounds, an unknown algorithm was"| __truncated__
 $ iterations            : int 0
 $ objective             : num Inf
 $ solution              : num 0.979
 $ version               : chr "2.9.1"
 $ num.evals             : num 1
 - attr(*, "class")= chr "nloptr"

An example independent of lme4 would be nice. It's too bad that the objective function in this case can't be cut and pasted:

Browse[1]> fn
function (theta) 
.Call(lmer_Deviance, pp$ptr(), resp$ptr(), as.double(theta))
<environment: 0x10b6c9628>

But there is more to investigate because the starting values of these two optimizations are somehow different. So the breakage is probably upstream of the call to nloptwrap that I've debugged. I'll dig a bit more later.

[jaganmn]

But there is more to investigate because the starting values of these two optimizations are somehow different. So the breakage is probably upstream of the call to nloptwrap that I've debugged. I'll dig a bit more later.

Actually, under nlopt 2.8.0, the value of par in the evaluation frame of lme4::nloptwrap differs before and after the call to nloptr::nloptr, suggesting that either nloptr or lme4 is failing to copy before modifying. Just trying the first example in help("nloptr") suggests to me that nloptr is not the culprit. My naive guess is that changes in nlopt 2.9.0 have exposed a low level "feature" of lme4. FYI @bbolker

debug: res <- nloptr(x0 = par, eval_f = fn, lb = lower, ub = upper, 
    opts = control, ...)
Browse[1]> par
[1] 0.9787605
Browse[1]> n
debug: with(res, list(par = solution, fval = objective, feval = iterations, 
    conv = if (status < 0 || status == 5) status else 0, message = message))
Browse[1]> par
[1] 0.7525807

[bbolker]

Yikes

…

On Thu, Dec 12, 2024, 10:54 AM Mikael Jagan ***@***.***> wrote: But there is more to investigate because the starting values of these two optimizations are somehow different. So the breakage is probably upstream of the call to nloptwrap that I've debugged. I'll dig a bit more later. Actually, under nlopt 2.8.0, the value of par in the evaluation frame of lme4::nloptwrap differs before and after the call to nloptr::nloptr, suggesting that either *nloptr* or *lme4* is failing to copy before modifying. Just trying the first example in help("nloptr") suggests to me that *nloptr* is not the culprit. My naive guess is that changes in nlopt 2.9.0 have exposed a low level behaviour of *lme4*. FYI @bbolker <https://github.com/bbolker> — Reply to this email directly, view it on GitHub <#173 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAATIRT7ZASGQ6A3FTBJJSD2FGWULAVCNFSM6AAAAABSC3GDT2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZZGM2TENZUGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[aadler]

For those of us less fluent in lme4, would someone kindly describe what the potential problem is and how the nloptr changes in 2.9.0 led to this, please?

[jaganmn]

Sorry, I've not had time to dig further and clarify things. I'd hoped that I'd left enough breadcrumbs. I'll try to look again soon. Luckily no fallout on CRAN yet, seemingly because the check machines are not yet getting nlopt >= 2.9. But I know that the Fedora machines are going to be upgraded to F40 soon, and nlopt >= 2.9 is being built there ...

https://packages.fedoraproject.org/pkgs/NLopt/NLopt-devel/

Debian seems slower:

https://packages.debian.org/trixie/libnlopt-dev

The Windows and macOS machines are using 2.8.0 and 2.7.1, respectively:

https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/ucrt3/toolchain_libs/mxe/src/nlopt.mk
https://github.com/R-macos/recipes/blob/master/recipes/nlopt

FreeBSD, though not tested by CRAN, is already at 2.9.1:

https://www.freshports.org/math/nlopt

[bbolker]

Entertainingly, https://nlopt.readthedocs.io/ (which certainly looks official) reports 2.7.1 as being the latest version ... the NEWS file doesn't list anything that looks like it should make a difference, but obviously you're seeing something real. I'll dig into this when I get a chance (hopefully, soon).

[aadler]

For Windows, maybe we can ask @kalibera for a version of Rtools with nlopt 2.9.1 and see if nloptr "survives"?

[eddelbuettel]

Or use a container and install nloptr from source, and have that used?

[bbolker]

Containers had occurred to me: I'll try this when I get a chance.

@eddelbuettel:

[jaganmn]

Check logs under today's R-devel, if it saves people time ... Both lme4 and nloptr show new problems when checked under nlopt 2.9.1. I'd still guess (based on the debugging output from December) that the new problems in lme4 are due to low level assumptions of lme4 itself, not due to any problem in nloptr.

00check.lme4_1.1-36_nlopt_2.8.0.log
00check.lme4_1.1-36_nlopt_2.9.1.log
00check.nloptr_2.1.1_nlopt_2.8.0.log
00check.nloptr_2.1.1_nlopt_2.9.1.log

[bbolker]

To summarize:

• nloptr + nlopt 2.8.0: OK
• nloptr + nlopt 2.9.1:

Error in nloptr(5, fn, lb = lb, ub = ub, opts = c(alg, ctl)) :
STRING_ELT() can only be applied to a 'character vector', not a 'NULL'
Calls: ... run_test_dir -> lapply -> FUN -> eval -> eval -> nloptr

• lme4 with 2.8.0: OK
• lme4 with 2.9.1: as in tests fail: 'lmer.Rnw' using 'UTF-8'... failed lme4/lme4#817

fm01ML <- lmer(Yield ~ 1|Batch, Dyestuff, REML = FALSE)
Warning in optwrap(optimizer, devfun, getStart(start, rho$pp), lower = rho$lower, :
convergence code -2 from nloptwrap: NLOPT_INVALID_ARGS: Invalid arguments (e.g. lower bounds are bigger than upper bounds, an unknown algorithm was specified, etcetera).
Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge with max|grad| = 1.01096 (tol = 0.002, component 1)
system.time(
tpr <- profile(fm01ML, optimizer="Nelder_Mead", which="beta_")
)## fast; as only one beta parameter is profiled over -> 0.09s (2022)
Error in profile.merMod(fm01ML, optimizer = "Nelder_Mead", which = "beta_") :
Profiling over both the residual variance and
fixed effects is not numerically consistent with
profiling over the fixed effects only (relative difference: 1);
consider adjusting devmatchtol

Running ‘vcov-etc.R’
Running the tests in 'tests/AAAtest-all.R' failed.
Last 13 lines of output:
AIC(x) not equal to 1763.939344.
1/1 mismatches
[1] Inf - 1764 == Inf
── Error ('test-summary.R:32:3'): lmer ─────────────────────────────────────────
Error in optwrap(optimizer, devfun, getStart(start, rho$pp), lower = rho$lower, control = control, adj = FALSE, verbose = verbose, ...): (converted from warning) convergence code -2 from nloptwrap: NLOPT_INVALID_ARGS: Invalid arguments (e.g. lower bounds are bigger than upper bounds, an unknown algorithm was specified, etcetera).
Backtrace:
▆
└─lme4::lmer(y ~ x.1 + x.2 + (1 + x.1 | g), control = C1) at test-summary.R:32:3
└─lme4::optimizeLmer(...)
└─lme4:::optwrap(...)

[aadler]

Yup, that's the test for PRAXIS, which was changed by @jschueller in stevengj/nlopt#528. Now we have to figure out WHAT was changed and how to fix it.

[bbolker]

sigh, that strengthens @jaganmn's argument that there is really something wrong in lme4 as well ...

[jaganmn]

Re: nloptr

It seems that parameter grad of the nlopt_func passed to nlopt_set_min_objective is newly non-NULL, even though NLOPT_LN_PRAXIS is a derivative-free algorithm and the NLopt documentation promises that grad is NULL for derivative-free algorithms. That seems to break nloptr here:

nloptr/src/nloptr.c

Lines 284 to 296 in 6d4943a

	// gradient
	if (grad) {
	// result needs to be a list in this case
	SEXP R_gradient;
	PROTECT( R_gradient = getListElement( result, "gradient" ) );
	// recode the return value from SEXP to double
	for (i=0;i<n;i++) {
	grad[i] = REAL( R_gradient )[i];
	}
	UNPROTECT( 1 );
	}

More precisely, when nloptr is built against nlopt >= 2.9, the branch condition can be true even where result is numeric rather than a list of the form list(objective, gradient), leading to erroneous use of STRING_ELT by getListElement. As a stopgap, maybe you need to start branching on grad && !isNumeric(result).

Applying that patch gets you past the original error while uncovering a bunch of other differences ...

* checking tests ...
  Running ‘tinytest.R’
 ERROR
Running the tests in 'tests/tinytest.R' failed.
Last 13 lines of output:
   diff| Attributes differ
  ----- FAILED[data]: test-wrapper-lbgfs.R<73--73>
   call| expect_identical(lbfgsTest$iter, lbfgsControl$iterations)
   diff| Expected '189', got '218'
  ----- FAILED[data]: test-wrapper-tnewton.R<43--43>
   call| expect_identical(tnTest$par, tnControl$solution)
   diff| Mean relative difference: 0.2434543
  ----- FAILED[data]: test-wrapper-tnewton.R<44--44>
   call| expect_identical(tnTest$value, tnControl$objective)
   diff| Expected '770.986244842934', got '412.195230372667'
  ----- FAILED[data]: test-wrapper-tnewton.R<45--45>
   call| expect_identical(tnTest$iter, tnControl$iterations)
   diff| Expected '1003', got '1004'
  Error: 59 out of 459 tests failed
  Execution halted

tinytest.Rout.fail.log

I would start by submitting to the NLopt maintainers a minimal reproducible example showing that grad can be non-NULL when it is documented to be NULL. If they agree that there was a breaking change, then maybe CRAN can be convinced to delay upgrades to NLopt ...

[aadler]

Hi, @jaganmn. May I request that you open the issue with nlopt, being that right now you have the clearest understanding of what happened?

[jaganmn]

The behaviour that I reported in my last comment was not reproducible independently of nloptr, leading me to discover that all of these problems are due to a mistake in nloptr's Autoconf machinery. In Makevars.in, you have

PKG_CPPFLAGS = -I../inst/include @NLOPT_CPPFLAGS@

implying that nloptr/inst/include appears earlier in the preprocessor search path than any path determined by configure, implying that the preprocessor gets nlopt.h from nloptr/inst/include instead of from any detected system installation, even when linking against a system libnlopt. It so happens that nlopt 2.9.0 removed an entry (NLOPT_LD_LBFGS_NOCEDAL) from an enumeration in nlopt.h listing all supported optimization algorithms. Users compiling nlopt.h from nlopt < 2.9 while linking against libnlopt from nlopt >= 2.9 were seeing symptoms of an off-by-one error: if they wanted Algorithm N, then they got Algorithm N+1.

I imagine that a suitable patch for configure.ac and Makevars.in will solve the problems seen in the check logs, also downstream for lme4. I see @astamm and @eddelbuettel in the git-blame, so I'll leave it to them.

Note that you use NLOPT_LD_LBFGS_NOCEDAL in nloptr/src/nloptr.c. That usage and all of your unit tests for that algorithm need to be conditioned on nlopt < 2.9. It's too bad that nlopt.h does not provide version macros. You might want to use configure to define your own.

[bbolker]

I imagine that a suitable patch for configure.ac and Makevars.in will solve the ... downstream [problems] for lme4

this is consistent at least with the NLOPT_INVALID_ARGS error ...

[eddelbuettel]

Wow, nice work. Part of it is clearly also

implying that the preprocessor gets nlopt.h from nloptr/inst/include instead of from any detected system installation, even when linking against a system libnlopt

Such mixing is almost always a bad idea so when system linking is done system headers should be used. I see no nlopt.h below inst/include/ (ie here) so I am not sure how the situation could have arisen.

[jaganmn]

It's in the tarball that I downloaded from CRAN. git-blame suggests maintainer thinko?

nloptr/.gitignore

Lines 2 to 4 in 6d4943a

	inst/include/nlopt.f
	inst/include/nlopt.h
	inst/include/nlopt.hpp

[eddelbuettel]

You are pointing at .gitignore, not the files. But maybe they are missing from .Rbuildignore. Is that what you are trying to say?

Because indeed, this looks like a mistake:

edd@rob:/tmp/r$ wget -q https://cloud.r-project.org/src/contrib/nloptr_2.1.1.tar.gz
edd@rob:/tmp/r$ tar tvzf nloptr_2.1.1.tar.gz | grep inst/include
drwxr-xr-x ligges/users       0 2022-01-20 01:19 nloptr/inst/include/
-rw-r--r-- ligges/users    3925 2024-06-24 10:47 nloptr/inst/include/nlopt.f
-rw-r--r-- ligges/users   15819 2024-06-24 10:47 nloptr/inst/include/nlopt.h
-rw-r--r-- ligges/users   22423 2024-06-24 10:47 nloptr/inst/include/nlopt.hpp
-rw-r--r-- ligges/users   18345 2022-01-20 01:19 nloptr/inst/include/nloptrAPI.h
edd@rob:/tmp/r$ 

[aadler]

The behaviour that I reported in my last comment was not reproducible independently of nloptr, leading me to discover that all of these problems are due to a mistake in nloptr's Autoconf machinery.

Wow. Thank you very much!!

Note that you use NLOPT_LD_LBFGS_NOCEDAL in nloptr/src/nloptr.c. That usage and all of your unit tests for that algorithm need to be conditioned on nlopt < 2.9. It's too bad that nlopt.h does not provide version macros. You might want to use configure to define your own.

Hmmm, that was a change in 2.9.0 <Dropped unused LD_LBFGS_NOCEDAL enum value>. Being that it was mooted, I vote that we simply check for that string and return an error. @astamm, this invalidates one of my PRs, but that's fine; I'll yank it. Enough is changing in 2.9+ that we should make sure it works as is before any tweaks for efficiency. EDIT Actually, it doesn't if I just remove that line and update the order, it should fall out as "unknown algorithm". Regardless, let's get this working first :)

@jaganmn, thank you again for your time and expertise!

[jaganmn]

You are pointing at .gitignore, not the files. But maybe they are missing from .Rbuildignore. Is that what you are trying to say?

More or less. I just meant that I can see how the files might have been overlooked at the time, given that git was asked to ignore them and that they are located in a directory that R CMD check tends to leave alone.

[aadler]

@astamm, we should reopen this issue until the configuration and anything else necessary is completed.

[astamm]

It seems a lot has been discussed already. I agree that priority is to make nloptr work as is with latest nlopt version. Then, I'll get to the other PRs. I have a larger bandwidth in the next months to look into all this.

[aadler]

Interestingly, the version of nlopt in Rtools44 is 2.8.0 per the news. I emailed R-sig-windows asking that it not be updated beyond 2.8.0 until we work this out.

[aadler]

This may help us. I'd prefer someone who knows C better than I work on it, but if no one else will, I will try:

https://github.com/jump-dev/NLopt.jl/pull/251/files

[aadler]

The nlopt project reversed that commit!!

stevengj/nlopt#588

@jaganmn, may I trouble to run the test that failed and see if nlopt 2.10 passes now?

[eddelbuettel]

The nlopt project reversed that commit!!

Ace! That is good news.

We should however also clean up the issue uncovered by @jaganmn (see comment above) and not leave 'random' nlopt header files around which was part of the hickup we saw here. Header and library version must mix, keeping one half around asks for trouble.

[jaganmn]

Yes, I'll try to do that this week some time. But I should repeat that, strictly speaking, the ABI-breaking change in nlopt would not have broken nloptr so badly (if at all) if not for the bug in nloptr that I described. Specifically, PKG_CPPFLAGS should not contain -I../inst/include, and the tarball submitted to CRAN should not contain static, extraneous headers under inst/include.

[eddelbuettel]

Exactly correct statement by @jaganmn