KESTREL - Remote Solver Execution on NEOS Servers

Table of Contents

Background

The Kestrel client/server is a way of sending your GAMS solve job to be solved via the NEOS Server from within your usual GAMS modeling environment and receiving results that can be processed as with any local solver. Starting with distribution 23.7 the Kestrel solver is part of the GAMS distribution. The solve statement using the GAMS/KESTREL solver invokes a client program that sends your problem to a solver running on one of the NEOS Server's remote computers. The results from the NEOS Server are eventually returned through Kestrel to GAMS, where you can view and manipulate them locally in the usual way. The introduction below covers everything you need to know to start using GAMS/KESTREL. Further information about more advanced features and other uses of Kestrel can be found at the Kestrel page on the NEOS Server. Please also make you familiar with the NEOS optimization service terms of use before submitting jobs via GAMS/KESTREL.

Kestrel and IBM DOcloud

With GAMS 24.6 the GAMS/Kestrel client has been adjusted to work with IBM's cloud service Decision Optimization on Cloud (DOcloud). When the Kestrel option docloudurl is set, the generated model instance will be automatically translated into an MPS format and send to DOcloud and solved by Cplex in IBM's cloud. So no GAMS/Cplex license is required on your local machine. You as a user need to register with IBM on the web page mentioned about and pay IBM for the solution time spend in IBM's cloud. IBM provides details how to get started with DOcloud.

Using GAMS/KESTREL

The Kestrel solver can be used to solve a GAMS model remotely. For example, consider the trnsport model. It can be solved locally in GAMS through the following statements, 

model transport /all/;
solve transport using lp minimizing z;

which specify the trnsport model and solve it with the default linear programming solver. We can add an option statement to the code to explicitly specify the solver. For example, if we change the linear programming solver to GAMS/MINOS, the code becomes 

model transport /all/;
option lp=minos;
solve transport using lp minimizing z;

To solve the same problem remotely through the NEOS Server, we simply change the linear programming solver to Kestrel. In addition we have to provide an email address. This can be specified either in an option file or by setting the environment variable NEOS_EMAIL. In order to set this permanently, it is recommended to add NEOS_EMAIL to your gamsconfig.yaml file.

Without an option file Kestrel will submit the model instance to the NEOS server and use the default LP solver on NEOS. 

option lp=kestrel;
solve transport using lp minimizing z;

We can support a Kestrel option file and determine the solver on the NEOS server: 

transport.optfile=1;
option lp=kestrel;
solve transport using lp minimizing z;
$echo email jdoe@jeangreyhigh.edu > kestrel.opt
$echo kestrel_solver minos >> kestrel.opt

The statement transport.optfile=1 specifies that an options file, called kestrel.opt, will be used. The options file contains the email address, the remote solver name as well as any options for the remote solver. We instruct the Kestrel solver to use GAMS/MINOS as the remote solver by writing the following kestrel.opt file: 

email jdoe@jeangreyhigh.edu
kestrel_solver minos

If you do not know what solvers are available via GAMS/KESTREL on NEOS, submitting a job with a nonexistent solver set will return a list of enabled solvers. If you want to change the URL of the server, you can specify the option neos_server. The complete format of the parameter is protocol://host:port.

A subsequent run of the code through the GAMS interpreter results in the transport model being solved through the NEOS Server with the GAMS/MINOS solver. Once the job is submitted to the NEOS Server, a job number, password, and Web address are displayed to the screen, which provide information on accessing the job and viewing the intermediate output, for example, 

--- Executing KESTREL
Job has been submitted to Kestrel
Kestrel/NEOS Job number    : 93478
Kestrel/NEOS Job password  : utiwtxTK
Check the following URL for progress report :
     https://neos-server.org/neos/cgi-bin/nph-neos-solver.cgi?admin=results&jobnumber=93478&pass=utiwtxTK

To look at the solver's output while it is running, point your browser at the URL given in the Kestrel output as shown above, and click on View Intermediate Results in the web page that appears. This will take you to another page that shows all of the output produced by the solver for your problem so far. To track the solver's progress, simply update this page periodically.

If the NEOS Server or the network becomes unavailable after the submission, a particular job can be retrieved by setting both the kestrel job and kestrel password in the options file. 

kestrel_solver minos
kestrel_job 93478
kestrel_password utiwtxTK

Re-issuing the command gams trnsport with this options file will retrieve the results for the specified job number.

By specifying neos_username and neos_user_password in the option file, you can submit an authenticated job using your NEOS user account. Authenticated jobs will appear in your user account on the NEOS website. 

neos_username username
neos_user_password password

Using GAMS/KESTREL with IBM DOcloud

In order to use GAMS/Kestrel to solve model instances with Cplex on IBM's DOcloud, the only thing that needs to be done is to provide a different option file. The option docloud_url triggers the submission to IBM's DOcloud rather than NEOS. You need an account on DOcloud (and pay IBM for the Cplex solution time in their cloud). Some additional account information needs to be specified in the Kestrel option docloud_key. The use of Cplex in DOcloud is limited to solve a model instance, special options that are available in the GAMS/Cplex system, like IIS or FeasOpt cannot be used (yet) through Kestrel/DOcloud. Moreover, for problems with discrete variables, only the primal solution is returned. If you want to specify Cplex options this needs to be done with a Cplex parameter file which is quite different from the GAMS/Cplex option file. You can use a GAMS/Cplex option file and have the additional option WriteParam to produce such a Cplex parameter file. The Cplex parameter file might have a few more options compared to the GAMS/Cplex option file because some GAMS options set Cplex parameters. Please note that you can produce the Cplex parameter file with GAMS/Cplex even if you do not have a license for GAMS/Cplex as long as you run a demo sized model (e.g. from the GAMS Model Library) to produce the Cplex parameter file. The following GAMS/Cplex optfile 

lpmethod 4
advind 0
eprhs 1e-4
scaind 1
writeparam cplex.prm

produces the Cplex parameter file cplex.prm which can be submitted to Kestrel/DOcloud via the option docloud_prmfile: 

CPLEX Parameter File Version 12.6.3.0
CPX_PARAM_ADVIND                 0
CPX_PARAM_EPRHS                  1.00000000000000e-04
CPX_PARAM_SIMDISPLAY             2
CPX_PARAM_SCAIND                 1
CPX_PARAM_TILIM                  1.00000000000000e+03
CPX_PARAM_LPMETHOD               4
CPX_PARAM_THREADS                1
CPX_PARAM_PARALLELMODE           1

Users of GAMS/Kestrel on platforms other than Windows, Linux, or macOS have to manually provide a Python 3 installation that has access to the following modules: