GAMS control is a sub-module of the Python API that allows for full control over the GAMS system (data, model instances, and solving). It can be used in conjunction with other Python API sub-modules (numpy and transfer) to boost performance when pushing/pulling data to/from a GAMS model. The gams.control package provides objects to interact with the General Algebraic Modeling System (GAMS). Objects in this package allow convenient exchange of input data and model results (GamsDatabase), help to create and run GAMS models (GamsJob), that can be customized by GAMS options (GamsOptions). Furthermore, it introduces a way to solve a sequence of closely related model instances in the most efficient way (GamsModelInstance).

A GAMS program can include other source files (e.g. $include), load data from GDX files (e.g. $GDXIN or execute_load), and create PUT files. All these files can be specified with a (relative) path and therefore an anchor into the file system is required. The base class GamsWorkspace manages the anchor to the file system. If external file communication is not an issue in a particular Python application, temporary directories and files will be managed by objects in the namespace.

With the exception of GamsWorkspace the objects in the gams.control package cannot be accessed across different threads unless the instance is locked. The classes themself are thread safe and multiple objects of the class can be used from different threads (see below for restrictions on solvers that are not thread safe within the GamsModelInstance class).

Note: If you use multiple instances of the GamsWorkspace in parallel, you should avoid using the same working directory. Otherwise you may end up with conflicting file names.

The GAMS control Python API lacks support for the following GAMS components: Acronyms, support for GAMS compilation/execution errors (GamsJob.run just throws an exception), structured access to listing file, and proper support for solver options.

Currently only Cplex, Gurobi, and SoPlex fully utilize the power of solving GamsModelInstances. Some solvers will not even work in a multi-threaded application using GamsModelInstances. For some solvers this is unavoidable because the solver library is not thread safe (e.g. MINOS), other solvers are in principle thread safe but the GAMS link is not (e.g. SNOPT). Moreover, GamsModelInstances are not available for quadratic model types (QCP, MIQCP, RMIQCP).

Recommended Import

GAMS control is available with the following import statement once the API has been installed:

>>> import gams

Other sub-modules must be imported with separate import statements.

Specifying a GAMS System Directory

There are several ways to specify which system directory should be used. On all platforms, the system directory can be specified in the GamsWorkspace constructor. If no system directory is specified by the user, The API tries to find one automatically:

Windows: Try to find a system directory in the Windows registry.
Linux: Try to find a system directory in the PATH first. If none was found, search LD_LIBRARY_PATH.
OS X: Try to find a system directory in the PATH first. If none was found, search DYLD_LIBRARY_PATH.

The environment variable PATH can be set as follows on Linux and macOS:

export PATH=<Path/To/GAMS>:$PATH

GAMS

Note: On Linux and macOS it is recommended to specify the PATH only instead of (DY)LD_LIBRARY_PATH since this might cause problems loading the correct version of certain modules (e.g. gdx).

Important Classes of the API

This section provides a quick overview of some fundamental classes of the GAMS control API. Their usage is demonstrated by an extensive set of examples in the How to use the API section.

gams::control::workspace::GamsWorkspace Class
gams::control::execution::GamsJob Class
gams::control::database::GamsDatabase Class
gams::control::options::GamsOptions Class
gams::control::execution::GamsModelInstance Class

How to use the API

The GAMS distribution provides several examples that illustrate the usage of the API. [GAMSDIR]\api\python\examples\control contains multiple examples dealing with the well-known transportation problem. In further course of this tutorial we discuss these examples step by step and introduce new elements of the API in detail.

We recommend to open the aforementioned files to gain a complete overview of the examples. Down below we explain the examples with the help of selected code snippets.

How to import packages/modules from the GAMS control API (transport1.py)

Before we can start using the GAMS control API, it needs to be installed by following the instructions from the Getting Started section. Afterwards we can use the API by importing the GamsWorkspace class like this:

from gams import GamsWorkspace

Conventional Python packages/modules can by imported like that:

import os

import sys

How to choose the GAMS system (transport1.py)

By default the GAMS system is determined automatically. In case of having multiple GAMS systems on your machine, the desired system can be specified via an additional argument when the workspace is created. If we type python transport1.py C:/GAMS/42 we use GAMS 42 to run transport1.py even if our default GAMS system might be a different one. This is managed by the following code:

...
sys_dir = sys.argv[1] if len(sys.argv) > 1 else None
ws = GamsWorkspace(system_directory=sys_dir)
...

How to export data to GDX (transport_gdx.py)

Although the GAMS control Python API offers much more than exchanging data between Python and GDX, a common use case is the export and import of GDX files. The central class for this purpose is GamsDatabase. We assume that the data to be exported is available in Python data structures.

...
    plants = ["Seattle", "San-Diego"]
    markets = ["New-York", "Chicago", "Topeka"]
    capacity = {"Seattle": 350.0, "San-Diego": 600.0}
    demand = {"New-York": 325.0, "Chicago": 300.0, "Topeka": 275.0}
    distance = {
        ("Seattle", "New-York"): 2.5,
        ("Seattle", "Chicago"): 1.7,
        ("Seattle", "Topeka"): 1.8,
        ("San-Diego", "New-York"): 2.5,
        ("San-Diego", "Chicago"): 1.8,
        ("San-Diego", "Topeka"): 1.4,
    }
...

Different GAMS symbols are represented using different Python data structures. The data for the GAMS sets is represented using Python lists of strings (e.g. plants and markets). On the other hand, GAMS parameters are represented by Python dictionaries (e.g. capacity and demand). Note that the representation of the two dimensional parameter distance uses Python tuples for storing the keys. The choice of data structures can also be different, but the used structures in this example fit well for representing GAMS data with standard Python data structures.

A new GamsDatabase instance can be created using GamsWorkspace.add_database.

...
    # create new GamsDatabase instance
    db = ws.add_database()
...

We start adding GAMS sets using the method GamsDatabase.add_set which takes the name and the dimension as arguments. The third argument is an optional explanatory text. A for-loop iterates through plants and adds new records to the recently created GamsSet instance i using GamsSet.add_record.

...
    # add 1-dimensional set 'i' with explanatory text 'canning plants' to the GamsDatabase
    i = db.add_set("i", 1, "canning plants")
    for p in plants:
        i.add_record(p)
...

GamsParameter instances can be added by using the method GamsDatabase.add_parameter. It has the same signature as GamsDatabase.add_set. Anyhow, in this example we use GamsDatabase.add_parameter_dc instead which takes a list of GamsSet instances instead of the dimension for creating a parameter with domain information.

...
    # add parameter 'a' with domain 'i'
    a = db.add_parameter_dc("a", [i], "capacity of plant i in cases")
    for p in plants:
        a.add_record(p).value = capacity[p]
...

As soon as all data is prepared in the GamsDatabase, the method GamsDatabase.export can be used to create a GDX file.

...
    # export the GamsDatabase to a GDX file with name 'data.gdx' located in the 'working_directory' of the GamsWorkspace
    db.export("data.gdx")
...

How to import data from GDX (transport_gdx.py)

Data can be imported from a GDX file using GamsWorkspace.add_database_from_gdx. The method takes a path to a GDX file and creates a GamsDatabase instance.

...
    # add a new GamsDatabase and initialize it from the GDX file just created
    db2 = ws.add_database_from_gdx("data.gdx")
...

Reading the data from the GamsSet i into a list can be done as follows:

...
    # read data from symbols into Python data structures
    i = [rec.keys[0] for rec in db2["i"]]
...

A Python list is created using list comprehensions. i is retrieved by querying the GamsDatabase db2. The returned GamsSet object can be iterated using a for-loop to access the records of the set. Each record is of type GamsSetRecord and can be asked for its keys.

You can do the same for GamsParameters. Instead of creating a Python list, we want to have the data in the form of a Python dictionary. GamsParameterRecords can not only be asked for their keys, but also for their value. The following code snippet shows how to read the one dimensional parameter a into a Python dictionary using dict comprehensions.

...
    a = {rec.keys[0]: rec.value for rec in db2["a"]}
...

For multi dimensional symbols, we choose the Python dictionary keys to be tuples instead of string. We access the keys as usual, but do not address a specific key. Instead, we take the whole list of keys and turn it into a tuple.

...
    d = {tuple(rec.keys): rec.value for rec in db2["d"]}
...

Scalars can be read into a Python identifier by accessing the value of the first and only record.

...
    f = db2["f"].first_record().value
...

How to run a GamsJob from file (transport1.py)

At first we create our workspace using ws = GamsWorkspace(). Afterwards we load the trnsport model from the GAMS model library which puts the corresponding gms file in our working directory. Note that you can create a GamsJob with any other gms file you might have created on your own as long as it is located in the current working directory. Then the GamsJob job can be defined using the add_job_from_file method and afterwards we run the job.

...
    ws.gamslib("trnsport")
    job = ws.add_job_from_file("trnsport.gms")
    job.run()
...

How to retrieve a solution from an output database (transport1.py)

The following lines create the solution output and illustrate the usage of the GamsJob.out_db property to get access to the GamsDatabase created by the run method. To retrieve the content of variable x we use squared brackets that internally call the get_symbol method.

...
    for rec in job.out_db["x"]:
        print(
            f"x({rec.key(0)},{rec.key(1)}): level={rec.level} marginal={rec.marginal}"
        )
...

Note that instead of using the squared brackets we could also use

...
    for rec in job.out_db.get_symbol("x"):
...

How to specify the solver using GamsOptions (transport1.py)

The solver can be specified via the GamsOptions class and the GamsWorkspace.add_options method. The GamsOptions.all_model_types property sets xpress as default solver for all model types that can be handled by the solver. Then we run our GamsJob job with the new GamsOption.

...
    opt = ws.add_options()
    opt.all_model_types = "xpress"
    job.run(opt)
...

How to run a job with a solver option file and capture its log output (transport1.py)

At first we create the file xpress.opt with content algorithm=barrier which will be used as solver option file and is stored in the current working directory. Afterwards we use a GamsOption just like in the preceding example and set GamsOption.optfile property to 1 to tell the solver to look for a solver option file. In addition, we specify the argument output in order to write the log of the GamsJob into the file transport1_xpress.log.

...
    with open(os.path.join(ws.working_directory, "xpress.opt"), "w") as file:
        file.write("algorithm=barrier")
    opt.optfile = 1
    with open("transport1_xpress.log", "w") as log:
        job.run(opt, output=log)
...

Instead of writing the log output to a file, any object that provides the functions write and flush can be used. In order to write the log directly to stdout, we can use the following code:

...
    job.run(opt, output=sys.stdout)
...

How to use include files (transport2.py)

In this example, as in many succeeding, the data text and the model text are separated into two different strings. Note that these strings accessed via GAMS_DATA and GAMS_MODEL are using GAMS syntax. At first we write an include file tdata.gms that contains the data but not the model text and save it in our current working directory.

...
    with open(os.path.join(ws.working_directory, "tdata.gms"), "w") as file:
        file.write(GAMS_DATA)
...

Afterwards we create a GamsJob using the GamsWorkspace.add_job_from_string method. GamsOptions.defines is used like the 'double dash' GAMS parameters, i.e. it corresponds to --incname=tdata on the command line where incname is used as name for the include file in GAMS_MODEL as shown below.

...
    job = ws.add_job_from_string(GAMS_MODEL)
    opt = ws.add_options()
    opt.defines["incname"] = "tdata"
    job.run(opt)
...

The string GAMS_MODEL contains the following lines to read in the data.

...
$if not set incname $abort 'no include file name for data file provided'
$include %incname%
...

How to read data from string and export to GDX (transport3.py)

We read the data from the string GAMS_DATA. Note that this contains no model but only data definition in GAMS syntax. By running the corresponding GamsJob a GamsDatabase is created that is available via the GamsJob.out_db property. We can use the GamsDatabase.export method to write the content of this database to a GDX file tdata.gdx in the current working directory.

...
    job = ws.add_job_from_string(GAMS_DATA)
    job.run()
    job.out_db.export(os.path.join(ws.working_directory, "tdata.gdx"))
...

How to run a job using data from GDX (transport3.py)

This works quite similar to the usage of an include file explained in How to use include files (transport2.py).

...
    job = ws.add_job_from_string(GAMS_MODEL)
    opt = ws.add_options()
    opt.defines["gdxincname"] = "tdata"
    opt.all_model_types = "xpress"
    job.run(opt)
...

Note that there are some minor changes in GAMS_MODEL compared to preceding examples due to the usage of a GDX instead of an include file.

...
$if not set gdxincname $abort 'no include file name for data file provided'
$gdxIn %gdxincname%
$load i j a b d f
$gdxIn
...

How to run a job using implicit database communication (transport3.py)

This example does basically the same as the two preceding examples together. We create two GamsJobs job_data and job_model where the first one contains only the data and the second one contains only the model without data. After running job_data the corresponding out_db can be read in directly just like a GDX file. Note that the database needs to be passed to the GamsJob.run method as additional argument.

...
    job_data = ws.add_job_from_string(GAMS_DATA)
    job_model = ws.add_job_from_string(GAMS_MODEL)
    job_data.run()
    opt.defines["gdxincname"] = job_data.out_db.name
    job_model.run(opt, databases=job_data.out_db)
...

How to define data using Python data structures (transport4.py)

We use Python lists to define the sets and Python dictionaries for the parameter definition.

...
    plants = ["Seattle", "San-Diego"]
    markets = ["New-York", "Chicago", "Topeka"]
    capacity = {"Seattle": 350.0, "San-Diego": 600.0}
    demand = {"New-York": 325.0, "Chicago": 300.0, "Topeka": 275.0}
    distance = {
        ("Seattle", "New-York"): 2.5,
        ("Seattle", "Chicago"): 1.7,
        ("Seattle", "Topeka"): 1.8,
        ("San-Diego", "New-York"): 2.5,
        ("San-Diego", "Chicago"): 1.8,
        ("San-Diego", "Topeka"): 1.4,
    }
...

How to prepare a GamsDatabase from Python data structures (transport4.py)

At first we create an empty GamsDatabase db using the GamsWorkspace.add_database method. Afterwards we prepare the database. To add a set to the database we use the GamsSet class and the GamsDatabase.add_set method with arguments describing the identifier, dimension and explanatory text. To add the records to the database we iterate over the elements of our Python data structure and add them by using the GamsSet.add_record method.

For parameters the procedure is pretty much the same. Note that the table that specifies the distances in GAMS can be treated as parameter with dimension 2.

The GamsJob can be run like explained in the preceding example How to run a job using implicit database communication (transport3.py).

...
    db = ws.add_database()
 
    i = db.add_set("i", 1, "canning plants")
    for p in plants:
        i.add_record(p)
 
    j = db.add_set("j", 1, "markets")
    for m in markets:
        j.add_record(m)
 
    a = db.add_parameter_dc("a", [i], "capacity of plant i in cases")
    for p in plants:
        a.add_record(p).value = capacity[p]
 
    b = db.add_parameter_dc("b", [j], "demand at market j in cases")
    for m in markets:
        b.add_record(m).value = demand[m]
 
    d = db.add_parameter_dc("d", [i, j], "distance in thousands of miles")
    for k, v in distance.items():
        d.add_record(k).value = v
 
    f = db.add_parameter("f", 0, "freight in dollars per case per thousand miles")
    f.add_record().value = 90
 
    job = ws.add_job_from_string(GAMS_MODEL)
    opt = ws.add_options()
    opt.defines["gdxincname"] = db.name
    opt.all_model_types = "xpress"
    job.run(opt, databases=db)
...

How to initialize a GamsCheckpoint by running a GamsJob (transport5.py)

The following lines of code conduct several operations. While the first line simply creates a GamsCheckpoint, the second one uses the GamsWorkspace.add_job_from_string method to create a GamsJob containing the model text and data but no solve statement. Afterwards the run method gets the GamsCheckpoint as argument. That means the GamsCheckpoint cp captures the state of the GamsJob.

...
    cp = ws.add_checkpoint()
    job = ws.add_job_from_string(GAMS_MODEL)
    job.run(checkpoint=cp)
...

How to initialize a GamsJob from a GamsCheckpoint (transport5.py)

Note that the string GAMS_MODEL contains the entire model and data definition plus an additional demand multiplier and scalars for model and solve status but no solve statement:

...
   bmult 'demand multiplier' / 1 /;
...
   demand(j) 'satisfy demand at market j';
...
Scalar ms 'model status', ss 'solve status';
...

In transport5.py we create a list with eight different values for this demand multiplier.

...
    bmult = [0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3]
...

For each entry of that list we create a GamsJob using the GamsWorkspace.add_job_from_string method. Besides another string which resets the demand multiplier bmult, specifies the solve statement and assigns values to the scalars ms and ss we pass the checkpoint cp as additional argument. This results in a GamsJob combined from the checkpoint plus the content provided by the string.

We run the GamsJob and print some interesting data from the out_db.

...
    for b in bmult:
        job = ws.add_job_from_string(
            f"bmult={b}; solve transport min z use lp; ms=transport.modelstat; ss=transport.solvestat;",
            cp,
        )
        job.run()
        print(f"Scenario bmult={b}:")
        print(f"  Modelstatus: {job.out_db['ms'].find_record().value}")
        print(f"  Solvestatus: {job.out_db['ss'].find_record().value}")
        print(f"  Obj: {job.out_db['z'].find_record().level}")
...

NOTE: Some of the demand multipliers cause infeasibility. Nevertheless, GAMS keeps the incumbent objective function value. Therefore the model status and the solve status provide important information for a correct solution interpretation.

How to run multiple GamsJobs in parallel using a GamsCheckpoint (transport6.py)

With the exception of GamsWorkspace the objects in the gams.control package cannot be accessed across different threads unless the instance is locked. The classes themselves are thread safe and multiple objects of the class can be used from different threads (see below for restrictions on solvers that are not thread safe within the GamsModelInstance class).

Note: If you use multiple instances of the GamsWorkspace in parallel, you should avoid using the same working directory. Otherwise you may end up with conflicting file names.

Currently only Cplex, Gurobi, and SoPlex fully utilize the power of solving GamsModelInstances. Some solvers will not even work in a multi-threaded application using GamsModelInstances. For some solvers this is unavoidable because the solver library is not thread safe (e.g. MINOS), other solvers are in principle thread safe but the GAMS link is not (e.g. SNOPT). Moreover, GamsModelInstances are not available for quadratic model types (QCP, MIQCP, RMIQCP).

This example illustrates how to run the jobs known from transport5.py in parallel. We initialize the GamsCheckpoint cp and introduce a demand multiplier as we did before:

...
    cp = ws.add_checkpoint()
    job = ws.add_job_from_string(GAMS_MODEL)
    job.run(checkpoint=cp)
    bmult = [0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3]
...

Furthermore, we introduce a lock object io_lock that will be used to avoid mixed up output from the parallel jobs. We create one scenario for each entry of bmultlist and cause a thread to begin execution.

...
    io_lock = Lock()
    threads = {}
    for b in bmult:
        threads[b] = Thread(target=run_scenario, args=(ws, cp, io_lock, b))
        threads[b].start()
    for b in bmult:
        threads[b].join()
...

In function run_scenario a GamsJob is created and run just like in the preceding example of transport5.py. The output section is also the same except for the fact that it is 'locked' by the object io_lock which means that the output section cannot be executed simultaneously for multiple demand multipliers.

...
def run_scenario(workspace, checkpoint, io_lock, b):
    job = workspace.add_job_from_string(
        f"bmult={b}; solve transport min z use lp; ms=transport.modelstat; ss=transport.solvestat;",
        checkpoint,
    )
    job.run()
    # we need to make the ouput a critical section to avoid messed up report informations
    io_lock.acquire()
    print(f"Scenario bmult={b}:")
    print(f"  Modelstatus: {job.out_db['ms'].first_record().value}")
    print(f"  Solvestatus: {job.out_db['ss'].first_record().value}")
    print(f"  Obj: {job.out_db['z'].first_record().level}")
    io_lock.release()
...

While the output in transport5.py is strictly ordered subject to the order of the elements of bmult in transport6.py the output blocks might change their order but the blocks describing one scenario are still appearing together due to the io_lock object.

How to create a GamsModelInstance from a GamsCheckpoint (transport7.py)

In transport7.py the usage of GamsModelInstance is demonstrated.

At first checkpoint cp is created as in the preceding examples. Note that the GamsJob job again contains no solve statement and the demand multiplier is already included with default value 1. We create the GamsModelInstance mi using the GamsCheckpoint.add_modelinstance method.

...
    cp = ws.add_checkpoint()
    job = ws.add_job_from_string(GAMS_MODEL)
    job.run(checkpoint=cp)
    mi = cp.add_modelinstance()
...

How to modify a parameter of a GamsModelInstance using GamsModifier (transport7.py)

A GamsModelInstance uses a sync_db to maintain the data. We define bmult as GamsParameter using the GamsDatabase.add_parameter method and specify gurobi as solver. Afterwards the GamsModelInstance is instantiated with 3 arguments, the solve statement, GamsModifier bmult and GamsOptions opt. The GamsModifier means that bmult is modifiable while all other parameters, variables and equations of GamsModelInstance mi stay unchanged. We use the GamsParameter.add_record method to assign a value to bmult. That value can be varied afterwards using the GamsParameter.first_record method to reproduce our well-known example with different demand multipliers.

...
    bmult = mi.sync_db.add_parameter("bmult", 0, "demand multiplier")
    opt = ws.add_options()
    opt.all_model_types = "cplex"
 
    mi.instantiate("transport use lp min z", GamsModifier(bmult), opt)
 
    bmult.add_record().value = 1.0
    bmult_list = [0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3]
 
    for b in bmult_list:
        bmult.first_record().value = b
        mi.solve()
        print(f"Scenario bmult={b}:")
        print(f"  Modelstatus: {mi.model_status}")
        print(f"  Solvestatus: {mi.solver_status}")
        print(f"  Obj: {mi.sync_db['z'].first_record().level}")
...

How to modify a variable of a GamsModelInstance using GamsModifier (transport7.py)

We create a GamsModelInstance using the GamsCheckpoint.add_modelinstance method. Afterwards we define x as GamsVariable and a GamsParameter xup that will be used as upper bound for x. At the following instantiate method GamsModifier has 3 arguments. The first one says that x is modifieable, the second determines which part of the variable (lower bound, upper bound or level) can be modified and the third specifies the GamsParameter that holds the new value, in this case xup.

In the following loops we set the upper bound of one link of the network to zero, which means that no transportation between the corresponding plant and market is possible, and solve the modified transportation problem.

...
    mi = cp.add_modelinstance()
    x = mi.sync_db.add_variable("x", 2, VarType.Positive)
    xup = mi.sync_db.add_parameter("xup", 2, "upper bound on x")
 
    # instantiate the GamsModelInstance and pass a model definition and GamsModifier to declare upper bound of x mutable
    mi.instantiate("transport use lp min z", GamsModifier(x, UpdateAction.Upper, xup))
    mi.solve()
 
    for i in job.out_db["i"]:
        for j in job.out_db["j"]:
            xup.clear()
            xup.add_record((i.key(0), j.key(0))).value = 0
            mi.solve()
            print(f"Scenario link blocked: {i.key(0)} - {j.key(0)}")
            print(f"  Modelstatus: {mi.model_status}")
            print(f"  Solvestatus: {mi.solver_status}")
            print(f"  Obj: {mi.sync_db['z'].find_record().level}")
...

How to use a queue to solve multiple GamsModelInstances in parallel (transport8.py)

We initialize a GamsCheckpoint cp from a GamsJob. Then we define a list that represents the different values of the demand multiplier. That list will be used like a queue where we extract the last element first. The objects list_lock and io_lock are used later to avoid multiple reading of one demand multiplier and messed up output. Then we call function scen_solve multiple times in parallel. The number of parallel calls is specified by nr_workers.

...
    cp = ws.add_checkpoint()
 
    job = ws.add_job_from_string(GAMS_MODEL)
    job.run(checkpoint=cp)
 
    bmult_list = [1.3, 1.2, 1.1, 1.0, 0.9, 0.8, 0.7, 0.6]
 
    list_lock = Lock()
    io_lock = Lock()
 
    # start 2 threads
    nr_workers = 2
    threads = {}
    for i in range(nr_workers):
        threads[i] = Thread(
            target=scen_solve, args=(cp, bmult_list, list_lock, io_lock)
        )
        threads[i].start()
    for i in range(nr_workers):
        threads[i].join()

In function scen_solve we create and instantiate a GamsModelInstance as in the preceding examples and make parameter bmult modifiable. Note that we choose cplex as solver because it is thread safe (gurobi would also be possible).

We have two critical sections that are locked by the objects list_lock and io_lock. Note that the pop method removes and returns the last element from the list and deletes it. Once the list is empty the loop terminates.

...
def scen_solve(checkpoint, bmult_list, list_lock, io_lock):
    list_lock.acquire()
    mi = checkpoint.add_modelinstance()
    list_lock.release()
    bmult = mi.sync_db.add_parameter("bmult", 0, "demand multiplier")
    opt = ws.add_options()
    opt.all_model_types = "cplex"
 
    # instantiate the GamsModelInstance and pass a model definition and GamsModifier to declare bmult mutable
    mi.instantiate("transport use lp min z", GamsModifier(bmult), opt)
    bmult.add_record().value = 1.0
 
    while True:
        # dynamically get a bmult value from the queue instead of passing it to the different threads at creation time
        list_lock.acquire()
        if not bmult_list:
            list_lock.release()
            return
        b = bmult_list.pop()
        list_lock.release()
        bmult.first_record().value = b
        mi.solve()
 
        # we need to make the ouput a critical section to avoid messed up report informations
        io_lock.acquire()
        print(f"Scenario bmult={b}:")
        print(f"  Modelstatus: {mi.model_status}")
        print(f"  Solvestatus: {mi.solver_status}")
        print(f"  Obj: {mi.sync_db['z'].first_record().level}")
        io_lock.release()
...

How to fill a GamsDatabase by reading from MS Access (transport9.py)

This example illustrates how to import data from Microsoft Access to a GamsDatabase. There are a few prerequisites required to run transport9.py successfully.

We import pyodbc.
Note that an architecture mismatch might cause problems. The bitness of your MS Access, Python, pyodbc and GAMS should be identical (64 bit).

We call a function read_data_from_access that finally returns a GamsDatabase as shown below.

...
    db = read_from_access(ws)
...

The function read_from_access begins with the creation of an empty database. Afterwards we set up a connection to the MS Access database transport.accdb which can be found in [GAMSDIR]\apifiles\Data. To finally read in GAMS sets and parameters we call the functions read_set and read_parameter that are explained down below.

...
def read_from_access(ws):
    db = ws.add_database()
 
    # connect to database
    str_access_conn = r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=..\..\..\..\apifiles\Data\transport.accdb"
 
    try:
        connection = pyodbc.connect(str_access_conn)
    except Exception as e:
        raise Exception(f"Error: Failed to create a database connection.\n{e}")
 
    # read GAMS sets
    read_set(connection, db, "SELECT Plant FROM plant", "i", 1, "canning plants")
    read_set(connection, db, "SELECT Market FROM Market", "j", 1, "markets")
 
    # read GAMS parameters
    read_parameter(
        connection,
        db,
        "SELECT Plant,Capacity FROM Plant",
        "a",
        1,
        "capacity of plant i in cases",
    )
    read_parameter(
        connection,
        db,
        "SELECT Market,Demand FROM Market",
        "b",
        1,
        "demand at market j in cases",
    )
    read_parameter(
        connection,
        db,
        "SELECT Plant,Market,Distance FROM Distance",
        "d",
        2,
        "distance in thousands of miles",
    )
 
    connection.close()
    return db
...

The function read_set adds a set to the GamsDatabase that is filled with the data from the MS Access file afterwards. The function read_parameter works quite similar.

...
def read_set(connection, db, query_string, set_name, set_dim, set_exp=""):
    try:
        cursor = connection.cursor()
        cursor.execute(query_string)
        data = cursor.fetchall()
 
        if len(data[0]) != set_dim:
            raise Exception(
                "Number of fields in select statement does not match setDim"
            )
 
        i = db.add_set(set_name, set_dim, set_exp)
 
        for row in data:
            keys = []
            for key in row:
                keys.append(str(key))
            i.add_record(keys)
 
    except Exception as ex:
        raise Exception(
            "Error: Failed to retrieve the required data from the database.\n{0}".format(
                ex
            )
        )
    finally:
        cursor.close()
...

Once we read in all the data we can create a GamsJob from the GamsDatabase and run it as usual.

How to fill a GamsDatabase by reading from MS Excel (transport10.py)

This example illustrates how to read data from Excel, or to be more specific, from [GAMSDIR]\apifiles\Data\transport.xlsx.

At first you have to download the openpyxl package:

pip install openpyxl

Now you should be able to run transport10.py.

In transport10.py the model is given as string without data like in many examples before and the Excel file transport.xlsx is located at [GAMSDIR]\apifiles\Data. At first we define the workbook to read from and the different sheet names. To ensure to have the same number of markets and plants in all spreadsheets, we conduct a little test that checks for the number of rows and columns. Our workspace is only created if this test yields no errors.

...
    wb = load_workbook(
        os.path.join(*[os.pardir] * 4, "apifiles", "Data", "transport.xlsx")
    )
    capacity = wb["capacity"]
    demand = wb["demand"]
    distance = wb["distance"]
 
    # number of markets/plants have to be the same in all spreadsheets
    if (
        distance.max_column - 1 != demand.max_column
        or distance.max_row - 1 != capacity.max_row
    ):
        raise Exception("Size of the spreadsheets doesn't match")
...

Now we can create a GamsDatabase and read in the data contained in the different worksheets. We iterate over the columns and read in the set names and the corresponding parameter values.

...
    db = ws.add_database()
    i = db.add_set("i", 1, "Plants")
    j = db.add_set("j", 1, "Markets")
    capacity_param = db.add_parameter_dc("a", [i], "Capacity")
    demand_param = db.add_parameter_dc("b", [j], "Demand")
    distance_param = db.add_parameter_dc("d", [i, j], "Distance")
 
    for c in capacity.iter_cols():
        key = c[0].value
        i.add_record(key)
        capacity_param.add_record(key).value = c[1].value
 
    for c in demand.iter_cols():
        key = c[0].value
        j.add_record(key)
        demand_param.add_record(key).value = c[1].value
 
    for c in range(2, distance.max_column + 1):
        for r in range(2, distance.max_row + 1):
            keys = (
                distance.cell(row=r, column=1).value,
                distance.cell(row=1, column=c).value,
            )
            v = distance.cell(row=r, column=c).value
            distance_param.add_record(keys).value = v
...

Note that we can name sets and parameters just like in the database but we don't have to. Now we can run our GamsJob as usual.

...
    job = ws.add_job_from_string(GAMS_MODEL)
    opt = ws.add_options()
    opt.defines["gdxincname"] = db.name
    opt.all_model_types = "xpress"
    job.run(opt, databases=db)
    for rec in job.out_db["x"]:
        print(
            f"x({rec.key(0)},{rec.key(1)}): level={rec.level} marginal={rec.marginal}"
        )
...

How to create and use a save/restart file (transport11.py)

In transport11.py we demonstrate how to create and use a save/restart file. Usually such a file should be supplied by an application provider but in this example we create one for demonstration purpose. Note that the restart is launched from a GamsCheckpoint.

We create a directory tmp with internal identifier w_dir in the directory we are currently in. This file will be used as working directory later. From the main function we call the function create_save_restart giving it directory tmp and the desired name for the save/restart file (tbase) as arguments.

...
    working_dir = os.path.join(os.curdir, "tmp")
    create_save_restart(sys_dir, os.path.join(working_dir, "tbase"))
...

In function create_save_restart we create a workspace with the given working directory (w_dir refers to tmp). Then we create a GamsJob from a string. Note that the string given via get_base_model_text contains the basic definitions of sets without giving them a content (that is what $onempty is used for). Afterwards we specify a GamsOption to only compile the job but do not execute it. Then we create a checkpoint cp that is initialized by the following run of the GamsJob and stored in the file given as argument to the function, in our case tbase. This becomes possible because the add_checkpoint method accepts identifiers as well as file names as argument.

...
def create_save_restart(sys_dir, cp_file_name):
    ws = GamsWorkspace(os.path.dirname(cp_file_name), sys_dir)
    job_1 = ws.add_job_from_string(GAMS_BASE_MODEL)
    opt = ws.add_options()
    opt.action = Action.CompileOnly
    cp = ws.add_checkpoint(os.path.basename(cp_file_name))
    job_1.run(opt, cp)
...

So what you should keep in mind before we return to further explanations of the main function is, that the file tbase is now in the current working directory and contains a checkpoint that will work exactly like a restart file.

In the main function we define some data using Python data structures as we already did in [transport4.py] (How to define data using Python data structures (transport4.py)) before we create the GamsWorkspace and a GamsDatabase.

...
    sys_dir = sys.argv[1] if len(sys.argv) > 1 else None
    working_dir = os.path.join(os.curdir, "tmp")
    ws = GamsWorkspace(working_dir, sys_dir)
    db = ws.add_database()
...

Afterwards we set up the GamsDatabase like we already did in [transport4.py] (How to prepare a GamsDatabase from Python data structures (transport4.py)). Once this is done we run a GamsJob using this data plus the checkpoint stored in file tbase.

...
    cp_base = ws.add_checkpoint("tbase")
    job = ws.add_job_from_string(GAMS_MODEL, cp_base)
    opt = ws.add_options()
    opt.defines["gdxincname"] = db.name
    opt.all_model_types = "xpress"
    job.run(opt, databases=db)
...

Note that the string from which we create job is different to the one used to prepare the checkpoint stored in tbase and is only responsible for reading in the data from the GamsDatabase correctly. The entire model definition is delivered by the checkpoint cp_base which is equal to the one we saved in tbase.

Table of Contents