Table of Contents
- Motivation
- Concept
- Simple Example
- Syntax
- Python
- Interface between GAMS and Python
- Using the Object-oriented API
- Exchange via Files and Environment Variables
- Multiple Independent Python Sessions
- Troubleshooting Embedded Python Code
- Performance Considerations of Embedded Python Code
- Extending GMSPython
- Using an External Python 3 Installation
- Building your own Embedded Python Code Library
- Connect
Motivation
GAMS uses relational data tables as a basic data structure. With these, GAMS code for parallel assignment and equation definition is compact, elegant, and efficient. However, traditional data structures (arrays, lists, dictionaries, trees, graphs, ...) are not natively or easily available in GAMS. Though it is possible to represent such data structures in GAMS, the GAMS code working with such structures can easily become unwieldy, obfuscating, or inefficient. Also, in the past it was not easy to connect libraries from other systems for special algorithms (e.g. graph algorithms, matrix operations, ...) to GAMS without some data programming knowledge and a deep understanding of internal representation of GAMS data (e.g. the GDX API).
The Embedded Code Facility addresses this need and extends the connectivity of GAMS to other programming languages. It allows the use of external code (e.g. Python) during compile and execution time. GAMS symbols are shared with the external code, so no communication via disk is necessary. It utilizes an API (this API can be found in apifiles/C/api/emblib*.h
together with the code for the Python and Connect embedded code library apifiles/C/api/embpyoo.c/h
) and additional source code for common tasks so that the user can concentrate on the task at hand and not the mechanics of moving data in and out of GAMS.
Concept
As pointed out in section Motivation, the main idea of the Embedded Code Facility is to enable the use of external code in GAMS and give this code direct in-memory access to the GAMS database (or better: to GAMS symbols, namely sets, parameters, variables, and equations). This can be done by defining sections in the GAMS code which contain no GAMS code, but code written in a defined external code. These sections can be used at both GAMS compile and execution time (compare section GAMS Compile Time and Execution Time Phase). Details about how to do this can be found in the Syntax section.
- Note
- We will continuously extend this feature to different programming languages. At the moment only Python and Connect are supported.
Also, the system provides some help to develop and debug the external code independent of GAMS first. More about this topic can be found in section Troubleshooting Embedded Python Code.
The communication of the data between the GAMS part of the program and the embedded code part was inspired by the existing interface to GDX in many ways. For example, using dollar control options $on/offMulti, $onMultiR, and $on/offFiltered it is possible to decide if data changed in the embedded code should replace the GAMS data or get merged with it and just like loading data from GDX, one can decide if data from embedded code should change the GAMS database filtered or domain checked.
Simple Example
Python
In a very first example we look into some Python code that helps to split some label names that are already present in GAMS. We do this at compile time in order to read the broken up pieces as individual sets (country
and city
) into GAMS plus some mapping sets (mccCountry
and mccCity
) between the original labels and the new labels. The compile-time embedded code starts with a $onEmbeddedCode
followed by the type of code to expect (Python:
). The lines between $onEmbeddedCode
and $offEmbeddedCode
is Python code. We do not want to go into Python details, but the first few lines initialize some empty Python list
objects (mccCountry
and mccCity
) as well as some empty Python set
objects. In the Python for
loop that follows we iterate over all individual labels of the GAMS set cc
. Python gets access to the GAMS set cc
via the member function get
of the implicitly defined Python object gams
. get
returns an object that is iterable and can be used in a Python for
loop. The type of the records one gets depend on the dimensionality and type of the GAMS symbol. In the loop body we use the Python split
function to extract the first (r[0]
is country) and second (r[1]
is city) part of the label. The three strings cc
, r[0]
, r[1]
are used to build up the Python list
objects that store the information for the maps mccCountry
and mccCity
and the Python set
objects that store the labels for the new sets country
and city
. The Python set
has the advantage to store a label just once even if we add it multiple times. Python prepares to send items back to GAMS via the gams
member function set
that can deal with both Python list
and Python set
objects. The command $offEmbeddedCode
is followed by a list (without separating commas) of GAMS symbols that instructs the GAMS compiler to read these symbols back. This list is optional and can be left out. GAMS will import all symbols set by the Python code. The GAMS compiler (without executing execution-time embedded code) does not know which symbols will be implicitly loaded, hence it cannot define symbols. If such a symbol is referenced further down in the compilation process, the compiler will complain with error $141: Symbol declared but no values have been assigned.
One can avoid this situation by explicitly list the symbols on the end/pauseEmbeddedCode
line.
Note that GAMS syntax is case insensitive while Python is case sensitive. Hence, the strings that represent the GAMS symbol names in the Python code can have any casing (e.g. gams.get("cc")
or gams.get("CC")
), while the corresponding Python objects need to have consistent casing throughout the Python code.
The following code presents the entire embeddedSplit example from the GAMS Data Utilities Library:
Set cc / "France - Paris", "France - Lille", "France - Toulouse"
"Spain - Madrid", "Spain - Cordoba", "Spain - Seville", "Spain - Bilbao"
"USA - Washington DC", "USA - Houston", "USA - New York",
"Germany - Berlin", "Germany - Munich", "Germany - Bonn" /
country
city
mccCountry(cc,country<) Mapping between country and related elements in set cc
mccCity(cc,city<) Mapping between city and related elements in set cc;
$onEmbeddedCode Python:
mccCountry = []
mccCity = []
for cc in gams.get("cc"):
r = str.split(cc, " - ", 1)
mccCountry.append((cc,r[0]))
mccCity.append((cc,r[1]))
gams.set("mccCountry",mccCountry)
gams.set("mccCity",mccCity)
$offEmbeddedCode mccCountry mccCity
Option mccCountry:0:0:1, mccCity:0:0:1;
Display country, city, mccCountry, mccCity;
The display in the listing file looks as follows:
---- 25 SET country Spain , USA , Germany, France ---- 25 SET city Berlin , Bilbao , Cordoba , Madrid New York , Washington DC, Paris , Houston Munich , Lille , Seville , Bonn Toulouse ---- 25 SET mccCountry France - Paris .France France - Lille .France France - Toulouse .France Spain - Madrid .Spain Spain - Cordoba .Spain Spain - Seville .Spain Spain - Bilbao .Spain USA - Washington DC.USA USA - Houston .USA USA - New York .USA Germany - Berlin .Germany Germany - Munich .Germany Germany - Bonn .Germany ---- 25 SET mccCity France - Paris .Paris France - Lille .Lille France - Toulouse .Toulouse Spain - Madrid .Madrid Spain - Cordoba .Cordoba Spain - Seville .Seville Spain - Bilbao .Bilbao USA - Washington DC.Washington DC USA - Houston .Houston USA - New York .New York Germany - Berlin .Berlin Germany - Munich .Munich Germany - Bonn .Bonn
The second example demonstrates the use of embedded code at execution time. The syntax for the execution time embedded code in this example is identical to the compile time variant with the exception of the keywords that start and end the embedded code section: embeddedCode
and endEmbeddedCode
. It is important to understand that the execution of the code happens at GAMS execution time, so e.g. no new labels can be produced and send back to GAMS. In this example we use some Python code to generate a random permutation of set elements of set i
and store this in a two dimensional set p
. In this example we do not use a loop to iterate through the elements of a GAMS system but make use of the fact that the Python object returned by gams.get("i")
is iterable and can in its entirety be stored in the Python list
with name i
with the short and powerful command i = list(gams.get("i"))
. The permutation of elements in list p
which is a copy of list i
is created by the Python statement random.shuffle(p)
. The following code presents the entire example:
Set i /i1*i10/
p(i,i) "permutation";
embeddedCode Python:
import random
i = list(gams.get("i"))
p = list(i)
random.shuffle(p)
for idx in range(len(i)):
p[idx] = (i[idx], p[idx])
gams.set("p", p)
endEmbeddedCode p
option p:0:0:1;
display p;
The display in the listing file looks as follows:
---- 11 SET p permutation i1 .i1 i2 .i7 i3 .i5 i4 .i2 i5 .i10 i6 .i6 i7 .i9 i8 .i4 i9 .i8 i10.i3
Connect
In the following example, we need to process a CSV file that cannot be read directly by GAMS under $onDelim. So we instruct Connect to read the CSV file into the Connect database and then write to GAMS:
Set dates, stocks;
Parameter stockprice(dates<,stocks<);
$onText
Read a CSV file that looks as follows:
date;symbol;price
2016/01/04;AAPL;105,35
2016/01/04;AXP;67,59
2016/01/04;BA;140,50
...
and can't be read directly by GAMS under $onDelim because some labels
include '/' and are not quoted, use ';' as field separator, and ',' as
decimal character.
$offText
$onEmbeddedCode Connect:
- CSVReader:
file: stockprice.csv
name: stockprice
indexColumns: [1, 2]
valueColumns: [3]
fieldSeparator: ';'
decimalSeparator: ','
- GAMSWriter:
symbols:
- name: stockprice
$offEmbeddedCode
display stockprice;
We leave the CSV file processing to the CSVReader and then export the stockprice
symbol to GAMS. Please note that the symbols written to GAMS by the GAMSWriter do not have to be explicitly listed on the $offEmbeddedCode
line, this is optional. Moreover, the Connect agents GAMSReader and GAMSWriter are only available under embedded code.
We might use the data to calculate an optimal portfolio. The share of each stock in this optimal portfolio is represented by the parameter share(stocks)
and can be exported to Excel using the PandasExcelWriter. Before exporting to Excel, the data first needs to be transferred from GAMS to Connect using the GAMSReader:
* Continues the code from the previous example
Parameter share(stocks);
* For demonstration we fill share with random numbers
share(stocks) = uniform(0,1);
embeddedCode Connect:
- GAMSReader:
symbols:
- name: share
- PandasExcelWriter:
file: share.xlsx
symbols:
- name: share
range: Share!A1
endEmbeddedCode
Syntax
This section explains the GAMS functions/keywords which were introduced to enable the Embedded Code Facility. The first subsection deals with the syntax for compile time, the second with the syntax for execution time (compare section GAMS Compile Time and Execution Time Phase).
Compile Time
There are three dollar control options to start an embedded code section at compile time:
$onEmbeddedCode[.tag] [Connect|Python]: [arguments]
$onEmbeddedCodeS[.tag] [Connect|Python]: [arguments]
$onEmbeddedCodeV[.tag] [Connect|Python]: [arguments]
Lines following one of the above statements are passed on to the embedded code engine (e.g. Connect or the Python interpreter) until this dollar control option, which ends the embedded code section at compile time, is hit:
$offEmbeddedCode[.tag] {symbol[<[=]embSymbol[.dimX]]}
These dollar control options are explained here in more detail. The usage of the optional arguments
by the embedded code is explained in the specific embedded code engine section below.
- Note
- The optional
arguments
from the$onEmbeddedCode[S|V][.tag]
statement can be accessed asgams.arguments
in the Python code. - The optional
output
symbols
from the$offEmbeddedCode[.tag]
statement need to be written to by the embedded code. - More about the specific GAMS Python syntax can be found below.
- The optional
Execution Time
At execution time an embedded code section is started with one of these statements:
embeddedCode[.tag] [Connect|Python]: [arguments]
embeddedCodeS[.tag] [Connect|Python]: [arguments]
embeddedCodeV[.tag] [Connect|Python]: [arguments]
Similar to the compile time alternatives $onEmbeddedCode[S|V][.tag], the first two variants are synonyms which allow parameter substitution in the code that follows, while the last variant does not allow this but passes the code verbatim to the embedded code engine. The usage of the optional arguments
by the embedded code is explained in the specific embedded code engine section below. The arguments passed to the embedded code engine at execution time can be extended by the put_utility command ECArguments
. For example,
put_utility 'ECArguments' / ' world...';
embeddedCode Python: Hello
gams.printLog(gams.arguments)
endEmbeddedCode
prints Hello world...
.
Lines following one of the statements embeddedCode[S|V]
are passed on to the embedded code engine (e.g. the Python interpreter) until one of the following two statements, which end the embedded code section at execution time, is hit:
endEmbeddedCode[.tag] {output symbols}
pauseEmbeddedCode[.tag] {output symbols}
When the optional tag
suffix (which can be any string) is used to start an embedded code section, the same one has to be used to end/pause that section. Both statements end the embedded code section and switch back to GAMS syntax in the following lines. Also, both statements can be followed by a GAMS symbol or a list of GAMS symbols which would get updated in the GAMS database after the embedded code got executed. If output
symbols
are specified, they need to be written to (e.g. by gams.set
for embedded Python code) by the embedded code before.
To continue a previously paused embedded code section one of the following statements is used:
continueEmbeddedCode [handle]: [arguments]
continueEmbeddedCodeS [handle]: [arguments]
continueEmbeddedCodeV [handle]: [arguments]
As seen before, the first two variants are synonyms which allow parameter substitution in the embedded code that follows, while the last variant does not allow this but passes the code verbatim to the interpreter. Again, the usage of the optional arguments
by the embedded code is explained in the specific embedded code engine section below.
New in these statements is the optional handle
. If omitted, the last code section that was paused will be continued. However, sometimes one might need to maintain different embedded code sections active in parallel and independent of each other. There is a new function to store a handle of the last embedded code section that was executed which could then later be used to continue a specific paused code section:
handle = embeddedHandle;
What it exactly means to end or pause/continue an embedded code section depends on the embedded code engine and is explained in more detail with the code engine below. Independent of the embedded code engine it is important to note that if GAMS stops and restarts while executing a solve
statement (with solveLink=0) continuing any embedded code section causes the code to fail with the following error message:
This happens because under solvelink=0 the GAMS process terminates when the solver starts up. This causes the loss of the state of the embedded code environment that is linked into the GAMS process as a shared library. Under all other settings of solveLink the GAMS process is not stopped and restarted and hence a paused embedded code environment can be continued after the solve statement has carried out.
Python
GAMS and Python complement each other in many different ways. GAMS has a compact but readable syntax for data assignment and equation definition statements. Python is a great scripting language to do more traditional type programming especially string manipulation which is completely missing in GAMS. Moreover, the vast number of packages help solving scientific problems outside the scope of GAMS. In order to open this world to our customers in an easy way, GAMS comes with a Python 3 installation. By default, this installation is used in the Embedded Code Facility for Python and is ready to be used with the GAMS Python API. The Python installation is located in [GAMS directory]/GMSPython
. In contrast to earlier versions this installation comes without the Python package manager pip
and therefore is not easily extendable. GMSPython
comes with a selection of third party packages and modules that are either required by the APIs of the GAMS Python API collection or that are particularly helpful from within embedded Python code. Those packages are located in [GAMS directory]/apifiles/Python/thirdparty
. Running the GMSPython
interpreter like follows will display all available modules:
[GAMS directory]/GMSPython/python -c"help('modules')"
The list of redistributed third party packages is subject to constant changes and it is not recommended to rely on their availability. If you need packages that do not come with GMSPython
, we recommend that you install your own version of Python and follow a few steps to connect GAMS with your Python installation. It is also possible to install pip
for GMSPython
using get-pip. While any Python installation should work in combination with GAMS we can recommend the Anaconda
or Miniconda
Python distributions. These Python distributions work with environments which allow you to have many different Python installations at the same time that don't get in each other's way.
During the lifetime of the GAMS process a single Python interpreter is used and the code "lives" in Python's global scope. This means that objects defined in one embedded Python code section are still there in another embedded Python code section:
$onEmbeddedCode Python:
s = 'Hello world'
$offEmbeddedCode
$onEmbeddedCode Python:
gams.printLog(s)
$offEmbeddedCode
will produce the following output in the GAMS log:
--- Starting compilation --- main.gms(3) 2 Mb --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll --- main.gms(6) 2 Mb --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll Hello world --- main.gms(7) 2 Mb
If the GAMS process stops and restarts during execution, e.g. because of a solve with solveLink=0, the Python interpreter will be started new. This can cause complications if GAMS is in a sequence of pause/continueEmbeddedCode
as discussed above.
Interface between GAMS and Python
The Python class ECGamsDatabase
is the interface between GAMS and Python. An instance of this class is automatically created when an embedded code section is entered and can be accessed using the identifier gams
. The following methods can be used in order to interact with GAMS:
gams.get(symbolName, keyType=KeyType.STRING, keyFormat=KeyFormat.AUTO, valueFormat=ValueFormat.AUTO, recordFormat=RecordFormat.AUTO)
This method retrieves an iterable object representing the symbol identified with
symbolName
. Typically there are two possibilities to access the records. Iterating using e.g. afor
loop provides access to the individual records. By callinglist()
on the iterable object, a list containing all the data is created. Several optional parameters can be used in order to modify the format of the retrieved data:
keyType
: Determines the data type of the keys. It can be eitherKeyType.STRING
(labels) orKeyType.INT
(label indexes). The default setting isKeyType.STRING
.keyFormat
: Specifies the representation of the keys. Possible values are as follows:
KeyFormat.TUPLE
: Encapsulate keys in a tupleKeyFormat.FLAT
: No encapsulationKeyFormat.SKIP
: Keys are skipped and do not appear in the retrieved dataKeyFormat.AUTO
(default): Depending on the dimension of the GAMS symbol, a default format is applied:
- Zero dimensional/scalar:
KeyFormat.SKIP
- One dimensional:
KeyFormat.FLAT
- Multi dimensional:
KeyFormat.TUPLE
valueFormat
: Specifies the representation of the values. Possible values are as follows:
ValueFormat.TUPLE
: Encapsulate values in a tupleValueFormat.FLAT
: No encapsulationValueFormat.SKIP
: Values are skipped and do not appear in the retrieved dataValueFormat.AUTO
(default): Depending on the type of the GAMS symbol, a default formats is applied:
- Set:
ValueFormat.SKIP
- Parameter:
ValueFormat.FLAT
- Variable/Equation:
ValueFormat.TUPLE
recordFormat
Specifies the encapsulation of records into tuples. Possible values are as follows:
RecordFormat.TUPLE
: Encapsulates every record in a tupleRecordFormat.FLAT
: No encapsulation. Throws an exception if it can not be applied. It is guaranteed that the length of a retrieved Python list is equal to the number of records of the corresponding GAMS symbol. This principle leads to an incompatibility ofRecordFormat.FLAT
whenever a record consists of more than one item (e.g. multi dimensional symbols, variables and equations which have five numeric values).RecordFormat.AUTO
(default): Depending on the number of items that represent a record, a default format is applied. If possible this is alwaysRecordFormat.FLAT
.GAMS special values
NA
,INF
, and-INF
will be mapped to IEEE special valuesfloat('nan')
,float('inf')
, andfloat('-inf')
. GAMS special valueEPS
will be either mapped to0
or to the small numeric value4.94066E-324
depending on the setting of flag gams.epsAsZero.The following Python code shows some examples of
gams.get
and illustrates the use of different formats:Set i / i1 text 1, i2 text 2 / j / j1*j2 / Scalar p0 /3.14/; Parameter p1(i) / #i 3.14 / p2(i,j) / i1.#j 3.14 / Variable v0 / fx 3.14 /; equation e1(i) / #i.fx 3.14 / e2(i,j) / i1.#j.fx 3.14 /; $onEmbeddedCode Python: # scalar parameter gams.printLog(f"{list(gams.get('p0'))}") # prints [3.14] gams.printLog(f"{list(gams.get('p0', recordFormat=RecordFormat.TUPLE))}") # prints [(3.14,)] # one dimensional parameters: gams.printLog(f"{list(gams.get('p1'))}") # prints [('i1', 3.14), ('i2', 3.14)] gams.printLog(f"{list(gams.get('p1', keyFormat=KeyFormat.TUPLE))}") # prints [(('i1',), 3.14), (('i2',), 3.14)] gams.printLog(f"{list(gams.get('p1', valueFormat=ValueFormat.TUPLE))}") # prints [('i1', (3.14,)), ('i2', (3.14,))] gams.printLog(f"{list(gams.get('p1', keyFormat=KeyFormat.TUPLE, valueFormat=ValueFormat.TUPLE))}") # prints [('i1', (3.14,)), ('i2', (3.14,))] # two dimensional parameter: gams.printLog(f"{list(gams.get('p2'))}") # prints [(('i1', 'j1'), 3.14), (('i1', 'j2'), 3.14)] gams.printLog(f"{list(gams.get('p2', keyFormat=KeyFormat.FLAT))}") # prints [('i1', 'j1', 3.14), ('i1', 'j2', 3.14)] # one dimensional sets: gams.printLog(f"{list(gams.get('i'))}") # prints ['i1', 'i2'] gams.printLog(f"{list(gams.get('i',valueFormat=ValueFormat.FLAT))}") # prints [('i1', 'text 1'), ('i2', 'text 2')] # scalar variables/equations gams.printLog(f"{list(gams.get('v0'))}") # prints [(3.14, 0.0, 3.14, 3.14, 1.0)] # one dimensional variables/equations: gams.printLog(f"{list(gams.get('e1'))}") # prints [('i1', (3.14, 0.0, 3.14, 3.14, 1.0)), ('i2', (3.14, 0.0, 3.14, 3.14, 1.0))] gams.printLog(f"{list(gams.get('e1', valueFormat=ValueFormat.FLAT))}") # prints [('i1', 3.14, 0.0, 3.14, 3.14, 1.0), ('i2', 3.14, 0.0, 3.14, 3.14, 1.0)] gams.printLog(f"{list(gams.get('e1', keyFormat=KeyFormat.TUPLE))}") # prints [(('i1',), (3.14, 0.0, 3.14, 3.14, 1.0)), (('i2',), (3.14, 0.0, 3.14, 3.14, 1.0))] # two dimensional variables/equations: gams.printLog(f"{list(gams.get('e2'))}") # prints [(('i1', 'j1'), (3.14, 0.0, 3.14, 3.14, 1.0)), (('i1', 'j2'), (3.14, 0.0, 3.14, 3.14, 1.0))] gams.printLog(f"{list(gams.get('e2', keyFormat=KeyFormat.FLAT, valueFormat=ValueFormat.FLAT))}") # prints [('i1', 'j1', 3.14, 0.0, 3.14, 3.14, 1.0), ('i1', 'j2', 3.14, 0.0, 3.14, 3.14, 1.0)] # using label indexes instead of labels gams.printLog(f"{list(gams.get('p1', keyType=KeyType.INT))}") # prints [(1, 3.14), (2, 3.14)] gams.printLog(f"{list(gams.get('i', keyFormat=KeyFormat.TUPLE, valueFormat=ValueFormat.TUPLE, keyType=KeyType.INT))}") # prints [((1,), ('text 1',)), ((2,), ('text 2',))] gams.printLog(f"{list(gams.get('e2', keyType=KeyType.INT))}") # prints [((1, 3), (3.14, 0.0, 3.14, 3.14, 1.0)), ((1, 4), (3.14, 0.0, 3.14, 3.14, 1.0))] $offEmbeddedCode
gams.set(symbolName, data, mergeType=MergeType.DEFAULT, domCheck=DomainCheckType.DEFAULT, mapKeys=lambda x:x)This method sets the data for the GAMS symbol identified with
symbolName
. The parameterdata
takes a Python list or set containing items that represent the records of the symbol. It is also possible to pass an instance of a subclass of _GamsSymbol (e.g. GamsParameter or GamsSet) when using the Object-oriented GAMS Python API in an embedded code section. In case of a Python list or set, depending on the type and the dimension of the symbol, different formats can be used in order to specify the data. Different formats can not be mixed within one list. In general each record needs to be represented as a tuple containing the keys and the value field(s). Keys and/or values can also be enclosed in a tuple. Keys can be entered as labels (string) or label indexes (int). The argumentmapKeys
allows to pass a callable to remap the elements of the key (e.g. turn them explicitly into strings viamapKeys=str
). Value fields depend on the type of the symbol:
- Parameters: One numerical value
- Sets: explanatory text (optional)
- Variable/Equations: Five numerical values: level, marginal, lower bound, upper bound, scale/prior/stage
IEEE special values
float('nan')
,float('inf')
, andfloat('-inf')
will be remapped to GAMS special valuesNA
,INF
, and-INF
. The small numeric value4.94066E-324
will be mapped into GAMS special valueEPS
.The following Python code gives some examples on different valid formats for different symbol types and dimensions:
Set i / i1 text 1, i2 text 2 / j / j1*j2 / ii(i); Scalar p0; Parameter p1(i) p2(i,j); equation e1(i) e2(i,j); $onEmbeddedCode Python: # scalar parameter gams.set('p0', [3.14]) gams.set('p0', [(3.14,)]) # one dimensional parameters: gams.set('p1', [("i1", 3.14), ("i2", 3.14)]) gams.set('p1', [(("i1",), 3.14), (("i2",), 3.14)]) gams.set('p1', [("i1", (3.14,)), ("i2", (3.14,))]) gams.set('p1', [(("i1",), (3.14,)), (("i2",), (3.14,))]) # two dimensional parameter: gams.set('p2', [('i1', 'j1', 3.14), ('i1', 'j2', 3.14)]) gams.set('p2', [(('i1', 'j1'), (3.14,)), (('i1', 'j2'), (3.14,))]) # one dimensional sets: gams.set('ii', ['i1', 'i2']) gams.set('ii', [('i1',), ('i2',)]) # one dimensional sets with explanatory text gams.set('ii', [('i1', "text 1"), ('i2', "text 2")]) gams.set('ii', [(('i1',), ("text 1",)), (('i2',), ("text 2",))]) # one dimensional variables/equations: gams.set('e1', [("i1", 3.14, 0, 0, 10, 1), ("i2", 3.14, 0, 0, 10, 1)]) gams.set('e1', [("i1", (3.14, 0, 0, 10, 1)), ("i2", (3.14, 0, 0, 10, 1))]) gams.set('e1', [(("i1",), (3.14, 0, 0, 10, 1)), (("i2",), (3.14, 0, 0, 10, 1))]) # two dimensional variables/equations: gams.set('e2', [("i1", "j1", 3.14, 0, 0, 10, 1), ("i1", "j2", 3.14, 0, 0, 10, 1)]) gams.set('e2', [(("i1", "j1"), (3.14, 0, 0, 10, 1)), (("i1", "j2"), (3.14, 0, 0, 10, 1))]) # using label indexes instead of labels gams.set('p1', [((1,), (3.14,)), ((2,), (3.14,))]) # one dimensional parameter gams.set('ii', [((1,), ("text 1",)), ((2,), ("text 2",))]) # one dimensional set gams.set('e2', [((1, 3), (3.14, 0, 0, 10, 1)), ((1, 4), (3.14, 0, 0, 10, 1))]) # two dimensional equation/variable $offEmbeddedCode p0 p1 p2 ii e1 e2
The optional parameter
mergeType
specifies if data in a GAMS symbol is merged (MergeType.MERGE
) or replaced (MergeType.REPLACE
). If left atMergeType.DEFAULT
it depends on the setting of $on/offMulti[R] if GAMS does a merge, replace, or trigger an error during compile time. During execution timeMergeType.DEFAULT
is the same asMergeType.MERGE
. The optional parameterdomCheck
specifies if Domain Checking is applied (DomainCheckType.CHECKED
) or if records that would cause a domain violation are filtered (DomainCheckType.FILTERED
). If left atDomainCheckType.DEFAULT
it depends on the setting of $on/offFiltered if GAMS does a filtered load or checks the domains during compile time. During execution timeDomainCheckType.DEFAULT
is the same asDomainCheckType.FILTERED
.
- Note
- When calling
gams.set()
in an embedded code section during execution time, new labels that are not known to the current GAMS program can not be added. The attempt will result in an execution error.gams.getUel(idx)Returns the label corresponding to the label index
idx
gams.mergeUel(label)Adds
label
to the GAMS universe if it was unknown and returns the corresponding label index.
- Note
- When calling
gams.mergeUel()
in an embedded code section during execution time, new labels that are not known to the current GAMS program can not be added. The attempt will result in an execution error.gams.getUelCount()Returns the number of labels.
gams.printLog(msg)msg
to log.gams.argumentsContains the command line that was passed to the Python interpreter of the embedded code section. The syntax for passing arguments to the Python interpreter can be seen above.
gams.epsAsZeroFlag to read GAMS
EPS
as 0 (True
) or as a small number,4.94066E-324
, when set toFalse
. Default isTrue
.gams.wsProperty to retrieve an instance of GamsWorkspace that allows to use the Object-oriented GAMS Python API. The instance is created when the property is read for the first time using a temporary working directory. A different working directory can be specified using gams.wsWorkingDir. For debug output, the property gams.debug can be set to a value from DebugLevel
gams.wsWorkingDirProperty that can be specified before accessing gams.ws for the first time in order to set the working directory. Setting the property after the first call to gams.ws will have no effect. of the created GamsWorkspace.
gams.dbProperty to retrieve an instance of GamsDatabase. The instance is created when the property is read for the first time and allows to access the GAMS symbols using the methods of the Object-oriented GAMS Python API.
gams.debugProperty that can be set to a value from DebugLevel for debug output. Default is
DebugLevel.Off
(0
). Setting this property affects both the debug output from embedded code and the debug output from the Object-oriented API. The property needs to be changed before accessing gams.ws for the first time in order to take effect in the Object-oriented API. Setting the property after the first call to gams.ws will have no effect on the GamsWorkspace.
Using the Object-oriented API
The ECGamsDatabase class provides mechanisms for using the Object-oriented GAMS Python API in an embedded code section. The property gams.ws
can be used to get an instance of GamsWorkspace. The property gams.db
allows to access an instance of GamsDatabase that can be used to read and write data from the internal GAMS database like it can be done using gams.get
and gams.set
but using the access mechanisms of the GamsDatabase class. In addition to the different reading and writing mechanisms offered by the OO-API other classes like GamsModelInstance can be used inside embedded Python code. This combination provides programmatic access to a generated model instance and exchange of update parameters and results between Python and GAMS. The GAMS Library model embmiex1 gives an example of how to combine embedded Python code and a GamsModelInstance
which utilized the library include $libInclude pyEmbMI
.
Moreover, the GAMS Transfer Python API can be combine with the OO-API in the embedded code context as the following example demonstrates:
Set i / i1*i1000 /; alias(i,j);
Parameter a(i,j), aInv(i,j);
a(i,j) = 1 / (ord(i)+ord(j) - 1);
a(i,i) = 1;
embeddedCode Python:
import gamstransfer as gt
import numpy as np
m = gt.Container(gams.db)
A = m.data["a"].toDense()
m.data['aInv'].setRecords(np.linalg.inv(A))
m.write(gams.db, ["aInv"])
endEmbeddedCode aInv
Exchange via Files and Environment Variables
The Python class ECGamsDatabase provides read and write access to GAMS symbols. There are two other communication methods that can be used at GAMS compile time: files and environment variables. At compile time the Python code can produce a text file that can be included into GAMS via $include
as in the following example:
$onEmbeddedCode Python: 10
with open('i.txt', 'w') as f:
for i in range(int(gams.arguments)):
f.write(f'i{i+1}\n')
$offEmbeddedCode
Set i /
$include i.txt
/;
display i;
Here the Python code received the number of elements to write to a text file via the argument after Python:
. This text file is then included in the data statement of the GAMS set i
. The display in the listing file looks as follows:
---- 20 SET i i1 , i2 , i3 , i4 , i5 , i6 , i7 , i8 , i9 , i10
Python provides many packages to read input files for many different formats and hence can be used to transform such formats to a GAMS compatible input format, as an alternative to providing the data via list
objects and the gams.set functionality.
The second alternative to exchange information at compile time are environment variables. GAMS and Python allow to get and set environment variables and hence can be conveniently used to exchange small pieces of information. The following code provides an example where the maximum value of a parameter b
is needed to build a set k
:
Set i / i1*i5 /;
Parameter b(i) / i1 2, i2 7, i3 59, i4 2, i5 47 /;
$onEmbeddedCode Python:
import os
os.environ["MAXB"] = str(int(max([b for i,b in gams.get("b")])))
gams.printLog(f'max value in b is {os.environ["MAXB"]}')
$offEmbeddedCode
$if x%sysEnv.MAXB%==x $abort MAXB is not set
Set k "from 0 to max(b)" / 0*%sysEnv.MAXB% /;
$eval CARDK card(k)
$log card(k)=%CARDK%
It is important to understand that os.environ
is initialized when Python imports the os
module. Hence, if we later set an environment variable in GAMS (via $setEnv ABC abc
or put_utility "setEnv" / "ABC" / "abc"
) and access this in some embedded code via os.environ["ABC"]
, this will not provide the expected value "abc". In such a case the environment variable needs to be accessed via gams.get_env("ABC")
(and can be stored in os.environ["ABC"]
) as demonstrated by the following code:
$onEmbeddedCode Python:
import os
os.environ["TEST"] = 'Hello world'
$offEmbeddedCode
$log TEST=%sysEnv.TEST%
$setEnv ABC abc
$onEmbeddedCode Python:
gams.printLog(f'ABC={os.environ.get("ABC", "ABC not set")}')
gams.printLog(f'ABC={gams.get_env("ABC")}')
$offEmbeddedCode
which produces the following output:
--- Starting compilation --- main.gms(4) 2 Mb --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll TEST=Hello world --- main.gms(10) 2 Mb --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll ABC=ABC not set ABC=abc
Obviously, the set k
can also be filled directly with data from within Python:
Set i / i1*i5 /;
Parameter b(i) / i1 2, i2 7, i3 59, i4 2, i5 47 /;
Set k(*) "from 0 to max(b)";
$onEmbeddedCode Python:
kmax = max([b for i,b in gams.get("b")])
gams.set('k',[str(k) for k in range(int(kmax+1))])
$offEmbeddedCode k
In both cases the resulting GAMS symbol k
is a set with elements 0
to 59
.
Note that the (*)
in the declaration of k
is necessary to inform the compiler of dimensionality of the symbol k
. GAMS symbol declarations can be done without domain list, so a set k
does not necessarily mean that this is a one dimensional set. The compiler will learn the dimensionality from the use of k
. For example, if k
shows up as a domain set in a declaration (parameter p(k)
) it is clear that k
is a one dimensional set. Without knowing the dimensionality k
will not be available with embedded code or exported (at compile-time) to GDX and the following code
Set k;
$onEmbeddedCode Python:
gams.set('k',['0','1','2'])
$offEmbeddedCode k
will fail with compilation error Cannot load/unload symbol with unknown dimension
.
Multiple Independent Python Sessions
At execution time the user has the ability to pause and continue an embedded code segment. As explained at the beginning, the user Python code lives in Python's global scope. Nevertheless, besides some performance aspects one can also allows work with multiple independent Python sessions. Due to some Python module incompatibilities (e.g. numpy
) the independent Python sessions have to be enabled with a command line option pyMultInst set to 1. After the pauseEmbeddedCode
we can obtain and store the handle of the last embedded code execution via function embeddedHandle
. The handle needs to be supplied when we continue the Python session via continueEmbeddedCode
. The different Python sessions are fairly separate as shown in the example below. Here we save the GAMS scalar ord(i)
in a Python object i
five times. The value for i
that we store in the five different Python sessions is 1 to 5. In the subsequent loop we activate the Python session with the appropriate handle and print the value of i
:
$if not %sysEnv.GMSPYTHONMULTINST%==1 $abort.noError Start with command line option pyMultInst=1
Set i / i1*i5 /;
Parameter h(i) 'session handle';
loop(i,
put_utility 'setEnv' / 'ORDI' / ord(i):0:0;
embeddedCode Python:
i = int(gams.get_env("ORDI"))
gams.printLog(f'{i}')
pauseEmbeddedCode
h(i) = embeddedHandle;
);
loop(i,
continueEmbeddedCode h(i):
gams.printLog(f'{i}')
endEmbeddedCode
);
The GAMS log shows the value of i
in the different Python sessions:
--- Starting execution: elapsed 0:00:00.002 --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll 1 --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll 2 --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll 3 --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll 4 --- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll 5 --- Execute embedded library embpycclib64.dll 1 --- Execute embedded library embpycclib64.dll 2 --- Execute embedded library embpycclib64.dll 3 --- Execute embedded library embpycclib64.dll 4 --- Execute embedded library embpycclib64.dll 5 *** Status: Normal completion
Troubleshooting Embedded Python Code
The GAMS compiler ensures that the number of errors during execution time is minimized. While the logic of the GAMS program might be flawed there is nothing (with a few exceptions) that the GAMS system cannot execute. This is different if we embed foreign code in a GAMS program. The GAMS compiler does not understand the foreign code syntax and just skips over it. Only when the code is executed we will find out if everything works as expected. If the embedded code contains some (Python) syntax errors the Python parser will inform us about this and the message will appear in the GAMS log. For example, the following Python code using the gams.printLog function two times will generate a syntax error:
$onEmbeddedCode Python:
gams.printLog('hello')
gams.printLog('world...')
$offEmbeddedCode
The GAMS log will provide some guidance:
--- Execute embedded library libembpycclib64.so File "<string>", line 7 gams.printLog('world...') ^ IndentationError: unexpected indent Python error! Return code from Python execution: -1 *** Error executing embedded code section: *** Check log above
Moreover, if the Python code raises an exception which is not handled within the code this will also lead to a compilation or execution error in GAMS depending at what phase the embedded code is executed.
embeddedCode Python:
raise Exception('something is wrong')
endEmbeddedCode
will produce the following GAMS log and an execution time error:
--- Initialize embedded library libembpycclib64.so --- Execute embedded library libembpycclib64.so Exception from Python (line 0): something is wrong *** Error at line 1: Error executing "embeddedCode" section: Check log above
- Note
- It is good practice to raise a Python exception if an error occurs. In any case using
exit()
needs to be avoided since it terminates the executable in an uncontrolled way.
The Python code is executed as part of the GAMS process and GAMS gives control to the Python interpreter when executing the embedded code. So in the worst case if the Python interpreter crashes, the entire GAMS process will crash. Therefore, it is important to be able to test and debug the embedded Python code independent of GAMS. In the following examples we call the Python interpreter executable as part of a GAMS job. In principle this can be tested and debugged completely independent of GAMS where a GDX file represents the content of the GAMS database.
In the first example we mimic the embedded code facility at compile time by exporting the entire GAMS database to a GDX file debug.gdx
. With $on/offEcho
we write the embedded code with a few extra lines at the top and bottom surrounded by a try/except block and execute the Python interpreter via $call
. One of the extra lines at the end of the embedded code triggers the creation of a GDX result file debugOut.gdx
which can be imported in subsequent $gdxin/$load
commands.
Set i /i1*i10/
p(i,i) permutation;
$gdxOut debug.gdx
$unload
$gdxOut
$onEcho > debug.py
from gamsemb import *
gams = ECGAMSDatabase(r'%gams.sysdir% '[:-2], 'debug.gdx')
try:
import random
i = list(gams.get("i"))
p = list(i)
random.shuffle(p)
for idx in range(len(i)):
p[idx] = (i[idx], p[idx])
gams.set("p", p)
gmdWriteGDX(gams._gmd,'debugOut.gdx',1);
except Exception as e:
print(str(e))
$offEcho
$call.checkErrorLevel ="%gams.sysdir%GMSPython/python" debug.py
$gdxIn debugOut.gdx
$loadDC p
Option p:0:0:1;
Display p;
In the second example we mimic the embedded code facility at execution time by exporting the entire GAMS database to a GDX file debug.gdx
via execute_unload 'debug.gdx';
. We write the embedded code with a few extra lines at the top and bottom surrounded by a try/except block via the put
facility and execute the Python interpreter via execute
. Identical to the compile time example, we export the result to the GDX file debugOut.gdx
which can be imported via the execute_load
statement.
Set i /i1*i10/
p(i,i) permutation;
execute_unload 'debug.gdx';
file fpy / 'debug.py' /; put fpy;
$onPutS
from gamsemb import *
gams = ECGAMSDatabase(r'%gams.sysdir% '[:-2],'debug.gdx')
try:
import random
i = list(gams.get("i"))
p = list(i)
random.shuffle(p)
for idx in range(len(i)):
p[idx] = (i[idx], p[idx])
gams.set("p", p)
gmdWriteGDX(gams._gmd,'debugOut.gdx',1);
except Exception as e:
print(str(e))
$offPut
putClose fpy;
execute.checkErrorLevel '="%gams.sysdir%GMSPython/python" debug.py';
execute_load 'debugOut.gdx', p;
Option p:0:0:1;
Display p;
Automatic Indentation
All embedded Python code is automatically wrapped in a try-except
block that requires an indentation (two spaces). While this additional indentation makes no difference for most Python code, it does make a difference when it comes to multi-line strings. Therefore, multi-line strings created with triple quotes (either single or double quotes) are excluded from this rule to avoid unintentional spaces, e.g. when writing to a file:
$onEmbeddedCode Python:
f = open("file.txt", "w")
f.write('''some
multi
line
string
''')
f.close()
$offEmbeddedCode
Still, when using the line continuation character (\
) in combination with strings, one has to be aware of the extra spaces that are added automatically:
$onEmbeddedCode Python:
s = "some\
multi\
line\
string"
gams.printLog(s)
$offEmbeddedCode
This will print some multi line string
instead of somemultilinestring
due to the automatically applied indentation.
Performance Considerations of Embedded Python Code
If the same embedded code section (e.g. in a loop) is executed many times there are a few considerations to be taken into account in order to get the best performance. For this we will experiment with the example from the introduction. We look for a random permutation of a set i
. In addition we have a cost matrix c(i,ii)
and we are looking for the least cost permutation. We should just formulate this as matching in a bi-partite graph but in order to demonstrate some performance considerations we will repeatedly call the Python code that provides a random permutation and we will evaluate the cost of the permutation in GAMS and store the value of the cheapest one. Here is the naive implementation using embedded code:
Set i / i1*i50 /
p(i,i) permutation;
Alias (i,ii);
Parameter c(i,i) cost of permutation;
c(i,ii) = uniform(-50,50);
Set iter / 1*100 /;
Scalar tcost
minTCost / +inf /;
loop(iter,
embeddedCode Python:
import random
i = list(gams.get("i"))
p = list(i)
random.shuffle(p)
gams.set("p", [ (i[idx], p[idx]) for idx in range(len(i)) ])
endEmbeddedCode p
tcost = sum(p, c(p));
if (tcost < minTCost, minTCost = tcost);
);
Display minTCost;
Even though the Python interpreter stays alive the fresh start and end of an embedded code section goes together with the setup and initialization of multiple data structure, including the gams
object of type ECGAMSDatabase
. We can avoid the repeated setup and initialization by using pause
and continue
:
Set i / i1*i50 /
p(i,i) permutation;
Alias (i,ii);
Parameter c(i,i) cost of permutation;
c(i,ii) = uniform(-50,50);
embeddedCode Python:
import random
pauseEmbeddedCode
Set iter / 1*100 /;
Scalar tcost
minTCost / +inf /;
loop(iter,
continueEmbeddedCode:
i = list(gams.get("i"))
p = list(i)
random.shuffle(p)
gams.set("p", [ (i[idx], p[idx]) for idx in range(len(i)) ])
pauseEmbeddedCode p
tcost = sum(p, c(p));
if (tcost < minTCost, minTCost = tcost);
);
continueEmbeddedCode:
pass
endEmbeddedCode
Display minTCost;
The last embedded code execution of the Python pass
statement is to clean up and terminate the Python session. Since the amount of data is relatively small, there is little difference in the running time between these variants. The performance impact of independent Python session (command line option pyMultInst) is significant. While the pause/continue code runs at the same speed (roughly 0.2 seconds) independent of the pyMultInst setting, the original code runs much slower (about 6.6 seconds) because every iteration results in a newly created session.
There are other performance considerations. We can actually extract the set i
once before the GAMS loop starts. Moreover, we can work with label indexes rather than the labels itself. Indexes are integers and are often faster than labels that are stored as strings. The only difference to the code above is the extraction method of the Python list i
by i = list(gams.get("i",keyType=KeyType.INT))
.
Extending GMSPython
While we recommend to use your own Python installation if you need additional packages, there are ways to extend the Python system that comes with GAMS in GMSPython
. Here are the steps:
- Get
pip
viaget-pip
(https://pip.pypa.io/en/stable/installing/) - Update
pip
- Install additional packages
We highly recommend installing pip
and additional packages in the user site (use --user
when running get-pip
and pip
). This is especially important for macOS users. Installing files in the GAMS system directory may interfere with the file notarization and may prevent GAMS from starting properly. Here is a typical dialog for Linux/macOS using curl
to download get-pip.py
. If you don't have curl
or wget
, use your web browser for downloading.
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py /path/to/gams/GMSPython/python get-pip.py --user ~/.local/bin/pip install -U pip ~/.local/bin/pip install geocoder --user
Using an External Python 3 Installation
Let's assume you don't want to work with the GAMS distributed GMSPython
but you want to connect GAMS with a Python 3 installation of your liking. In order for the Embedded Code Facility to work, you need to make the alternative Python interpreter aware of the required modules provided by GAMS by following the Getting started steps in the Python API tutorial.
In addition to this you need to do one more step to allow GAMS to find your Python installation when executing Embedded Python code: you need to point to the Python dynamic load library or shared object. On Windows this file is called python3X.dll
, on Linux libpython3.X.so
, and on macOS libpython3.X.dylib
. The X
stands for the minor version number of Python 3. While embedded code has been built and tested for Python 3.8, chances are that you can also point to other Python 3 libraries. Since the expert level API is required, the set of versions is limited to what GAMS supports (currently GAMS ships these for Python >=3.6). Limited experiments with the embedded code library for Python 3.8 with Python 3.6, 3.7, and 3.9 libraries were successful. Python 3.10 is not supported yet. You need to set the environment variable GMSPYTHONLIB
to point GAMS to the Python library:
- Windows: set GMSPYTHONLIB=c:\path\to\python\python38.dll
- Linux: export GMSPYTHONLIB=/path/to/python/lib/libpython3.8.so
- Mac OSX: export GMSPYTHONLIB=/path/to/python/lib/libpython3.8.dylib
GAMS will extract the Python home from the location of the Python library. In case this extraction does not work (because of an unconventional Python installation), you can instruct GAMS to set the Python home via the additional environment variable GMSPYTHONHOME
. Please note, that this environment variable only needs to be set for GAMS Embedded Python code, so it is sufficient to add these variables to the GAMS configuration YAML file.
- Note
- When using environment-based Python distributions like
Anaconda
orMiniconda
, it is not sufficient to setGMSPYTHONLIB
(andGMSPYTHONHOME
). The environment to be used needs to be activated properly before (e.g.conda activate myEnv
). Packages likenumpy
which have been installed using conda (conda install numpy
) won't work otherwise. In addition to settingGMSPYTHONLIB
we therefore recommend to start GAMS itself and also tool likeGAMS Studio
from an activated conda environment in order to make sure that the environment and all installed packages work as expected.
Although it is recommended to use a self-contained Python installation, it is possible to use a virtual environment instead (e.g. via venv
). In this case, GMSPYTHONLIB
has to be set to the Python library of the installation from which the environment was derived from. In additon, the PYTHONPATH
environment variable needs to be set to the site-packages
directory of the virtual environment in order to make its packages available. Note that with this setup the site-packages
directory of the original Python installation will also be found by GAMS Embedded Python code.
Building your own Embedded Python Code Library
Although the Embedded Code Facility and its binary components are part of the GAMS distribution, it is possible to build it manually from source using the following commands. The exact command line might change depending on the compiler and the operating system in use:
- Windows:
cd [GAMS directory]/apifiles/C/api icl.exe -Feembpycclib64.dll -IC:\path\to\python\include embpyoo.c gmdcc.c gclgms.c -LD -link -nodefaultlib:python38.lib
- Linux:
cd [GAMS directory]/apifiles/C/api gcc -o libembpycclib64.so -nostartfiles -shared -Wl,-Bsymbolic -m64 -pthread -fPIC -Ipath/to/your/python/include/python3.8 embpyoo.c gmdcc.c gclgms.c -lm -ldl
- Mac OS X:
cd [GAMS directory]/apifiles/C/api gcc -o libembpycclib64.dylib -dynamiclib -shared -m64 -install_name @rpath/libembpycclib64.dylib -Ipath/to/your/python/include/python3.8 embpyoo.c gmdcc.c gclgms.c -lm -ldl
Connect
The Connect framework gives unified and platform independent access to data exchange with different formats (e.g. CSV and Excel). The instructions how to access the various data sources are given in YAML syntax. These YAML instructions are the code for the embedded Connect code. The embedded Connect code engine will parse the YAML instructions and use Connect agents to act upon the instructions. The agents GAMSReader and GAMSWriter send data back and forth to GAMS. A simple example is given above.
Embedded code Connect is build on top of embedded code Python. Similar considerations (e.g. Python interpreter is loaded once) carry over from Python to Connect. Moreover, the agent PythonCode has access to the object gams
of type ECGamsDatabase
which is used for communication of data with GAMS. In contrast to embedded Python code, the code of the agent PythonCode
does not execute in the global scope but in a separate block. This means that variables set in the code are not available in another PythonCode
section. The following code ends up in the except clause and prints Cannot access my_i
.
embeddedCode Connect:
- PythonCode:
code: |
my_i = 1
- PythonCode:
code: |
try:
gams.printLog(f'{my_i}')
except:
gams.printLog('Cannot access my_i')
endEmbeddedCode
Since Python allows to add attributes to an object at runtime, the my_i
variable can be added to the gams
object and hence becomes available in different PythonCode
sections and even in different embedded Connect code parts (when pause/continue is used):
embeddedCode Connect:
- PythonCode:
code: |
gams.my_i = 1
- PythonCode:
code: |
gams.printLog(f'{gams.my_i}')
pauseEmbeddedCode
continueEmbeddedCode:
- PythonCode:
code: |
gams.printLog(f'{gams.my_i}')
endEmbeddedCode
Even though embedded code Python and Connect share the same basis, they run in different instantiations of the Python interpreter and hence don't share e.g. the same gams
object.
Substitutions in Embedded Connect Code
The GAMS compiler has mechanisms to substitute compile-time variables to parameterize GAMS code including embedded code (unless the V
, e.g. $onEmbeddedCodeV
, of embedded code is used). Here is an example how to use double-dash parameters to allow the user to set the CSV file name via the GAMS command line (gams mymodel.gms --csvfilename=mystockprice.csv
):
$if not set CSVFILENAME $set CSVFILENAME stockprice.csv
$onEmbeddedCode Connect:
- CSVReader:
file: %CSVFILENAME%
name: stockprice
indexColumns: [1, 2]
valueColumns: [3]
- GAMSWriter:
symbols:
- name: stockprice
$offEmbeddedCode
Not only user defined compile-time variables will be substituted, but also all other compile time variables, e.g. %gams.scrDir%
or %system.dirSep%
. Another way to pass substitution instructions down to embedded Connect code are the embedded code arguments
:
$if not set MYCSVFILENAME $set MYCSVFILENAME stockprice.csv
$onEmbeddedCodeV Connect: --CSVFILENAME=%MYCSVFILENAME%
- CSVReader:
file: %CSVFILENAME%
name: stockprice
indexColumns: [1, 2]
valueColumns: [3]
- GAMSWriter:
symbols:
- name: stockprice
$offEmbeddedCode
Together with the put_utility command ECArguments
that appends some arbitrary text to the embedded code arguments this can be used to parameterize the embedded Connect code at execution time, e.g. to create files based on label names:
Set i /1*5/; Parameter p(i);
loop(i,
p(i) = uniform(0,1);
put_utility 'ECArguments' / '--CSVFILENAME=' i.tl:0 '.csv';
EmbeddedCodeV Connect:
- GAMSReader:
symbols:
- name: p
- CSVWriter:
name: p
file: %CSVFILENAME%
endEmbeddedCode
);
Please note that while the GAMS command line parameters allow various formats to specify double-dash parameters, the embedded code Connect substitution arguments must be in the form --key=value
, so a =
character to separate key and value with no spaces. The value
might be a (double) quoted string though. Please also note that the embedded code arguments set with put_utility command ECArguments
will be applied to any subsequent embedded code section. So one might want to clear the argument by adding a put_utility "ECArguments" / "";
after the relevant embedded code section.