GAMS Transfer

GAMS Transfer is a package to maintain GAMS data outside a GAMS script in a programming language like Python or Matlab. It allows the user to add GAMS symbols (Sets, Parameters, Variables and Equations), to manipulate GAMS symbols, as well as read/write symbols to different data endpoints. GAMS Transfer’s main focus is the highly efficient transfer of data between GAMS and the target programming language, while keeping those operations as simple as possible for the user. In order to achieve this, symbol records – the actual and potentially large-scale data sets – are stored in native data structures of the corresponding programming languages. The benefits of this approach are threefold: (1) The user is usually very familiar with these data structures, (2) these data structures come with a large tool box for various data operations, and (3) optimized methods for reading from and writing to GDX can transfer the data as a bulk – resulting in the high performance of this package. This tutorial describes, in detail, the use of GAMS Transfer within a Python environment.

Data within GAMS Transfer will be stored as Pandas DataFrame. The flexible nature of Pandas DataFrames makes them ideal for storing/manipulating sparse data. Pandas includes advanced operations for indexing and slicing, reshaping, merging and even visualization.

Pandas also includes a number of advanced data I/O tools that allow users to generate DataFrames directly from CSV (.csv), JSON (.json), HTML (.html), Microsoft Excel (.xls, .xlsx), SQL , pickle (.pkl), SPSS (.sav, .zsav), SAS (.xpt, .sas7bdat), etc.

Centering GAMS Transfer around the Pandas DataFrame gives GAMS users (on a variety of platforms – MacOS, Windows, Linux) access to tools to move data back and forth between their favorite environments for use in their GAMS models.

The goal of this tutorial is to introduce the user to GAMS Transfer and its functionality. This tutorial is not designed to teach the user how to effectively manipulate Pandas DataFrames; users seeking a deeper understanding of Pandas are referred to the extensive documentation.

# Install

The user must download and install the latest version of GAMS in order to install GAMS Transfer. GAMS Transfer is installed when the GAMS Python API is built and installed. The user is referred HERE for instructions on how to install the Python API files. GAMS Transfer and all GAMS Python API files are compatible with environment managers such as Anaconda.

# Design

Storing, manipulating, and transforming sparse data requires that it live within an environment – this data can then be linked together to enable various operations. In GAMS Transfer we refer to this "environment" as the Container, it is the main repository for storing and linking our sparse data. Symbols can be added to the Container from a variety of GAMS starting points but they can also be generated directly within the Python environment using convenient function calls that are part of the GAMS Transfer package; a symbol can only belong to one container at a time.

The process of linking symbols together within a container was inspired by typical GAMS workflows but leverages aspects of object oriented programming to make linking data a natural process. Linking data enables data operations like implicit set growth, domain checking, data format transformations (to dense/sparse matrix formats), etc. All of these details will be discussed in the following sections. For now, we highlight a few key aspects of GAMS Transfer that will help lay the foundation. Users can skip directly to examples using the Table of Contents.

## GAMS Special Values

The GAMS system contains five special values: UNDEF (undefined), NA (not available), EPS (epsilon), +INF (positive infinity), -INF (negative infinity). These special values must be mapped to their Python equivalents. GAMS Transfer follows the following convention to generate the 1:1 mapping:

• +INF is mapped to float("inf")
• -INF is mapped to float("-inf")
• EPS is mapped to -0.0 (mathematically identical to zero)
• NA is mapped to a special NaN
• UNDEF is mapped to float("nan")

GAMS Transfer syntax is designed to quickly get data into a form that is usable in further analyses or visualization; this mapping also highlights the preference for data that is of type float, which offers performance benefits within Pandas/NumPy. The user does not need to remember these constants as they are provided within the class SpecialValues as SpecialValues.POSINF, SpecialValues.NEGINF, SpecialValues.EPS, SpecialValues.NA, and SpecialValues.UNDEF. The SpecialValues class also contains methods to test for these special values. Some examples are shown below; already, we, begin to introduce some of the GAMS Transfer syntax.

Example (special values in a parameter)
from gamstransfer import *
m = Container()
x = Parameter(
m,
"x",
["*"],
records=[
("i1", 1),
("i2", SpecialValues.POSINF),
("i3", SpecialValues.NEGINF),
("i4", SpecialValues.EPS),
("i5", SpecialValues.NA),
("i6", SpecialValues.UNDEF),
],
description="special values",
)

The following DataFrame for x would look like:

In [1]: x.records
Out[1]:
uni_0 value
0 i1 1.0
1 i2 inf
2 i3 -inf
3 i4 -0.0
4 i5 NaN
5 i6 NaN

The user can now easily test for specific special values in the value column of the DataFrame (returns a boolean array):

In [1]: SpecialValues.isNA(x.records["value"])
Out[1]: array([False, False, False, False, True, False])

Other data structures can be passed into these methods as long as these structures can be converted into a numpy array with dtype=float. It follows that:

In [1]: SpecialValues.isEps(SpecialValues.EPS)
Out[1]: True
In [2]: SpecialValues.isPosInf(SpecialValues.POSINF)
Out[2]: True
In [3]: SpecialValues.isNegInf(SpecialValues.NEGINF)
Out[3]: True
In [4]: SpecialValues.isNA(SpecialValues.NA)
Out[4]: True
In [5]: SpecialValues.isUndef(SpecialValues.UNDEF)
Out[5]: True
In [6]: SpecialValues.isUndef(SpecialValues.NA)
Out[6]: False
In [6]: SpecialValues.isNA(SpecialValues.UNDEF)
Out[6]: False

Pandas DataFrames allow data columns to exist with mixed type (dtype=object). As such, GAMS Transfer allows special string representations ("+inf", "-inf", "eps", "na", or "undef") of these special values when setting symbol records as seen in the following example.

Example (special values defined by strings)
from gamstransfer import *
m = Container()
x = Parameter(
m,
"x",
["*"],
records=[
("i1", 1),
("i2", "+inf"),
("i3", "-inf"),
("i4", "eps"),
("i5", "na"),
("i6", "undef"),
],
description="special values",
)

These special strings will be immediately mapped to their float equivalents from the SpecialValues class in order to ensure that all data entries are float types.

Note
Python is case sensitive. GAMS Transfer is strict such that EPS will not map to -0.0 and an error will be generated; the user is responsible for transforming their data to use the lower case representations of special values.

## GAMS Transfer Standard Data Formats

This section is meant to introduce the standard format that GAMS Transfer expects for symbol records. It has already been mentioned that we store data as a Pandas DataFrame, but there is an assumed structure to the column headings and column types that will be important to understand. GAMS Transfer includes convenience functions in order to ease the burden of converting data from a user-centric format to one that is understood by GAMS Transfer. However, advanced users will want to convert their data first and add it directly to the Container to avoid making extra copies of (potentially large) data sets.

Set Records Standard Format

All set records (including singleton sets) are stored as a Pandas DataFrame with n number of columns, where n is the dimensionality of the symbol + 1. The first n-1 columns include the domain elements while the last column includes the set element explanatory text. Records are organized such that there is one record per row.

The names of the domain columns follow a pattern of <set_name>_<index_position>; a symbol dimension that is referenced to the universe is labeled uni_<index position>. The explanatory text column is called element_text and must take the last position in the DataFrame.

All domain columns must be a categorical data type and the element_text column must be a object type. Pandas allows the categories (basically the unique elements of a column) to be various data types as well, however GAMS Transfer requires that all these are type str. All rows in the element_text column must be type str.

Some examples:

m = Container()
i = Set(m, "i", records=["seattle", "san-diego"])
j = Set(m, "j", [i, "*"], records=[("seattle", "new-york"), ("san-diego", "st-louis")])
k = Set(m, "k", [i], is_singleton=True, records=["seattle"])
In [1]: i.records
Out[1]:
uni_0 element_text
0 seattle
1 san-diego
In [2]: j.records
Out[2]:
i_0 uni_1 element_text
0 seattle new-york
1 san-diego st-louis
In [3]: k.records
Out[3]:
i_0 element_text
0 seattle
Parameter Records Standard Format

All parameter records (including scalars) are stored as a Pandas DataFrame with n number of columns, where n is the dimensionality of the symbol + 1. The first n-1 columns include the domain elements while the last column includes the numerical value of the records. Records are organized such that there is one record per row. Scalar parameters have zero dimension, therefore they only have one column and one row.

The names of the domain columns follow a pattern of <set_name>_<index_position>; a symbol dimension that is referenced to the universe is labeled uni_<index_position>. The value column is called value and must take the last position in the DataFrame.

All domain columns must be a categorical data type and the value column must be a float type. Pandas allows the categories (basically the unique elements of a column) to be various data types as well, however GAMS Transfer requires that all these are type str.

Some examples:

m = Container()
i = Set(m, "i", records=["seattle", "san-diego"])
a = Parameter(m, "a", ["*"], records=[("seattle", 50), ("san-diego", 100)])
b = Parameter(
m,
"b",
[i, "*"],
records=[("seattle", "new-york", 32.2), ("san-diego", "st-louis", 123)],
)
c = Parameter(m, "c", records=90)
In [1]: a.records
Out[1]:
uni_0 value
0 seattle 50.0
1 san-diego 100.0
In [2]: b.records
Out[2]:
i_0 uni_1 value
0 seattle new-york 32.2
1 san-diego st-louis 123.0
In [3]: c.records
Out[3]:
value
0 90.0
Variable/Equation Records Standard Format

Variables and equations share the same standard data format. All records (including scalar variables/equations) are stored as a Pandas DataFrame with n number of columns, where n is the dimensionality of the symbol + 5. The first n-5 columns include the domain elements while the last five columns include the numerical values for different attributes of the records. Records are organized such that there is one record per row. Scalar variables/equations have zero dimension, therefore they have five columns and one row.

The names of the domain columns follow a pattern of <set_name>_<index position>; a symbol dimension that is referenced to the universe is labeled uni_<index_position>. The attribute columns are called level, marginal, lower, upper, and scale. These attribute columns must appear in this order. Attributes that are not supplied by the user will be assigned the default GAMS values for that variable/equation type; it is possible to not pass any attributes, GAMS Transfer would then simply assign default values to all attributes.

All domain columns must be a categorical data type and all the attribute columns must be a float type. Pandas allows the categories (basically the unique elements of a column) to be various data types as well, however GAMS Transfer requires that all these are type str.

Some examples:

from gamstransfer import *
import pandas as pd
m = Container()
i = Set(m, "i", records=["seattle", "san-diego"])
a = Variable(
m,
"a",
"free",
domain=[i],
records=pd.DataFrame(
[("seattle", 50), ("san-diego", 100)], columns=["city", "level"]
),
)
In [1]: a.records
Out[1]:
i_0 level marginal lower upper scale
0 seattle 50.0 0.0 -inf inf 1.0
1 san-diego 100.0 0.0 -inf inf 1.0

# Core Functionality

The reader has already had glimpses of the GAMS Transfer syntax, but in this section we will present more details about the core functionalty and dig further into the syntax. Specifically, we will discuss:

1. How to create a Container
2. How to add symbols to a Container
3. How to validate the data that is in the Container
4. How to defined domains implicitly with domain_forwarding
5. How to describe the data that is in the Container
6. How to transform data that is in the Container
7. Understand what the universe set is (UEL list)
8. How to reorder symbols in the Container
9. How to remove symbols from the Container
10. Tips for debugging data

All of these operations pertain specifically to the Container and the symbols within the Container. We will leave a detailed discussion about reading/writing data from/to GDX files until the next section.

## Creating a Container

The main object class within GAMS Transfer is called Container. The Container is the vessel that allows symbols to be linked together (through their domain definitions), it enables implicit set definitions, it enables structural manipulations of the data (matrix generation), and it allows the user to perform different read/write operations. Creating a container is a simple matter of initializing an object. For example:

from gamstransfer import *
m = Container()

This new Container object, here called m, contains a number of convenient properties and methods that allow the user to interact with the symbols that are in the Container. Some of these properties are simply used to filter out different types of symbols, other methods are used to numerically characterize the data within each symbol.

Symbols are organized in the container under the data container attribute. The dot notation (m.data) is used to access the underlying dict; symbols in this dict can then be retrieved with the standard bracket notation (m.data[<symbol_name>]). For convenience, there are list* methods that will filter the different types of symbols and return a list. Those methods and their arguments are described below:

Method Argument Type Description Required Default
listSymbols is_valid bool returns a list of symbols that are valid (is_valid=True), invalid (is_valid=False), or all symbols (is_valid=None) No None
listSets is_valid bool returns a list of sets that are valid (is_valid=True), invalid (is_valid=False), or all symbols (is_valid=None) No None
listAliases is_valid bool returns a list of aliases that are valid (is_valid=True), invalid (is_valid=False), or all symbols (is_valid=None) No None
listParameters is_valid bool returns a list of parameters that are valid (is_valid=True), invalid (is_valid=False), or all symbols (is_valid=None) No None
listVariable is_valid bool returns a list of variables that are valid (is_valid=True), invalid (is_valid=False), or all symbols (is_valid=None) No None
listEquations is_valid bool returns a list of equations that are valid (is_valid=True), invalid (is_valid=False), or all symbols (is_valid=None) No None

A valid symbol is one that has passed a number of internal quality checks to ensure that this data is in the GAMS Transfer standard format and is self-consistent. More details on data validity will be provided in Validating Data.

As an example, a Container that has been populated with symbol data from the trnsport.gms (we will discuss how to exchange data with GDX in Data Exchange with GDX) model would look like:

In [1]: m.data
Out[1]:
{'i': <src.gamstransfer.Set at 0x7fa2387750a0>,
'j': <src.gamstransfer.Set at 0x7fa238e74fa0>,
'a': <src.gamstransfer.Parameter at 0x7fa238e74cd0>,
'b': <src.gamstransfer.Parameter at 0x7fa238e746a0>,
'd': <src.gamstransfer.Parameter at 0x7fa23876b370>,
'f': <src.gamstransfer.Parameter at 0x7fa23876b400>,
'c': <src.gamstransfer.Parameter at 0x7fa23876b5e0>,
'x': <src.gamstransfer.Variable at 0x7fa23876b340>,
'z': <src.gamstransfer.Variable at 0x7fa23876b640>,
'cost': <src.gamstransfer.Equation at 0x7fa23876b2b0>,
'supply': <src.gamstransfer.Equation at 0x7fa23876b310>,
'demand': <src.gamstransfer.Equation at 0x7fa23876b460>}

We can then call the list* methods to explore the different types of symbols. The order of these lists mirror the order of appearance in the m.data dict.

In [2]: m.listSets()
Out[2]: ['i', 'j']
In [3]: m.listParameters()
Out[3]: ['a', 'b', 'd', 'f', 'c']
In [4]: m.listVariables()
Out[4]: ['x', 'z']
In [5]: m.listEquations()
Out[5]: ['cost', 'supply', 'demand']
In [6]: m.listAliases()
Out[6]: []

## Creating a Symbol

There are two different ways to create a symbol and add it to a Container.

• Use Container methods addSet, addAlias, addParameter, addVariable, and addEquation
• Use symbol constructors Set, Alias, Parameter, Variable and Equation The following tables detail the arguments used to create each symbol type and their default values.

The following table detail the argument and their default values for the symbol constructors.

Set constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name str Name of symbol Yes -
domain list List of domains given either as string ('*' for universe set) or as reference to a Set object No ["*"]
is_singleton bool Indicates if set is a singleton set (True) or not (False) No False
records many Symbol records No None
domain_forwarding bool Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No False
description str Description of symbol No ""
Alias constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name str Name of symbol Yes -
alias_with Set or Alias Set an alias is linked to Yes -
Parameter constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name str Name of symbol Yes -
domain list List of domains given either as string ('*' for universe set) or as reference to a Set object, an empty domain list will create a scalar parameter No []
records many Symbol records No None
domain_forwarding bool Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No False
description str Description of symbol No ""
Variable constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name str Name of symbol Yes -
type str Type of variable being created ["binary", "integer", "positive", "negative", "free", "sos1", "sos2", "semicont", "semiint"] No "free"
domain list List of domains given either as string ('*' for universe set) or as reference to a Set object, an empty domain list will create a scalar variable No []
records many Symbol records No None
domain_forwarding bool Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No False
description str Description of symbol No ""
Equation constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name str Name of symbol Yes -
type str Type of equation being created ['eq' (or 'E'/'e'), 'geq' (or 'G'/'g'), 'leq' (or 'L'/'l'), 'nonbinding' (or 'N'/'n'), 'cone' (or 'C'/'c'), 'external' (or 'X'/'x'), 'boolean' (or 'B'/'b')] Yes -
domain list List of domains given either as string ('*' for universe set) or as reference to a Set object, an empty domain list will create a scalar equation No []
records many Symbol records No None
domain_forwarding bool Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No False
description str Description of symbol No ""

The other symbol properties are implied by the arguments that the user specifies. For example, dimension is the number of elements in the domain list. All the above symbol properties can be modified at any time after the addition to the container, however, any changes could result in an invalid symbol. Care must be taken when modifying data directly.

Note
If any domain element is passed as string except for "*" (universe set), the domain_type of the symbol will be labeled as a relaxed symbol rather than regular. This means that domain checking will not be performed.
When using the Container methods to create new symbols (addSet, addParameter, addAlias, addVariable, addEquation) it is not necessary to pass the container reference as an argument. All other arguments have the same structure as in the tables above.

Three possibilities exist to assign symbol records to a set:

• Setting the property records directly
• Using the symbol method setRecords
• Setting the argument records in a symbol constructor/container method (internally calls setRecords)

If data is assigned to the records property directly, the user must take care that the data follow the standard format. If the user does not follow this format the symbol will be marked as invalid and therefore cannot be written to the data endpoint. Users that are concerned about memory use may want to set the records directly because this avoids creating a copy of the data in memory.

The symbol method setRecords is a convenience method that transforms the given data into an approved Pandas DataFrame format. Many native python data types can be easily transformed into DataFrames, so the setRecords method for Set objects will accept a number of different types for input. The setRecords method is called internally on any data structure that is passed through the records argument.

Example (create a 1D set from a list)
m = Container()
i = Set(m, "i", records=["seattle", "san-diego"])
In [1]: i.records
Out[1]:
uni_0 element_text
0 seattle
1 san-diego
Example (create a 1D set from a tuple)
j = Set(m, "j", records=("seattle", "san-diego"))
In [1]: j.records
Out[1]:
uni_0 element_text
0 seattle
1 san-diego
Example (create a 2D set from a list of tuples)
k = Set(m, "k", ["*", "*"], records=[("seattle", "san-diego")])
In [1]: k.records
Out[1]:
uni_0 uni_1 element_text
0 seattle san-diego
Example (create a 1D set from a DataFrame slice + unique())
dist = pd.DataFrame(
[
("seattle", "new-york", 2.5),
("seattle", "chicago", 1.7),
("seattle", "topeka", 1.8),
("san-diego", "new-york", 2.5),
("san-diego", "chicago", 1.8),
("san-diego", "topeka", 1.4),
],
columns=["from", "to", "thousand_miles"],
)
l = Set(m, "l", records=dist["from"].unique())
In [1]: l.records
Out[1]:
uni_0 element_text
0 seattle
1 san-diego
Note
The .unique() method preserves the order of appearance, unlike set().

Set element text is very handy when labeling specific set elements within a set. A user can add a set element text directly with a set element. Note that it is not required to label all set elements, as can be seen in the following example.

m = Container()
i = Set(
m,
"i",
records=[
("seattle", "home of sub pop records"),
("san-diego",),
("washington_dc", "former gams hq"),
],
)
In [1]: i.records
Out[1]:
uni_0 element_text
0 seattle home of sub pop records
1 san-diego
2 washington_dc former gams hq
Note
It is not possible to "add records" to an alias because an alias simply points to another set. If an alias is made from another alias, GAMS Transfer will recursively search for the root set and point directly to this set.

Three possibilities exist to assign symbol records to a parameter:

• Setting the property records directly
• Using the symbol method setRecords
• Setting the argument records in a symbol constructor/container method (internally calls setRecords)

If data is assigned to the records property directly, the user must take care that the data follow the standard format. If the user does not follow this format the symbol will be marked as invalid and therefore cannot be written to the data endpoint. Users that are concerned about memory use may want to set the records directly because this avoids creating a copy of the data in memory.

The symbol method setRecords is a convenience method that transforms the given data into an approved Pandas DataFrame format. Many native python data types can be easily transformed into DataFrames, so the setRecords method for Parameter objects will accept a number of different types for input. The setRecords method is called internally on any data structure that is passed through the records argument.

Example (create a 1D parameter from a list of tuples)
m = Container()
a = Parameter(m, "a", ["*"], records=[("seattle", 100), ("san-diego", 64)])
In [1]: a.records
Out[1]:
uni_0 value
0 seattle 100.0
1 san-diego 64.0
Example (create a 2D parameter from a tuple)
b = Parameter(
m,
"b",
["*", "*"],
records=[("seattle", "san-diego", 143), ("chicago", "madison", 147.3)],
)
In [1]: b.records
Out[1]:
uni_0 uni_1 value
0 seattle san-diego 143.0
Example (create a 2D set from a DataFrame)
dist = pd.DataFrame(
[
("seattle", "new-york", 2.5),
("seattle", "chicago", 1.7),
("seattle", "topeka", 1.8),
("san-diego", "new-york", 2.5),
("san-diego", "chicago", 1.8),
("san-diego", "topeka", 1.4),
],
columns=["from", "to", "thousand_miles"],
)
c = Parameter(m, "c", ["*", "*"], records=dist)
In [1]: c.records
Out[1]:
uni_0 uni_1 value
0 seattle new-york 2.5
1 seattle chicago 1.7
2 seattle topeka 1.8
3 san-diego new-york 2.5
4 san-diego chicago 1.8
5 san-diego topeka 1.4
Note
When passing a DataFrame in the records argument a deep copy of the data will be made. This copy will be converted to the GAMS Transfer standard format leaving the other copy untouched.

Three possibilities exist to assign symbol records to a variable:

• Setting the property records directly
• Using the symbol method setRecords
• Setting the argument records in a symbol constructor/container method (internally calls setRecords)

If data is assigned to the records property directly, the user must take care that the data follow the standard format. If the user does not follow this format the symbol will be marked as invalid and therefore cannot be written to the data endpoint. Users that are concerned about memory use may want to set the records directly because this avoids creating a copy of the data in memory.

The symbol method setRecords is a convenience method that transforms the given data into an approved Pandas DataFrame format. The setRecords method is called internally on any data structure that is passed through the records argument. The structure for Variable symbols is more complicated so the user must present setRecords with a DataFrame, however the user does not need to supply all necessary columns (i.e., "level", "marginal", "lower", "upper" and "scale"). Any columns that are not supplied will be created and filled with their GAMS default values.

Example (create a 1D free variable, specifying only level values)
m = Container()
x = Variable(
m,
"x",
"free",
["*"],
records=pd.DataFrame(
[("seattle", 100), ("san-diego", 64)], columns=["cities", "level"]
),
)
In [1]: x.records
Out[1]:
uni_0 level marginal lower upper scale
0 seattle 100.0 0.0 -inf inf 1.0
1 san-diego 64.0 0.0 -inf inf 1.0
Example (create a scalar variable)
z = Variable(m, "z", "free", records=pd.DataFrame([100], columns=["level"]))
In [1]: z.records
Out[1]:
level marginal lower upper scale
0 100.0 0.0 -inf inf 1.0
Example (create a 2D positive variable, specifying no numerical data)
b = Variable(
m,
"b",
"positive",
["*", "*"],
)
In [1]: b.records
Out[1]:
uni_0 uni_1 level marginal lower upper scale
0 seattle san-diego 0.0 0.0 0.0 inf 1.0
1 chicago madison 0.0 0.0 0.0 inf 1.0

Three possibilities exist to assign symbol records to an equation:

• Setting the property records directly
• Using the symbol method setRecords
• Setting the argument records in a symbol constructor/container method (internally calls setRecords)

If data is assigned to the records property directly, the user must take care that the data follow the standard format. If the user does not follow this format the symbol will be marked as invalid and therefore cannot be written to the data endpoint. Users that are concerned about memory use may want to set the records directly because this avoids creating a copy of the data in memory.

The symbol method setRecords is a convenience method that transforms the given data into an approved Pandas DataFrame format. The setRecords method is called internally on any data structure that is passed through the records argument. The structure for Equation symbols is more complicated so the user must present setRecords with a DataFrame, however the user does not need to supply all necessary columns (i.e., "level", "marginal", "lower", "upper" and "scale"). Any columns that are not supplied will be created and filled with their GAMS default values.

Example (create a 1D =L= equation, specifying only level values)
m = Container()
t = Equation(
m,
"t",
"leq",
["*"],
records=pd.DataFrame(
[("seattle", 100), ("san-diego", 64)], columns=["cities", "level"]
),
)
In [1]: t.records
Out[1]:
uni_0 level marginal lower upper scale
0 seattle 100.0 0.0 -inf inf 1.0
1 san-diego 64.0 0.0 -inf inf 1.0
Example (create a scalar =E= equation)
obj = Equation(m, "obj", "eq", records=pd.DataFrame([100], columns=["level"]))
In [1]: obj.records
Out[1]:
level marginal lower upper scale
0 100.0 0.0 -inf inf 1.0

## Validating Data

GAMS Transfer requires that the records for all symbols exist in a standard format (GAMS Transfer Standard Data Formats) in order for them to be understood and written successfully. It is certainly possible that the data could end up in a state that is inconsistent with the standard format (especially if setting symbol attributes directly). GAMS Transfer includes the .isValid() method in order to determine if a symbol is valid and ready for writing; this method returns a bool. For example, we create two valid sets and then check them with .isValid() to be sure.

Note
It is possible to run .isValid() on both the Container as well as the symbol object – .isValid() will also return a bool if there are any invalid symbols in the Container object.
Example (valid data)
from gamstransfer import *
m = Container()
i = Set(m, "i", records=["seattle", "san-diego", "washington_dc"])
j = Set(m, "j", i, records=["san-diego", "washington_dc"])
In [1]: i.isValid()
Out[1]: True
In [2]: j.isValid()
Out[2]: True
In [3]: m.isValid()
Out[3]: True

Now we create some data that is invalid due to domain violations in the set j.

Example (intentionally create domain violations)
from gamstransfer import *
m = Container()
i = Set(m, "i", records=["seattle", "san-diego", "washington_dc"])
j = Set(m, "j", i, records=["grayslake", "washington_dc"])
In [1]: i.isValid()
Out[1]: True
In [2]: j.isValid()
Out[2]: False
In [3]: m.isValid()
Out[3]: False

In this example, we know that the validity of the data is compromised by the domain violations, but there could be other subtle descrepancies that must be remedied before writing data. The user can get more detailed error reporting if the verbose argument is set to True. For example:

In [1]: j.isValid(verbose=True)
Exception: Symbol 'records' contain domain violations; ensure that all domain elements have been mapped properly to a category

## Domain Forwarding

GAMS includes the ability to define sets directly from data using the implicit set notation (see: Implicit Set Definition (or: Domain Defining Symbol Declarations)). This notation has an analogue in GAMS Transfer called domain_forwarding.

Note
It is possible to recursively update a subset tree in GAMS Transfer.

Domain forwarding is available as an argument to all symbol object constructors; the user would simply need to pass domain_forwarding=True.

In this example we have raw data that in the dist DataFrame and we want to send the domain information into the i and j sets – we take care to pass the set objects as the domain for parameter c.

m = Container()
i = Set(m, "i")
j = Set(m, "j")
dist = pd.DataFrame(
[
("seattle", "new-york", 2.5),
("seattle", "chicago", 1.7),
("seattle", "topeka", 1.8),
("san-diego", "new-york", 2.5),
("san-diego", "chicago", 1.8),
("san-diego", "topeka", 1.4),
],
columns=["from", "to", "thousand_miles"],
)
c = Parameter(m, "c", [i, j], records=dist, domain_forwarding=True)
In [1]: i.records
Out[1]:
uni_0 element_text
0 seattle
1 san-diego
In [2]: j.records
Out[2]:
uni_0 element_text
0 new-york
1 chicago
2 topeka
In [3]: c.records
Out[3]:
i_0 j_1 value
0 seattle new-york 2.5
1 seattle chicago 1.7
2 seattle topeka 1.8
3 san-diego new-york 2.5
4 san-diego chicago 1.8
5 san-diego topeka 1.4
Note
The element order in the sets i and j mirrors that in the raw data.

In this example we show that domain forwarding will also work recursively to update the entire set lineage – the domain forwarding occurs at the creation of every symbol object. The correct order of elements in set i is [z, a, b, c] because the records from j are forwarded first, and then the records from k are propagated through (back to i).

m = Container()
i = Set(m, "i")
j = Set(m, "j", i, records=["z"], domain_forwarding=True)
k = Set(m, "k", j, records=["a", "b", "c"], domain_forwarding=True)
In [1]: i.records
Out[1]:
uni_0 element_text
0 z
1 a
2 b
3 c
In [2]: j.records
Out[2]:
i_0 element_text
0 z
1 a
2 b
3 c
In [3]: k.records
Out[3]:
j_0 element_text
0 a
1 b
2 c

## Describing Data

The methods describeSets, describeParameters, describeVariables, and describeEquations allow the user to get a summary view of key data statistics. The returned DataFrame aggregates the output for a number of other methods (depending on symbol type). Column descriptions are tabulated below:

Property / Statistic Description Set Alias Parameter Variable Equation
name name of the symbol x x x x x
is_alias indicates if set is an alias (true) or not (false) x x x x x
is_singleton indicates if set is a singleton set (true) or not (false) x x x x x
domain domain labels (not object references) for the symbol x x x x x
domain_type domain types (i.e., relaxed or regular) x x x x x
dim dimension of the symbol x x x x x
num_recs number of symbol records x x x x x
cardinality full cartesian product of domain possibilities x x x x x
sparsity sparsity of symbol records w.r.t. to cardinality x x x x x
is_scalar indicates if the parameter is a scalar (true) or not (false) x
min_value minimum value x
mean_value mean value x
max_value maximum value x
where_min domain entry of record with min_value (first appearance) x
where_max domain entry of record with max_value (first appearance) x
count_eps number of EPS special values x
count_na number of NA special values x
count_undef number of UNDEF special values x
type variable or equation type x x
min_level minimum level value x x
mean_level mean level value x x
max_level maximum level value x x
where_max_abs_level domain entry of records with maximum absolute level value (first appearance) x x
count_eps_level number of EPS special values in the level x x
min_marginal minimum marginal value x x
mean_marginal mean marginal value x x
max_marginal maximum marginal value x x
where_max_abs_marginal domain entry of records with maximum absolute marginal value (first appearance) x x
count_eps_marginal number of EPS special values in the marginal x x

The methods used to generate these tables are also available directly to the user. These methods (and their arguments) are tabulated:

Method Description Argument Default
getCardinality get the full cartesian product of the domain - -
getSparsity get the sparsity of the symbol w.r.t the cardinality - -
getMaxValue get the maximum value columns value (parameter), level (variable/equation)
getMinValue get the minimum value columns value (parameter), level (variable/equation)
getMeanValue get the mean value columns value (parameter), level (variable/equation)
getMaxAbsValue get the maximum absolute value columns value (parameter), level (variable/equation)
whereMax find the domain entry of records with a maximum value (return first instance only) columns value (parameter), level (variable/equation)
whereMaxAbs find the domain entry of records with a maximum absolute value (return first instance only) columns value (parameter), level (variable/equation)
whereMin find the domain entry of records with a minimum value (return first instance only) columns value (parameter), level (variable/equation)
countNA number of NA special values in a column columns value (parameter), level (variable/equation)
countEps number of EPS special values in a column columns value (parameter), level (variable/equation)
countUndef number of UNDEF special values in a column columns value (parameter), level (variable/equation)
countPosInf number of +INF special values in a column columns value (parameter), level (variable/equation)
countNegInf number of -INF special values in a column columns value (parameter), level (variable/equation)

## Transforming Data

GAMS Transfer stores data in a "flat" format, that is, one record entry per DataFrame row. However, it is often necessary to convert this data format into a matrix format – GAMS Transfer enables users to do this with relative ease using the toDense and the toSparseCoo symbol methods. The toDense method will return a dense N-dimensional numpy array with each dimension corresponding to the GAMS symbol dimension; it is possible to output an array up to 20 dimensions (a GAMS limit). The toSparseCoo method will return the data in a sparse scipy COOrdinate format, which can then be efficiently converted into other sparse matrix formats.

Attention
Both the toDense and toSparseCoo methods do not transform the underlying DataFrame in any way, they only return the transformed data.
Note
toSparseCoo will only convert 2-dimensional data to the scipy COOrdinate format. A user interested in sparse data for an N-dimensional symbol will need to decide how to reshape the dense array in order to generate the 2D sparse format.
Attention
In order to use the toSparseCoo method the user will need to install the scipy package. Scipy is not provided with GMSPython.

Both the toDense and toSparseCoo method leverage the indexing that comes along with using categorical data types to store domain information. This means that linking symbols together (by passing symbol objects as domain information) impacts the size of the matrix. This is best demonstrated by a few examples.

Example (1D data w/o domain linking (i.e., a relaxed domain))
m = Container()
a = Parameter(m, "a", "i", records=[("a", 1), ("c", 3)])
In [1]: a.records
Out[1]:
i_0 value
0 a 1.0
1 c 3.0
In [2]: a.toDense()
Out[2]:
array([[1.],
[3.]])
In [3]: a.toSparseCoo()
Out[3]:
<2x1 sparse matrix of type '<class 'numpy.float64'>'
with 2 stored elements in COOrdinate format>

Note that the parameter a is not linked to another symbol, so when converting to a matrix, the indexing is referenced to the data structure in a.records. Defining a sparse parameter a over a set i allows us to extract information from the i domain and construct a very different dense matrix, as the following example shows:

Example (1D data w/ domain linking (i.e., a regular domain))
m = Container()
i = Set(m, "i", records=["a", "b", "c", "d"])
a = Parameter(m, "a", i, records=[("a", 1), ("c", 3)])
In [1]: i.records
Out[1]:
uni_0 element_text
0 a
1 b
2 c
3 d
In [2]: a.records
Out[2]:
i_0 value
0 a 1.0
1 c 3.0
In [3]: a.toDense()
Out[3]:
array([[1.],
[0.],
[3.],
[0.]])
In [4]: a.toSparseCoo()
Out[4]:
<4x1 sparse matrix of type '<class 'numpy.float64'>'
with 2 stored elements in COOrdinate format>
Example (2D data w/ domain linking)
m = Container()
i = Set(m, "i", records=["a", "b", "c", "d"])
a = Parameter(m, "a", [i, i], records=[("a", "a", 1), ("c", "c", 3)])
In [1]: i.records
Out[1]:
uni_0 element_text
0 a
1 b
2 c
3 d
In [2]: a.records
Out[2]:
i_0 i_1 value
0 a a 1.0
1 c c 3.0
In [3]: a.toDense()
Out[3]:
array([[1., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 3., 0.],
[0., 0., 0., 0.]])
In [4]: a.toSparseCoo()
Out[4]:
<4x4 sparse matrix of type '<class 'numpy.float64'>'
with 2 stored elements in COOrdinate format>

## The Universe Set

A Unique Element List (UEL) (aka the "universe" or "universe set") is an (i,s) pair where i is an identification number for a string s. GAMS uses UELs to efficiently store domain entries of a record by storing the UEL ID i of a domain entry instead of the actual string s. This avoids storing the same string multiple times. The concept of UELs also exists in Python/Pandas and is called a "categorical array". GAMS Transfer leverages these types in order to efficiently store strings and enable domain checking within the Python environment.

Each domain column in a DataFrame can be assigned a unique categorical type, the effect is that each symbol maintains its own list of UELs per dimension. The UEL IDs are numbered from 1 to the number of UELs stored independently for each dimension. It is possible to convert a categorical column to its ID number representation by using the categorical accessor x.records[<domain_column_label>].cat.codes; however, this type of data manipulation is not necessary within GAMS Transfer, but could be handy when debugging data.

Pandas offers the possibility to create categorical column types that are ordered or not; GAMS Transfer relies exclusively on ordered categorical data types. By using ordered categories, GAMS Transfer will order the UEL such that elements appear in the order in which they appeared in the data (which is how GAMS defines the UEL). There is a feature to GAMSTransfer that allows the user to make last minute modifications to the UEL order, should they desire.

GAMS Transfer does not actually keep track of the UEL separately from other symbols in the Container, it will be created at the last minute based on the order in which data is added to the container. However, the user can access the current state of the UEL with the .getUniverseSet() container method. For example, we set a two dimensional set:

from gamstransfer import *
m = Container()
j = Set(m, "j", ["*", "*"], records=[("i" + str(n), "j" + str(n)) for n in range(2)])
In [1]: j.records
Out[1]:
uni_0 uni_1 element_text
0 i0 j0
1 i1 j1
In [2]: m.getUniverseSet()
Out[2]: ['i0', 'j0', 'i1', 'j1']

Pandas also includes a number of methods that allow categories to be renamed, appended, etc. These methods may be useful for advanced users, but most users will probably find that modifying the original data structures and reseting the symbol records provides a simpler solution. The design of GAMS Transfer should enable to user to quickly move data back and forth, without worrying about the deeper mechanics of categorical data.

## Reordering Symbols

The order of the Container file requires the symbols to be sorted such that, for example, a Set used as domain of another symbol appears before that symbol. The Container will try to establish a valid ordering when writing the data. However, this operation can also be invoked manually by calling the method reorderSymbols.

## Rename Symbols

It is possible to rename a symbol even after it has been added to a Container. There are two methods that can be used to achieve the desired outcome:

• using the container method renameSymbol
• directly changing the name symbol property

We create a Container with two sets:

m = Container()
i = Set(m, "i", records=["seattle", "san-diego"])
j = Set(m, "j", records=["new-york", "chicago", "topeka"])
Example (changing the name of a symbol with the container method)
In [1]: m.renameSymbol("i","h")
In [2]: m.data
Out[2]:
{'h': <src.gamstransfer.Set at 0x7f9fc01fc070>,
'j': <src.gamstransfer.Set at 0x7f9f9080a220>}
Example (changing the name of a symbol with the container method)
In [1]: i.name = "h"
In [2]: m.data
Out[2]:
{'h': <src.gamstransfer.Set at 0x7f9fc01fc070>,
'j': <src.gamstransfer.Set at 0x7f9f9080a220>}
Note
Note that the renamed symbols maintain the original symbol order, this will prevent unnecessary reordering operations later in the workflow.

## Removing Symbols

Removing symbols from a container is easy when using the removeSymbols container method; this method accepts either a str or a list of str.

Attention
Once a symbol has been removed, it is possible to have hanging references as domain links in other symbols. The user will need to repair these other symbols with the proper domain links in order to avoid validity errors.

## Debugging Data

Several methods are provided as part of GAMS Transfer that enable operations that could be classified as "data debugging" rather than "describe" operations. These are: findDomainVolations, findEps, findNA, findUndef, findPosInf, and findNegInf. These methods and their arguments are tabulated below:

Method Description Argument Required Default
findDomainVolations find the index position of all records that contain a domain violation - - -
findEps find the index position of all records that contain an EPS special value None (parameter), column (variable/equation) Yes (variable/equation) -
findNA find the index position of all records that contain an NA special value None (parameter), column (variable/equation) Yes (variable/equation) -
findUndef find the index position of all records that contain an UNDEF special value None (parameter), column (variable/equation) Yes (variable/equation) -
findPosInf find the index position of all records that contain an +INF special value None (parameter), column (variable/equation) Yes (variable/equation) -
findNegInf find the index position of all records that contain an -INF special value None (parameter), column (variable/equation) Yes (variable/equation) -

## Full Example

It is possible to use everything we now know about GAMS Transfer to recreate the trnsport.gms results in GDX form. As part of this example we also introduce the write method (and generate new.gdx). We will discuss in more detail in the following section: Data Exchange with GDX.

from gamstransfer import *
# create an empty Container object
m = Container()
i = Set(m, "i", records=["seattle", "san-diego"], description="supply")
j = Set(m, "j", records=["new-york", "chicago", "topeka"], description="markets")
a = Parameter(m, "a", ["*"], description="capacity of plant i in cases")
b = Parameter(m, "b", j, description="demand at market j in cases")
d = Parameter(m, "d", [i, j], description="distance in thousands of miles")
f = Parameter(
m, "f", records=90, description="freight in dollars per case per thousand miles"
)
c = Parameter(
m, "c", [i, j], description="transport cost in thousands of dollars per case"
)
# set parameter records
cap = pd.DataFrame([("seattle", 350), ("san-diego", 600)], columns=["plant", "n_cases"])
a.setRecords(cap)
dem = pd.DataFrame(
[("new-york", 325), ("chicago", 300), ("topeka", 275)],
columns=["market", "n_cases"],
)
b.setRecords(dem)
dist = pd.DataFrame(
[
("seattle", "new-york", 2.5),
("seattle", "chicago", 1.7),
("seattle", "topeka", 1.8),
("san-diego", "new-york", 2.5),
("san-diego", "chicago", 1.8),
("san-diego", "topeka", 1.4),
],
columns=["from", "to", "thousand_miles"],
)
d.setRecords(dist)
# c(i,j) = f * d(i,j) / 1000;
cost = d.records.copy(deep=True)
cost["value"] = f.records.loc[0, "value"] * cost["value"] / 1000
c.setRecords(cost)
q = pd.DataFrame(
[
("seattle", "new-york", 50, 0),
("seattle", "chicago", 300, 0),
("seattle", "topeka", 0, 0.036),
("san-diego", "new-york", 275, 0),
("san-diego", "chicago", 0, 0.009),
("san-diego", "topeka", 275, 0),
],
columns=["to", "from", "level", "marginal"],
)
x = Variable(
m, "x", "positive", [i, j], records=q, description="shipment quantities in cases",
)
z = Variable(
m,
"z",
records=pd.DataFrame(data=[153.675], columns=["level"]),
description="total transportation costs in thousands of dollars",
)
cost = Equation(m, "cost", "eq", description="define objective function")
supply = Equation(m, "supply", "leq", i, description="observe supply limit at plant i")
demand = Equation(m, "demand", "geq", j, description="satisfy demand at market j")
# set equation records
cost.setRecords(
pd.DataFrame(data=[[0, 1, 0, 0]], columns=["level", "marginal", "lower", "upper"])
)
supplies = pd.DataFrame(
[
("seattle", 350, "eps", float("-inf"), 350),
("san-diego", 550, 0, float("-inf"), 600),
],
columns=["from", "level", "marginal", "lower", "upper"],
)
supply.setRecords(supplies)
demands = pd.DataFrame(
[
("new-york", 325, 0.225, 325),
("chicago", 300, 0.153, 300),
("topeka", 275, 0.126, 275),
],
columns=["from", "level", "marginal", "lower"],
)
demand.setRecords(demands)
m.write("new.gdx")

# Data Exchange with GDX

Up until now, we have been focused on using GAMS Transfer to create symbols in an empty Container using the symbol constructors (or their corresponding container methods). These tools will enable users to ingest data from many different formats and add them to a Container – however, it is also possible to read in symbol data directly from GDX files using the read container method. In the following sections, we will discuss this method in detail as well as the write method, which allows users to write out to new GDX files.

There are two main ways to read in GDX based data.

• Pass the file path directly to the Container constructor (will read all symbols and records)
• Pass the file path directly to the read method (default read all symbols, but can read partial files)

The first option here is provided for convenience and will, internally, call the read method. This method will read in all symbols as well as their records. This is the easiest and fastest way to get data out of a GDX file and into your Python environment. For the following examples we leverage the GDX output generated from the trnsport.gms model file.

Example (reading full data w/ Container constructor)
m = Container("trnsport.gdx")
In [1]: m.data
Out[1]:
{'i': <src.gamstransfer.Set at 0x7fdd21858d60>,
'j': <src.gamstransfer.Set at 0x7fdd21858dc0>,
'a': <src.gamstransfer.Parameter at 0x7fdd21858df0>,
'b': <src.gamstransfer.Parameter at 0x7fdd21858d90>,
'd': <src.gamstransfer.Parameter at 0x7fdd21858e80>,
'f': <src.gamstransfer.Parameter at 0x7fdd21858eb0>,
'c': <src.gamstransfer.Parameter at 0x7fdd21858ee0>,
'x': <src.gamstransfer.Variable at 0x7fdd21858f10>,
'z': <src.gamstransfer.Variable at 0x7fdd21858e50>,
'cost': <src.gamstransfer.Equation at 0x7fdd21858f70>,
'supply': <src.gamstransfer.Equation at 0x7fdd21858fa0>,
'demand': <src.gamstransfer.Equation at 0x7fdd21858fd0>}
In [2]: m.describeParameters()
Out[2]:
name is_scalar domain domain_type dim num_recs min_value mean_value max_value where_min where_max count_eps count_na count_undef cardinality sparsity
0 a False [i] regular 1 2 350.000 475.000 600.000 [seattle] [san-diego] 0 0 0 2 0.0
1 b False [j] regular 1 3 275.000 300.000 325.000 [topeka] [new-york] 0 0 0 3 0.0
4 c False [i, j] regular 2 6 0.126 0.176 0.225 [san-diego, topeka] [seattle, new-york] 0 0 0 6 0.0
2 d False [i, j] regular 2 6 1.400 1.950 2.500 [san-diego, topeka] [seattle, new-york] 0 0 0 6 0.0
3 f True [] none 0 1 90.000 90.000 90.000 None None 0 0 0 None None

A user could also read in data with the read method as shown in the following example.

m = Container()
In [1]: m.data
Out[1]:
{'i': <src.gamstransfer.Set at 0x7fdd21858d60>,
'j': <src.gamstransfer.Set at 0x7fdd21858dc0>,
'a': <src.gamstransfer.Parameter at 0x7fdd21858df0>,
'b': <src.gamstransfer.Parameter at 0x7fdd21858d90>,
'd': <src.gamstransfer.Parameter at 0x7fdd21858e80>,
'f': <src.gamstransfer.Parameter at 0x7fdd21858eb0>,
'c': <src.gamstransfer.Parameter at 0x7fdd21858ee0>,
'x': <src.gamstransfer.Variable at 0x7fdd21858f10>,
'z': <src.gamstransfer.Variable at 0x7fdd21858e50>,
'cost': <src.gamstransfer.Equation at 0x7fdd21858f70>,
'supply': <src.gamstransfer.Equation at 0x7fdd21858fa0>,
'demand': <src.gamstransfer.Equation at 0x7fdd21858fd0>}

It is also possible to read in a partial GDX file with the read method, as shown in the following example:

m = Container()
In [1]: m.data
Out[1]: {'x': <src.gamstransfer.Variable at 0x7fa728a2d9d0>}
In [2]: m.data["x"].records
Out[2]:
i_0 j_1 level marginal lower upper scale
0 seattle new-york 50.0 0.000 0.0 inf 1.0
1 seattle chicago 300.0 0.000 0.0 inf 1.0
2 seattle topeka 0.0 0.036 0.0 inf 1.0
3 san-diego new-york 275.0 0.000 0.0 inf 1.0
4 san-diego chicago 0.0 0.009 0.0 inf 1.0
5 san-diego topeka 275.0 0.000 0.0 inf 1.0

This syntax assumes that the user will always want to read in both the metadata as well as the actual data records, but it is possible to skip the reading of the records by passing the argument values=False.

m = Container()
In [1]: m.data
Out[1]: {'x': <src.gamstransfer.Variable at 0x7fa728a37220>}
In [2]: m.data["x"].summary
Out[2]:
{'name': 'x',
'type': 'positive',
'domain_objects': ['i', 'j'],
'domain_names': ['i', 'j'],
'dimension': 2,
'description': 'shipment quantities in cases',
'number_records': 6,
'domain_type': 'relaxed'}
In [3]: m.data["x"].records
Attention
The read method attempts to link the domain objects together (in order to have a "regular" domain_type) but if domain sets are not part of the read operation there is no choice but to default to a "relaxed" domain_type. This can be seen in the last example where we only read in the variable x and not the domain sets (i and j) that the variable is defined over. All the data will be available to the user, but domain checking is no longer possible. The symbol x will remain with "relaxed" domain type even if the user were to read in sets i and j in a second read call.

## Writing to GDX

A user can write data to a GDX file by simply passing a file path (as a string). The write method will then create the GDX and write all data in the Container.

Note
It is not possible to write the Container when any of its symbols are invalid. If any symbols are invalid an error will be raised and the user will need to inspect the problematic symbols (perhaps using a combination of the listSymbols(isValid=False) and isValid(verbose=True) methods).
Example
m.write("path/to/file.gdx")
Example (write a compressed GDX file)
m.write("path/to/file.gdx", compress=True)

Advanced users might want to specify an order to their UEL list (i.e., the universe set); recall that the UEL ordering follows that dictated by the data. As a convenience, it is possible to prepend the UEL list with a user specified order using the uel_priority argument.

Example (change the order of the UEL)
m = Container()
i = Set(m, "i", records=["a", "b", "c"])
m.write("foo.gdx", uel_priority=["a", "c"])

The original UEL order for this GDX file would have been ["a", "b", "c"], but at the last minute, the UEL order was changed to swap the positions of b and c. This can be verified with the gdxdump utility (using the uelTable argument):

gdxdump foo.gdx ueltable=foo
Set foo /
'a' ,
'c' ,
'b' /;
$onEmpty Set i(*) / 'a', 'c', 'b' /;$offEmpty
GAMS Development Corp.
GAMS Software GmbH

General Information and Sales
U.S. (+1) 202 342-0180
Europe: (+49) 221 949-9170