Main Classes

Container

The main object class within transfer is called Container. The Container is the vessel that allows symbols to be linked together (through their domain definitions), it enables implicit set definitions, it enables structural manipulations of the data (matrix generation), and it allows the user to perform different read/write operations.

Constructor

Constructor Arguments
Argument Type Description Required Default
load_from str or PathLike object, GMD Object Handle, GamsDatabase object, Container Points to the source of the data being read into the Container No None
system_directory str Absolute path to GAMS system directory No Attempts to find the GAMS installation by creating a GamsWorkspace object and loading the system_directory attribute.

Creating a Container is a simple matter of initializing an object. For example:

import gams.transfer as gt
m = gt.Container()

This new Container object, here called m, contains a number of convenient properties and methods that allow the user to interact with the symbols that are in the Container. Some of these methods are used to filter out different types of symbols, other methods are used to numerically characterize the data within each symbol.

Properties

Property Description Type Special Setter Behavior
data main dictionary that is used to store all symbol data (case preserving) CasePreservingDict -
modified Flag that identifies if the Container has been modified in some way. Container.modifed=False will reset this flag for all symbols in the container as well as the container itself. bool -
summary returns a brief summary of the Container dict -

Symbols are organized in the Container under the data Container attribute. The dot notation (m.data) is used to access the underlying dictionary. Symbols in this dictionary can then be retrieved with the standard bracket notation (m.data[<symbol_name>]). The Container is also subscriptable (i.e., m["i"] will return the i Set object just as if the user called m.data["i"]). The behavior of the data dictionary is has been customized to be case-insensitive (which mimics the behavior of GAMS) – m["i"] and m["I"] will return the same object.

In [1]: m.data
Out[1]:
{'i': <Set `i` (0x7f95b8d63e80)>,
'j': <Set `j` (0x7f95b8d63a60)>,
'a': <Parameter `a` (0x7f95b8d63ee0)>,
'b': <Parameter `b` (0x7f95b8d63d00)>,
'd': <Parameter `d` (0x7f95b8da86a0)>,
'f': <Parameter `f` (0x7f95b8da8670)>,
'c': <Parameter `c` (0x7f95b8da83d0)>,
'x': <Positive Variable `x` (0x7f95b8da83a0)>,
'z': <Free Variable `z` (0x7f95b8da8400)>,
'cost': <Eq Equation `cost` (0x7f95b8da82b0)>,
'supply': <Leq Equation `supply` (0x7f95b8da8280)>,
'demand': <Geq Equation `demand` (0x7f95b8da8580)>}

Symbol existence in the Container can be tested with an overloaded Python in operator. The following (case-insensitive) syntax is possible:

In [1]: 'i' in m
Out[1]: True
In [2]: 'I' in m
Out[2]: True
In [3]: i in m
Out[3]: True
Note
The final example assumes the existence of a separate symbol object called i.

The Container can also iterate through the symbols (will return a tuple of (symbol_name, symbol_object) using a for-loop as in the following example. The list and dict methods can be useful in creating lists and dictionaries of all symbols in the Container.

In [1]: m = gt.Container("out.gdx")
In [2]: for name, obj in m:
...: print(name, obj)
...:
i <Set `i` (0x7f9038533fa0)>
j <Set `j` (0x7f9038533fd0)>
a <Parameter `a` (0x7f9038533340)>
b <Parameter `b` (0x7f9038533f10)>
d <Parameter `d` (0x7f9038533f70)>
f <Parameter `f` (0x7f9038533160)>
c <Parameter `c` (0x7f90583b4bb0)>
x <Positive Variable `x` (0x7f90583b4070)>
z <Free Variable `z` (0x7f90583b5570)>
cost <Eq Equation `cost` (0x7f90583b44f0)>
supply <Leq Equation `supply` (0x7f9008c33df0)>
demand <Geq Equation `demand` (0x7f9008c33b80)>
In [3]: list(m)
Out[3]:
[('i', <Set `i` (0x7fced87d3790)>),
('j', <Set `j` (0x7fced882e530)>),
('a', <Parameter `a` (0x7fced882e9b0)>),
('b', <Parameter `b` (0x7fced882ebf0)>),
('d', <Parameter `d` (0x7fced882e800)>),
('f', <Parameter `f` (0x7fced882e9e0)>),
('c', <Parameter `c` (0x7fced882e320)>),
('x', <Positive Variable `x` (0x7fced882e650)>),
('z', <Free Variable `z` (0x7fced882e410)>),
('cost', <Eq Equation `cost` (0x7fcec8918640)>),
('supply', <Leq Equation `supply` (0x7fcec8918610)>),
('demand', <Geq Equation `demand` (0x7fcec89183a0)>)]
In [5]: dict(m)
Out[5]:
{'i': <Set `i` (0x7fced87d3790)>,
'j': <Set `j` (0x7fced882e530)>,
'a': <Parameter `a` (0x7fced882e9b0)>,
'b': <Parameter `b` (0x7fced882ebf0)>,
'd': <Parameter `d` (0x7fced882e800)>,
'f': <Parameter `f` (0x7fced882e9e0)>,
'c': <Parameter `c` (0x7fced882e320)>,
'x': <Positive Variable `x` (0x7fced882e650)>,
'z': <Free Variable `z` (0x7fced882e410)>,
'cost': <Eq Equation `cost` (0x7fcec8918640)>,
'supply': <Leq Equation `supply` (0x7fcec8918610)>,
'demand': <Geq Equation `demand` (0x7fcec89183a0)>}
Note
The len (length) function can be used to quickly find out how many symbols exist in the Container (len(m)).

Methods

Method Description Arguments/Defaults Returns
addAlias Container method to add an Alias name (str)
alias_with (Set, Alias)
Alias object
addEquation Container method to add an Equation name (str)
type (str)
domain=[] (str, list)
records=None (pandas.DataFrame, numpy.ndarry, None)
domain_forwarding=False (bool)
description="" (str)
uels_on_axes=False (bool)
Equation object
addParameter Container method to add a Parameter name (str)
domain=None (str, list, None)
records=None (pandas.DataFrame, numpy.ndarry, None)
domain_forwarding=False (bool)
description="" (str)
uels_on_axes=False (bool)
Parameter object
addSet Container method to add a Set name (str)
domain=None (str, list, None)
is_singleton=False (bool)
records=None (pandas.DataFrame, numpy.ndarry, None)
domain_forwarding=False (bool)
description="" (str)
uels_on_axes=False (bool)
Set object
addUniverseAlias Container method to add a UniverseAlias name (str) UniverseAlias object
addVariable Container method to add an Variable name (str)
type="free" (str)
domain=[] (str, list)
records=None (pandas.DataFrame, numpy.ndarry, None)
domain_forwarding=False (bool)
description="" (str)
uels_on_axes=False (bool)
Variable object
capitalizeUELs will capitalize all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
casefoldUELs will casefold all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
countDomainViolations get the count of how many records contain at least one domain violation for symbols (if symbols=None assume all symbols) symbols=None (str, list, None) dict
countDuplicateRecords returns the count of how many duplicate records exist for symbols (if symbols=None assume all symbols) symbols=None (str, list, None) dict
describeAliases create a summary table with descriptive statistics for Aliases symbols=None (None, str, list) - if None, assumes all aliases pandas.DataFrame
describEquations create a summary table with descriptive statistics for Equations symbols=None (None, str, list) - if None, assumes all equations pandas.DataFrame
describeParameters create a summary table with descriptive statistics for Parameters symbols=None (None, str, list) - if None, assumes all parameters pandas.DataFrame
describeSets create a summary table with descriptive statistics for Sets symbols=None (None, str, list) - if None, assumes all sets pandas.DataFrame
describeVariables create a summary table with descriptive statistics for Variables symbols=None (None, str, list) - if None, assumes all variables pandas.DataFrame
dropDomainViolations drop records that have domain violations for symbols (if symbols=None assume all symbols) symbols=None (str, list, None) None
dropDuplicateRecords drop records with duplicate domains from symbols in the Container – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) symbols=None (str, list, None) - if None, assumes all symbols
keep="first" (str, False)
None
getAliases return all alias objects in the container (is_valid=None), return all valid alias objects (is_valid=True), return all invalid alias objects (is_valid=False) in the container is_valid=None (bool, None) list
getDomainViolations gets domain violations that exist in the data for symbols (if symbols=None assume all symbols); returns a list of DomainViolation objects (or None if no violations) symbols=None (str, list, None) list or None
getEquations return all equation objects (is_valid=None), return all valid equation objects (is_valid=True), return all invalid equation objects (is_valid=False) in the container is_valid=None (bool, None)
types=None (list of equation types) - if None, assumes all types
list
getParameters return all parameter objects in the container (is_valid=None), return all valid parameter objects (is_valid=True), return all invalid parameter objects (is_valid=False) in the container is_valid=None (bool, None) list
getSets return all set objects in the container (is_valid=None), return all valid set objects (is_valid=True), return all invalid set objects (is_valid=False) in the container is_valid=None (bool, None) list
getSymbols returns a list of object references for symbols symbols (str, list) list
getUELs gets UELs from all symbols. If symbols=None and ignore_unused=False, return the full universe set. If symbols=None and ignore_unused=True, return a universe set that contains UELs that only appear in data. symbols=None (str, list, None)
ignore_unused=False (bool)
list
getVariables return all variable objects (is_valid=None), return all valid variable objects (is_valid=True), return all invalid variable objects (is_valid=False) in the container is_valid=None (bool, None)
types=None (list of variable types) - if None, assumes all types
list
hasDomainViolations returns True if there are domain violations in the records for symbols (if symbols=None assume all symbols), returns False if not. symbols=None (str, list, None) bool
hasDuplicateRecords returns True if there are duplicate records for symbols (if symbols=None assume all symbols), False if not. symbols=None (str, list, None) bool
isValid True if symbols in the Container are valid (if symbols=None assume all symbols) symbols=None (str, list, None) bool
ljustUELs will left justify all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
symbols=None (int, list, or None) - if None, assumes all symbols
self
listAliases list all aliases (is_valid=None), list all valid aliases (is_valid=True), list all invalid aliases (is_valid=False) in the container is_valid=None (bool, None) list
listEquations list all equations (is_valid=None), list all valid equations (is_valid=True), list all invalid equations (is_valid=False) in the container is_valid=None (bool, None)
types=None (list of equation types) - if None, assumes all types
list
listParameters list all parameters (is_valid=None), list all valid parameters (is_valid=True), list all invalid parameters (is_valid=False) in the container is_valid=None (bool, None) list
listSets list all sets (is_valid=None), list all valid sets (is_valid=True), list all invalid sets (is_valid=False) in the container is_valid=None (bool, None) list
listSymbols list all symbols (is_valid=None), list all valid symbols (is_valid=True), list all invalid symbols (is_valid=False) in the container is_valid=None (bool, None) list
listVariables list all variables (is_valid=None), list all valid variables (is_valid=True), list all invalid variables (is_valid=False) in the container is_valid=None (bool, None)
types=None (list of variable types) - if None, assumes all types
list
lowerUELs will lowercase all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
lstripUELs will left strip whitespace from all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
read main method to read load_from, can be provided with a list of symbols to read in subsets, records controls if symbol records are loaded or just metadata, mode controls if data is first read into categorical data structures or object type structures, encoding specifies the original file encoding and to properly decode special characters load_from (str or PathLike object, GMD Object Handle, GamsDatabase object, Container)
symbols="all" (str, list)
records=True (bool)
mode="category" (category or string)
encoding=None (str or None)
None
removeSymbols symbols to remove from the Container, also sets the symbols container to None. If symbols=None, will remove all symbols. symbols=None (str, list, None) None
removeUELs removes UELs from all symbols in all dimensions. If uels is None only unused UELs will be removed. If symbols is None UELs will be removed from all symbols. uels (str, list, None)
symbols=None (str, list, None)
None
renameSymbol rename a symbol in the Container old_name (str), new_name (str) None
renameUELs renames UELs (case-sensitive) that appear in symbols (for all dimensions). If symbols=None, rename UELs in all symbols. If allow_merge=True, the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** uels (dict)
symbols=None (str, list, None)
allow_merge=False (bool)
None
reorderSymbols reorder symbols in order to avoid domain violations - None
rjustUELs will right justify all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
symbols=None (int, list, or None) - if None, assumes all symbols
self
rstripUELs will right strip whitespace from all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
stripUELs will strip whitespace from all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
titleUELs will title (capitalize all individual words) in all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
upperUELs will uppercase all UELs in the Container or a subset of specified symbols, can be chained with other *UELs string operations symbols=None (str, list, None) - if None, assumes all symbols self
write main bulk write method to a write_to target write_to (str, PathLike object, GamsDatabase object, GMD Object)
symbols=None (None, str, list) - if None, assumes all symbols
compress=False (bool)
uel_priority=None (str, list)
merge_symbols=None (None, str, list)
mode=None (None, string, category) - if None, assumes string mode writing
eps_to_zero=True (bool) - default behavior will convert all -0.0 (EPS) values into 0.0 (drop the sign bit)
None

Set

There are two different ways to create a GAMS set and add it to a Container.

  1. Use Set constructor
  2. Use the Container method addSet (which internally calls the Set constructor)

Constructor

Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
description str Description of symbol No ""
domain list, str, or Set/Alias List of domains given either as string ('*' for universe set) or as reference to a Set/Alias object No ["*"]
domain_forwarding bool or list Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. No False
is_singleton bool Indicates if set is a singleton set (True) or not (False) No False
name str Name of symbol Yes -
records many Symbol records No None
uels_on_axes bool Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object No False
Note
Set records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, and has the same is_singleton and domain_forwarding state. The symbol description will only be updated if new text is provided.

Properties

Property Description Type Special Setter Behavior
container reference to the Container that the symbol belongs to Container -
description description of symbol str -
dimension dimension of symbol int setting is a shorthand notation to create ["*"] * n domains in symbol
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list, str, or Set/Alias -
domain_forwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. bool or list no effect after records have been set
domain_labels column headings for the records DataFrame list of str will add a _<dimension> tag to user supplied column names (if not unique)
domain_names string version of domain names list of str -
domain_type none, relaxed or regular depending on state of domain links str -
is_singleton bool if symbol is a singleton set bool -
modified Flag that identifies if the Set has been modified bool -
name name of symbol str sets the GAMS name of the symbol
number_records number of symbol records (i.e., returns len(self.records) if not None) int -
records the main symbol records pandas.DataFrame responsive to domain_forwarding state
summary output a dict of only the metadata dict -

Methods

Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol dimensions. If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
None
capitalizeUELs will capitalize all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
casefoldUELs will casefold all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
countDomainViolations returns the count of how many records contain at least one domain violation - int
countDuplicateRecords returns the count of how many (case insensitive) duplicate records exist - int
dropDomainViolations drop records from the symbol that contain a domain violation - None
dropDuplicateRecords drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) keep="first" (str, False) None
equals Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_element_text=True then check that all set elements have the same descriptive element text, otherwise skip. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. If verbose=True will return an exception from the asserter describing the nature of the difference. check_uels=True (bool)
check_element_text=True (bool)
check_meta_data=True (bool)
verbose=False (bool)
bool
findDomainViolations get a view of the records DataFrame that contain any domain violations - pandas.DataFrame
findDuplicateRecords get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) keep="first" (str, False) pandas.DataFrame
generateRecords convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1]. If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. density=1.0 (float, list)
seed=None (int, None)
None
getDomainViolations returns a list of DomainViolation objects if any (None otherwise) - list or None
getSparsity get the sparsity of the symbol w.r.t the cardinality - float
getUELs gets UELs from symbol dimensions. If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int; must specify a single dimension if passing codes. Returns only UELs in the data if ignore_unused=True, otherwise return all UELs. dimensions=None (int, list, None)
codes=None (int, list, None)
ignore_unused=False (bool)
list
hasDomainViolations returns True if there are domain violations in the records, returns False if not. - bool
hasDuplicateRecords returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. - bool
isValid checks if the symbol is in a valid format, throw exceptions if verbose=True, recheck a symbol if force=True verbose=False
force=True
bool
ljustUELs will left justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
lowerUELs will lowercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
lstripUELs will left strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
pivot Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1]. If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value index=None (str, list, None)
columns=None (str, list, None)
fill_value=None (int, float, str)
pd.DataFrame
renameUELs renames UELs (case-sensitive) that appear in the symbol dimensions. If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True, the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** uels (str, list, dict)
dimensions (int, list, None)
allow_merge=False (bool)
None
reorderUELs reorders the UELs in the symbol dimensions. If uels is None, reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. uels (str, list, dict, None)
dimensions (int, list, None)
None
removeUELs removes UELs that appear in the symbol dimensions, If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. uels=None (str, list, None)
dimensions=None (int, list, None)
bool
rjustUELs will right justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
rstripUELs will right strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
setRecords main convenience method to set standard pandas.DataFrame formatted records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). records (many types) None
setUELs set the UELs for symbol dimensions. If dimensions is None then set UELs for all dimensions. If rename=True, then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
rename=False (bool)
None
stripUELs will strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
titleUELs will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
toList convenience method to return symbol records as a Python list include_element_text=False (bool) list
upperUELs will uppercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self

Adding Set Records

Three possibilities exist to assign symbol records to a set (roughly ordered in complexity):

  1. Setting the argument records in the set constructor/container method (internally calls setRecords) - creates a data copy
  2. Using the symbol method setRecords - creates a data copy
  3. Setting the property records directly - does not create a data copy

If the data is in a convenient format, a user may want to pass the records directly within the set constructor. This is an optional keyword argument and internally the set constructor will simply call the setRecords method. The symbol method setRecords is a convenience method that transforms the given data into an approved Pandas DataFrame format (see Standard Data Formats). Many native Python data types can be easily transformed into DataFrames, so the setRecords method for Set objects will accept a number of different types for input. The setRecords method is called internally on any data structure that is passed through the records argument. We show a few examples of ways to create differently structured sets:

Example #1 - Create a 1D set from a list
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=["seattle", "san-diego"])
# NOTE: the above syntax is equivalent to -
# i = gt.Set(m, "i")
# i.setRecords(["seattle", "san-diego"])
# NOTE: the above syntax is also equivalent to -
# m.addSet("i", records=["seattle", "san-diego"])
# NOTE: the above syntax is also equivalent to -
# i = m.addSet("i")
# i.setRecords(["seattle", "san-diego"])
# NOTE: the above syntax is also equivalent to -
# m.addSet("i")
# m["i"].setRecords(["seattle", "san-diego"])
In [1]: i.records
Out[1]:
uni element_text
0 seattle
1 san-diego
Example #2 - Create a 1D set from a tuple
import gams.transfer as gt
m = gt.Container()
j = gt.Set(m, "j", records=("seattle", "san-diego"))
# NOTE: the above syntax is equivalent to -
# j = gt.Set(m, "j")
# j.setRecords(("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# m.addSet("j", records=("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# j = m.addSet("j")
# j.setRecords(("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# m.addSet("j")
# m["j"].setRecords(("seattle", "san-diego"))
In [1]: j.records
Out[1]:
uni element_text
0 seattle
1 san-diego
Example #3 - Create a 2D set from a list of tuples
import gams.transfer as gt
m = gt.Container()
k = gt.Set(m, "k", ["*", "*"], records=[("seattle", "san-diego")])
# NOTE: the above syntax is equivalent to -
# k = gt.Set(m, "k", ["*", "*"])
# k.setRecords([("seattle", "san-diego")])
# NOTE: the above syntax is also equivalent to -
# m.addSet("k", ["*","*"], records=[("seattle", "san-diego")])
# NOTE: the above syntax is also equivalent to -
# k = m.addSet("k", ["*","*"])
# k.setRecords([("seattle", "san-diego")])
# NOTE: the above syntax is also equivalent to -
# m.addSet("k", ["*","*"])
# m["k"].setRecords([("seattle", "san-diego")])
In [1]: k.records
Out[1]:
uni_0 uni_1 element_text
0 seattle san-diego
Example #4 - Create a 1D set from a DataFrame slice + .unique()
import gams.transfer as gt
m = gt.Container()
# note that the raw data is convenient to hold in a DataFrame
dist = pd.DataFrame(
[
("seattle", "new-york", 2.5),
("seattle", "chicago", 1.7),
("seattle", "topeka", 1.8),
("san-diego", "new-york", 2.5),
("san-diego", "chicago", 1.8),
("san-diego", "topeka", 1.4),
],
columns=["from", "to", "thousand_miles"],
)
l = gt.Set(m, "l", records=dist["from"].unique())
# NOTE: the above syntax is equivalent to -
# l = gt.Set(m, "l")
# l.setRecords(dist["from"].unique())
# NOTE: the above syntax is also equivalent to -
# m.addSet("l", records=dist["from"].unique())
# NOTE: the above syntax is also equivalent to -
# l = m.addSet("l")
# l.setRecords(dist["from"].unique())
# NOTE: the above syntax is also equivalent to -
# m.addSet("l")
# m["l"].setRecords(dist["from"].unique())
In [1]: l.records
Out[1]:
uni element_text
0 seattle
1 san-diego
Note
The .unique() method preserves the order of appearance, unlike set().

Set element text is very handy when labeling specific set elements within a set. A user can add a set element text directly with a set element. Note that it is not required to label all set elements, as can be seen in the following example.

Example #5 - Add set element text
import gams.transfer as gt
m = gt.Container()
i = gt.Set(
m,
"i",
records=[
("seattle", "home of sub pop records"),
("san-diego",),
("washington_dc", "former gams hq"),
],
)
# NOTE: the above syntax is equivalent to -
#
# i = gt.Set(m, "i")
# i_recs = [
# ("seattle", "home of sub pop records"),
# ("san-diego",),
# ("washington_dc", "former gams hq"),
# ]
#
# i.setRecords(i_recs)
# NOTE: the above syntax is also equivalent to -
# m.addSet("i", records=i_recs)
# NOTE: the above syntax is also equivalent to -
# i = m.addSet("i")
# i.setRecords(i_recs)
# NOTE: the above syntax is also equivalent to -
# m.addSet("i")
# m["i"].setRecords(i_recs)
In [1]: i.records
Out[1]:
uni element_text
0 seattle home of sub pop records
1 san-diego
2 washington_dc former gams hq
Example #6 - Create a 1D set from a Pandas Series
import gams.transfer as gt
import pandas as pd
s = pd.Series(index=["a", "b"])
m = gt.Container()
i = gt.Set(m, "i", records=s, uels_on_axes=True)
# NOTE: We pass the uels_on_axes=True argument in order to tell setRecords to reshape the data
In [1]: i.records
Out[1]:
uni element_text
0 a
1 b
# Now let's add in some element_text
s = pd.Series(index=["a", "b"], data=["node_1", "node_2"])
m = gt.Container()
i = gt.Set(m, "i", records=s, uels_on_axes=True)
In [2]: i.records
Out[2]:
uni element_text
0 a node_1
1 b node_2
# If uels_on_axes=False, setRecords will attempt to create a categorical data structure from s.values -- which cannot be done successfully if there are NaN values in s.values.
s = pd.Series(index=["a", "b"])
m = gt.Container()
i = gt.Set(m, "i", records=s)
ValueError: Categorical categories cannot be null
Example #7 - Create a 2D set from a Pandas Series
s = pd.Series(
index=pd.MultiIndex.from_tuples([("a", "b"), ("c", "d")]),
data=["link_1", "link_2"],
)
m = gt.Container()
i = gt.Set(m, "i", ["*", "*"], records=s, uels_on_axes=True)
# Here is the raw pandas.Series object
In [1]: s
Out[1]:
a b link_1
c d link_2
dtype: object
# When setting records the pandas.Series object will get converted into a pandas.DataFrame
In [2]: i.records
Out[2]:
uni_0 uni_1 element_text
0 a b link_1
1 c d link_2
Attention
The order of the set element could be surprising depending on how the pandas MultiIndex is created. Users should take care that the order of the elements returned from the getUELs method is correct.
Example #8 - Create a 2D set from a Pandas DataFrame with uels_on_axes=True
import gams.transfer as gt
import pandas as pd
# Create some data first
dim1 = ["a", "b"]
dim2 = ["e", "f"]
dim3 = ["z", "x", "y"]
df = pd.DataFrame(
index=dim1, columns=pd.MultiIndex.from_product([dim2, dim3]), dtype=bool
)
# now remove all set elements that contain the "z" element in the 3rd dimension
df.loc[:, (slice(None), "z")] = False
# now remove all set elements that contain the "a" element in the 1st dimension
df.loc["a", :] = False
In [1]: df
Out[1]:
e f
z x y z x y
a False False False False False False
b False True True False True True
# create the Container and add records
m = gt.Container()
i = gt.Set(m, "i", ["*"] * 3, records=df, uels_on_axes=True)
In [2]: i.records
Out[2]:
uni_0 uni_1 uni_2 element_text
0 b e x
1 b e y
2 b f x
3 b f y
Note
It not possible to set element_text when uels_on_axes=True

Directly Set Records

The primary advantage of the setRecords method is that transfer will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.

Example #1 - Directly set records (1D set)
import gams.transfer as gt
import pandas as pd
m = gt.Container()
i = gt.Set(m, "i", description="supply")
# create a standard format dataframe
df_i = pd.DataFrame(
data=[("seattle", ""), ("san-diego", "")], columns=["uni", "element_text"]
)
# need to create categorical column type, referencing elements already in df_i
df_i["uni"] = df_i["uni"].astype(
pd.CategoricalDtype(categories=df_i["uni"].unique(), ordered=True)
)
# set the records directly
i.records = df_i
In [1]: i.isValid()
Out[1]: True

Stepping through this example we take the following steps:

  1. Create an empty Container
  2. Create a GAMS set i in the Container, but do not set the records
  3. Create a Pandas DataFrame (manually, in this example) taking care to follow the standard format
  4. The DataFrame has the right shape and column labels so we can proceed to set the records.
  5. We need to cast the uni column as a categorical data type, so we create a custom ordered categorty type using pandas.CategoricalDtype
  6. Finally, we set the records directly by passing a reference to df_i into the symbol records attribute. The setter function of .records checks that a DataFrame is being set, but does not check validity. Thus, as a final step we call the .isValid() method to verify that the symbol is valid.
Attention
Users can debug their DataFrames by running <symbol_name>.isValid(verbose=True) to get feedback about their data.
Example #2 - Directly set records (1D subset)
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=["seattle", "san-diego"], description="supply")
j = gt.Set(m, "j", i, description="supply")
# create a standard format dataframe
df_j = pd.DataFrame(data=[("seattle", "")], columns=["i", "element_text"])
# create the categorical column type
df_j["i"] = df_j["i"].astype(i.records["uni"].dtype)
# set the records
j.records = df_j
In [1]: j.isValid()
Out[1]: True

This example is more subtle in that we want to create a set j that is a subset of i. We create the set i using the setRecords method but then set the records directly for j. There are two important details to note: 1) the column labels in df_j now reflect the standard format for a symbol with a domain set (as opposed to the universe) and 2) we create the categorical dtype by referencing the parent set (i) for the categories (instead of referencing itself).

Generate Set Records

Generating the initial pandas.DataFrame object could be difficult for Set symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords convenience function. Internally, generateRecords computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords will only work on symbols where <symbol>.domain_type == "regular").

Example #1 - Create a large (dense) 4D set
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Set(m, "a", [i, j, k, l])
# generate the records
a.generateRecords()
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l element_text
0 i0 j0 k0 l0
1 i0 j0 k0 l1
2 i0 j0 k0 l2
3 i0 j0 k0 l3
4 i0 j0 k0 l4
... ... ... ... ... ...
6249995 i49 j49 k49 l45
6249996 i49 j49 k49 l46
6249997 i49 j49 k49 l47
6249998 i49 j49 k49 l48
6249999 i49 j49 k49 l49
[6250000 rows x 5 columns]

It is also possible to generate a sparse set (randomly selected rows are removed from the dense dataframe) with the density argument to generateRecords.

Example #2 - Create a large (sparse) 4D set
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Set(m, "a", [i, j, k, l])
# generate the records
a.generateRecords(density=0.05)
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l element_text
0 i0 j0 k1 l4
1 i0 j0 k1 l13
2 i0 j0 k1 l19
3 i0 j0 k1 l23
4 i0 j0 k2 l1
... ... ... ... ... ...
312495 i49 j49 k48 l27
312496 i49 j49 k48 l30
312497 i49 j49 k49 l7
312498 i49 j49 k49 l32
312499 i49 j49 k49 l42
[312500 rows x 5 columns]
Example #3 - Create a large 4D set w/ only 1 sparse dimension
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Set(m, "a", [i, j, k, l])
# generate the records
a.generateRecords(density=[1, 0.05, 1, 1])
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l element_text
0 i0 j22 k0 l0
1 i0 j22 k0 l1
2 i0 j22 k0 l2
3 i0 j22 k0 l3
4 i0 j22 k0 l4
... ... ... ... ... ...
249995 i49 j36 k49 l45
249996 i49 j36 k49 l46
249997 i49 j36 k49 l47
249998 i49 j36 k49 l48
249999 i49 j36 k49 l49
[250000 rows x 5 columns]

Parameter

There are two different ways to create a GAMS parameter and add it to a Container.

  1. Use Parameter constructor
  2. Use the Container method addParameter (which internally calls the Parameter constructor)

Constructor

Constructor Arguments
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
description str Description of symbol No ""
domain list, str, or Set/Alias List of domains given either as string ('*' for universe set) or as reference to a Set/Alias object, an empty domain list will create a scalar parameter No []
domain_forwarding bool or list Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. No False
name str Name of symbol Yes -
records many Symbol records No None
uels_on_axes bool Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object No False
Note
Parameter records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, and has the same domain_forwarding state. The symbol description will only be updated if new text is provided.

Properties

Property Description Type Special Setter Behavior
container reference to the Container that the symbol belongs to Container -
description description of symbol str -
dimension dimension of symbol int setting is a shorthand notation to create ["*"] * n domains in symbol
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list, str, or Set/Alias -
domain_forwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. bool or list no effect after records have been set
domain_labels column headings for the records DataFrame list of str will add a _<dimension> tag to user supplied column names (if not unique)
domain_names string version of domain names list of str -
domain_type none, relaxed or regular depending on state of domain links str -
is_scalar True if the len(self.domain) = 0 bool -
modified Flag that identifies if the Parameter has been modified bool -
name name of symbol str sets the GAMS name of the symbol
number_records number of symbol records (i.e., returns len(self.records) if not None) int -
records the main symbol records pandas.DataFrame responsive to domain_forwarding state
shape a tuple describing the array dimensions if records were converted with .toDense() tuple -
summary output a dict of only the metadata dict -

Methods

Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol dimensions. If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
None
capitalizeUELs will capitalize all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
casefoldUELs will casefold all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
countDomainViolations returns the count of how many records contain at least one domain violation - int
countDuplicateRecords returns the count of how many (case insensitive) duplicate records exist - int
countEps total number of SpecialValues.EPS in value column - int or None
countNA total number of SpecialValues.NA in value column - int or None
countNegInf total number of SpecialValues.NEGINF in value column - int or None
countPosInf total number of SpecialValues.POSINF in value column - int or None
countUndef total number of SpecialValues.UNDEF in value column - int or None
dropDefaults an alias to dropZeros - None
dropDomainViolations drop records from the symbol that contain a domain violation - None
dropDuplicateRecords drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) keep="first" (str, False) None
dropEps drop records from the symbol that are GAMS EPS (zero 0.0 records will be retained) - None
dropMissing drop records from the symbol that are NaN (includes both NA and Undef special values) - None
dropNA drop records from the symbol that are GAMS NA - None
dropUndef drop records from the symbol that are GAMS Undef - None
dropZeros drop records from the symbol that are zero (GAMS EPS (-0.0) will not be dropped) - None
equals Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. rtol (relative tolerance) and atol (absolute tolerance) set equality tolerances. If verbose=True will return an exception from the asserter describing the nature of the difference. check_uels=True (bool)
check_meta_data=True (bool)
rtol=0.0 (float, None)
atol=0.0 (float, None)
verbose=False (bool)
bool
findDomainViolations get a view of the records DataFrame that contain any domain violations - pandas.DataFrame
findDuplicateRecords get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) keep="first" (str, False) pandas.DataFrame
findEps find positions of SpecialValues.EPS in value column - pandas.DataFrame or None
findNA find positions of SpecialValues.NA in value column - pandas.DataFrame or None
findNegInf find positions of SpecialValues.NEGINF in value column - pandas.DataFrame or None
findPosInf find positions of SpecialValues.POSINF in value column - pandas.DataFrame or None
findUndef find positions of SpecialValues.Undef in value column - pandas.DataFrame or None
generateRecords convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1]. If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. density=1.0 (float, list)
func=numpy.random.uniform(0,1) (callable)
seed=None (int, None)
None
getMaxAbsValue get the maximum absolute value in value column - float or None
getMaxValue get the maximum value in value column - float or None
getMeanValue get the mean value in value column - float or None
getMinValue get the minimum value in value column - float or None
getSparsity get the sparsity of the symbol w.r.t the cardinality - float
getUELs gets UELs from symbol dimensions. If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int; must specify a single dimension if passing codes. Returns only UELs in the data if ignore_unused=True, otherwise return all UELs. dimensions=None (int, list, None)
codes=None (int, list, None)
ignore_unused=False (bool)
list
hasDomainViolations returns True if there are domain violations in the records, returns False if not. - bool
hasDuplicateRecords returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. - bool
isValid checks if the symbol is in a valid format, throw exceptions if verbose=True, recheck a symbol if force=True verbose=False
force=True
bool
ljustUELs will left justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
lowerUELs will lowercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
lstripUELs will left strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
pivot Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1]. If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value index=None (str, list, None)
columns=None (str, list, None)
fill_value=None (int, float, str)
pd.DataFrame
removeUELs removes UELs that appear in the symbol dimensions, If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. uels=None (str, list, None)
dimensions=None (int, list, None)
bool
renameUELs renames UELs (case-sensitive) that appear in the symbol dimensions. If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True, the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** uels (str, list, dict)
dimensions (int, list, None)
allow_merge=False (bool)
None
reorderUELs reorders the UELs in the symbol dimensions. If uels is None, reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. uels (str, list, dict, None)
dimensions (int, list, None)
None
rjustUELs will right justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
rstripUELs will right strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
setRecords main convenience method to set standard pandas.DataFrame records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). records (many types). None
setUELs set the UELs for symbol dimensions. If dimensions is None then set UELs for all dimensions. If rename=True, then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
rename=False (bool)
None
stripUELs will strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
titleUELs will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
toDense convert symbol to a dense numpy.array format - numpy.array or None
toDict convenience method to return symbol records as a Python dictionary, orient can take values natural or columns and will control the shape of the dict. Must use orient="columns" if attempting to set symbol records with setRecords orient="natural" dict
toList convenience method to return symbol records as a Python list - list
toSparseCoo convert symbol to a sparse COOrdinate numpy.array format - sparse matrix format or None
toValue convenience method to return symbol records as a Python float. Only possible with scalar symbols. - float
upperUELs will uppercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
whereMax find the domain entry of records with a maximum value (return first instance only) - list of str or None
whereMaxAbs find the domain entry of records with a maximum absolute value (return first instance only) - list of str or None
whereMin find the domain entry of records with a minimum value (return first instance only) - list of str or None

Adding Parameter Records

Three possibilities exist to assign symbol records to a parameter (roughly ordered in complexity):

  1. Setting the argument records in the set constructor/container method (internally calls setRecords) - creates a data copy
  2. Using the symbol method setRecords - creates a data copy
  3. Setting the property records directly - does not create a data copy

If the data is in a convenient format, a user may want to pass the records directly within the parameter constructor. This is an optional keyword argument and internally the parameter constructor will simply call the setRecords method. The symbol method setRecords is a convenience method that transforms the given data into an approved Pandas DataFrame format (see Standard Data Formats). Many native Python data types can be easily transformed into DataFrames, so the setRecords method for Set objects will accept a number of different types for input. The setRecords method is called internally on any data structure that is passed through the records argument. We show a few examples of ways to create differently structured parameters:

Example #1 - Create a GAMS scalar
import gams.transfer as gt
m = gt.Container()
pi = gt.Parameter(m, "pi", records=3.14159)
# NOTE: the above syntax is equivalent to -
# pi = gt.Parameter(m, "pi")
# pi.setRecords(3.14159)
# NOTE: the above syntax is also equivalent to -
# m.addParameter("pi", records=3.14159)
# NOTE: the above syntax is also equivalent to -
# pi = m.addParameter("pi")
# pi.setRecords(3.14159)
# NOTE: the above syntax is also equivalent to -
# m.addParameter("pi")
# m["pi"].setRecords(3.14159)
In [14]: pi.records
Out[14]:
value
0 3.14159
Note
transfer will still convert scalar values to a standard format (i.e., a Pandas DataFrame with a single row and column).
Example #2 - Create a 1D parameter (defined over *) from a list of tuples
import gams.transfer as gt
m = gt.Container()
i = gt.Parameter(m, "i", ["*"], records=[("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is equivalent to -
# i = gt.Parameter(m, "i")
# i.setRecords([("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is also equivalent to -
# m.addParameter("i", records=[("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is also equivalent to -
# i = m.addParameter("i")
# i.setRecords([("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is also equivalent to -
# m.addParameter("i")
# m["i"].setRecords([("i" + str(i), i) for i in range(5)])
In [1]: i.records
Out[1]:
uni value
0 i0 0.0
1 i1 1.0
2 i2 2.0
3 i3 3.0
4 i4 4.0
Example #3 - Create a 1D parameter (defined over a set) from a list of tuples
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", ["*"], records=["i" + str(i) for i in range(5)])
a = gt.Parameter(m, "a", i, records=[("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is equivalent to -
# i = gt.Set(m, "i")
# i.setRecords(["i" + str(i) for i in range(5)])
# a = gt.Parameter(m, "a", i)
# a.setRecords([("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is also equivalent to -
# m.addSet("i", records=["i" + str(i) for i in range(5)])
# m.addParameter("a", i, records=[("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is also equivalent to -
# i = m.addSet("i")
# i.setRecords(["i" + str(i) for i in range(5)])
# a = m.addParameter("a", i)
# a.setRecords([("i" + str(i), i) for i in range(5)])
# NOTE: the above syntax is also equivalent to -
# m.addSet("i")
# m["i"].setRecords(["i" + str(i) for i in range(5)])
# m.addParameter("a", i)
# m["a"].setRecords([("i" + str(i), i) for i in range(5)])
In [1]: a.records
Out[1]:
i value
0 i0 0.0
1 i1 1.0
2 i2 2.0
3 i3 3.0
4 i4 4.0
Example #4 - Create a 2D parameter (defined over a set) from a DataFrame slice
import gams.transfer as gt
import pandas as pd
dist = pd.DataFrame(
[
("seattle", "new-york", 2.5),
("seattle", "chicago", 1.7),
("seattle", "topeka", 1.8),
("san-diego", "new-york", 2.5),
("san-diego", "chicago", 1.8),
("san-diego", "topeka", 1.4),
],
columns=["from", "to", "thousand_miles"],
)
m = gt.Container()
i = gt.Set(m, "i", ["*"], records=dist["from"].unique())
j = gt.Set(m, "j", ["*"], records=dist["to"].unique())
a = gt.Parameter(m, "a", [i, j], records=dist.loc[[0, 3], :])
# NOTE: the above syntax is equivalent to -
# i = gt.Set(m, "i")
# i.setRecords(dist["from"].unique())
# j = gt.Set(m, "j")
# j.setRecords(dist["to"].unique())
# a = gt.Parameter(m, "a", [i, j])
# a.setRecords(dist.loc[[0, 3], :])
# NOTE: the above syntax is also equivalent to -
# m.addSet("i", records=dist["from"].unique())
# m.addSet("j", records=dist["to"].unique())
# m.addParameter("a", i, records=dist.loc[[0, 3], :])
In [1]: a.records
Out[1]:
i j value
0 seattle new-york 2.5
3 san-diego new-york 2.5
Note
The original indexing is preserved when a user slices rows out of a reference dataframe.
Example #5 - Create a 2D parameter (defined over a set) from a matrix
import gams.transfer as gt
import pandas as pd
dist = pd.DataFrame(
[
("seattle", "new-york", 2.5),
("seattle", "chicago", 1.7),
("seattle", "topeka", 1.8),
("san-diego", "new-york", 2.5),
("san-diego", "chicago", 1.8),
("san-diego", "topeka", 1.4),
],
columns=["from", "to", "thousand_miles"],
)
m = gt.Container()
i = gt.Set(m, "i", ["*"], records=dist["from"].unique())
j = gt.Set(m, "j", ["*"], records=dist["to"].unique())
a = gt.Parameter(m, "a", [i, j], records=dist)
In [1]: a.toDense()
Out[1]:
array([[2.5, 1.7, 1.8],
[2.5, 1.8, 1.4]])
# use a.toDense() to create a new (and identical) parameter a2
a2 = gt.Parameter(m, "a2", [i, j], records=a.toDense())
# check that a is identical to a2
In [1]: a.equals(a2, check_meta_data=False)
Out[1]: True
Example #6 - Create a 2D parameter from an array using setRecords
import gams.transfer as gt
import numpy as np
import pandas as pd
m = gt.Container()
i = gt.Set(m, "i", records=["i" + str(i) for i in range(5)])
j = gt.Set(m, "j", records=["j" + str(j) for j in range(5)])
# create the parameter with linked domains (these will control the .shape of the symbol)
a = gt.Parameter(m, "a", [i, j])
# here we use the .shape property to easily generate a dense random array in numpy
a.setRecords(np.random.uniform(low=1, high=10, size=a.shape))
In [1]: a.toDense()
Out[1]:
array([[3.6694495 , 5.17395381, 1.99129484, 3.28315433, 1.44793791],
[1.06953243, 6.56331121, 5.26162554, 5.98098795, 8.30006 ],
[3.77213221, 5.82144901, 9.30035479, 9.12534285, 8.51970747],
[8.47965504, 7.84426304, 5.2442471 , 6.96666622, 6.55194415],
[5.62682779, 4.92509183, 8.94579609, 2.7724934 , 9.99576081]])
Example #7 - Create a 1D parameter from a pandas Series
import gams.transfer as gt
import pandas as pd
s = pd.Series(index=["a", "b", "c"], data=[i + 1 for i in range(3)])
m = gt.Container()
i = gt.Parameter(m, "i", ["*"], records=s, uels_on_axes=True)
In [1]: i.records
Out[1]:
uni value
0 a 1.0
1 b 2.0
2 c 3.0
Example #8 - Create a 2D parameter from a pandas Series
import gams.transfer as gt
import pandas as pd
dim1 = ["a", "b", "c"]
dim2 = ["z", "y", "x"]
s = pd.Series(
index=pd.MultiIndex.from_product([dim1, dim2]),
data=[i + 1 for i in range(len(dim1) * len(dim2))],
)
m = gt.Container()
i = gt.Parameter(m, "i", ["*", "*"], records=s, uels_on_axes=True)
In [1]: i.records
Out[1]:
uni_0 uni_1 value
0 a z 1.0
1 a y 2.0
2 a x 3.0
3 b z 4.0
4 b y 5.0
5 b x 6.0
6 c z 7.0
7 c y 8.0
8 c x 9.0
# NOTE: the order of the second dimension is automatically put into lexicographical order by Pandas
# This can be unexpected and undesirable from a GAMS perspective.
In [2]: s.index.levels[1]
Out[2]: Index(['x', 'y', 'z'], dtype='object')
# gams.transfer goes through extra work to maintain data order in the categorical data structures
# this can be see here where "z" is mapped to integer 0, "y" is mapped to 1 and "x" is mapped to 2
In [3]: i.records["uni_1"]
Out[3]:
0 z
1 y
2 x
3 z
4 y
5 x
6 z
7 y
8 x
Name: uni_1, dtype: category
Categories (3, object): ['z' < 'y' < 'x']
In [4]: i.records["uni_1"].cat.codes
Out[4]:
0 0
1 1
2 2
3 0
4 1
5 2
6 0
7 1
8 2
dtype: int8
Example #9 - Create a 2D parameter from a DataFrame (uels_on_axes=True)
import gams.transfer as gt
import pandas as pd
import numpy as np
dim1 = [f"d{i}" for i in range(2)]
dim2 = [f"d{i}" for i in range(2)]
dim3 = [f"d{i}" for i in range(2)]
dim4 = [f"d{i}" for i in range(2)]
rng = np.random.default_rng(seed=100)
df = pd.DataFrame(
data=rng.uniform(size=(2, 8)),
index=dim1,
columns=pd.MultiIndex.from_product([dim2, dim3, dim4]),
)
In [1]: df
Out[1]:
d0 d1
d0 d1 d0 d1
d0 d1 d0 d1 d0 d1 d0 d1
d0 0.834982 0.596554 0.288863 0.042952 0.973654 0.596472 0.790263 0.910339
d1 0.688154 0.189991 0.981479 0.284740 0.629273 0.581036 0.599912 0.535248
m = gt.Container()
i = gt.Parameter(m, "i", ["*"] * 4, records=df, uels_on_axes=True)
In [2]: i.records
Out[2]:
uni_0 uni_1 uni_2 uni_3 value
0 d0 d0 d0 d0 0.834982
1 d0 d0 d0 d1 0.596554
2 d0 d0 d1 d0 0.288863
3 d0 d0 d1 d1 0.042952
4 d0 d1 d0 d0 0.973654
5 d0 d1 d0 d1 0.596472
6 d0 d1 d1 d0 0.790263
7 d0 d1 d1 d1 0.910339
8 d1 d0 d0 d0 0.688154
9 d1 d0 d0 d1 0.189991
10 d1 d0 d1 d0 0.981479
11 d1 d0 d1 d1 0.284740
12 d1 d1 d0 d0 0.629273
13 d1 d1 d0 d1 0.581036
14 d1 d1 d1 d0 0.599912
15 d1 d1 d1 d1 0.535248

Directly Set Records

As with sets, the primary advantage of the setRecords method is that transfer will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.

Example #1 - Correctly set records (directly)
import gams.transfer as gt
import pandas as pd
import numpy as np
df = pd.DataFrame(
data=[
("h" + str(h), "m" + str(m), "s" + str(s))
for h in range(8760)
for m in range(60)
for s in range(60)
],
columns=["h", "m", "s"],
)
df["value"] = np.random.uniform(0, 100, len(df))
m = gt.Container()
hrs = gt.Set(m, "h", records=df["h"].unique())
mins = gt.Set(m, "m", records=df["m"].unique())
secs = gt.Set(m, "s", records=df["s"].unique())
df["h"] = df["h"].astype(hrs.records["uni"].dtype)
df["m"] = df["m"].astype(mins.records["uni"].dtype)
df["s"] = df["s"].astype(secs.records["uni"].dtype)
a = gt.Parameter(m, "a", [hrs, mins, secs])
# set records
a.records = df
In [1]: a.isValid()
Out[1]: True

In this example we create a large parameter (31,536,000 records and 8880 unique domain elements – we mimic data that is labeled for every second in one year) and assign it to a parameter with a.records. transfer requires that all domain columns must be a categorical data type, furthermore, this categorical must be ordered. The records setter function does very little work other than checking if the object being set is a DataFrame. This places more responsibility on the user to create a DataFrame that complies with the standard format. In Example #1 we take care to properly reference the categorical data types from the domain sets – and in the end a.isValid() = True.

Users will need to use the .isValid(verbose=True) method to debug any structural issues. As an example we incorrectly generate categorical data types by passing the DataFrame constructor the generic dtype="category" argument. This creates categorical column types but they are not ordered and they do not reference the underlying domain set. These errors result in a being invalid.

Example #2 - Incorrectly set records (directly)
import gams.transfer as gt
import pandas as pd
import numpy as np
df = pd.DataFrame(
data=[
("h" + str(h), "m" + str(m), "s" + str(s))
for h in range(8760)
for m in range(60)
for s in range(60)
],
columns=["h", "m", "s"],
dtype="category"
)
df["value"] = np.random.uniform(0, 100, len(df))
m = gt.Container()
hrs = gt.Set(m, "h", records=df["h"].unique())
mins = gt.Set(m, "m", records=df["m"].unique())
secs = gt.Set(m, "s", records=df["s"].unique())
a = gt.Parameter(m, "a", [hrs, mins, secs])
# set the records directly
a.records = df
In [1]: a.isValid()
Out[1]: False
In [2]: a.isValid(verbose=True)
Out[2]: Exception: Domain information in column 'h' for 'records' must be an ORDERED categorical type (i.e., <symbol_object>.records["h"].dtype.ordered = True)

Generate Parameter Records

Generating the initial pandas.DataFrame object could be difficult for Parameter symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords convenience function. Internally, generateRecords computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords will only work on symbols where <symbol>.domain_type == "regular").

Example #1 - Create a large (dense) 4D parameter
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Parameter(m, "a", [i, j, k, l])
# generate the records
a.generateRecords()
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l value
0 i0 j0 k0 l0 0.386390
1 i0 j0 k0 l1 0.671253
2 i0 j0 k0 l2 0.522057
3 i0 j0 k0 l3 0.037694
4 i0 j0 k0 l4 0.564205
... ... ... ... ... ...
6249995 i49 j49 k49 l45 0.573354
6249996 i49 j49 k49 l46 0.033717
6249997 i49 j49 k49 l47 0.410322
6249998 i49 j49 k49 l48 0.758310
6249999 i49 j49 k49 l49 0.920708
[6250000 rows x 5 columns]
Note
In Example #1 a large 4D parameter was generated – by default, the value of these records are randomly drawn numbers from the interval [0,1] (uniform distribution).

As with Sets, it is possible to generate a sparse parameter with the density argument to generateRecords. We extend this example by passing our own custom func argument that will control the behavior of the value columns. The func argument accepts a callable (i.e., a reference to a function).

Example #2 - Create a large (sparse) 4D parameter with normally distributed values
import gams.transfer as gt
import numpy as np
# create a custom function to pass to `generateRecords`
def value_dist(size, seed):
rng = np.random.default_rng(seed=seed)
return rng.normal(loc=10.0, scale=2.3, size=size)
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Parameter(m, "a", [i, j, k, l])
# generate the records
a.generateRecords(density=0.05, func=value_dist)
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l value
0 i0 j0 k0 l33 12.490579
1 i0 j0 k0 l43 9.460560
2 i0 j0 k0 l44 7.660337
3 i0 j0 k0 l47 8.811967
4 i0 j0 k1 l5 11.103291
... ... ... ... ... ...
312495 i49 j49 k48 l38 10.619791
312496 i49 j49 k48 l41 14.208250
312497 i49 j49 k48 l47 6.104145
312498 i49 j49 k49 l0 10.216812
312499 i49 j49 k49 l39 9.739771
[312500 rows x 5 columns]
In [3]: a.records["value"].mean()
Out[3]: 10.004072307451391
In [4]: a.records["value"].std()
Out[4]: 2.292569938350144
Note
The custom callable function reference must expose a size argument. It might be tedious to know the exact number of the records that will be generated, especially if a fractional density is specified; therefore, the generateRecords method will pass in the correct size automatically. Users are encouraged to use the Numpy suite of random distributions when generating samples – custom functions have the potential to be computationally burdensome if a symbol has a large number of records.
Example #3 - Create a large 4D parameter with 1 sparse dimension
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Parameter(m, "a", [i, j, k, l])
# generate the records
a.generateRecords(density=[1, 0.05, 1, 1])
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l value
0 i0 j30 k0 l0 0.473084
1 i0 j30 k0 l1 0.192571
2 i0 j30 k0 l2 0.060711
3 i0 j30 k0 l3 0.655477
4 i0 j30 k0 l4 0.629535
... ... ... ... ... ...
249995 i49 j32 k49 l45 0.442380
249996 i49 j32 k49 l46 0.002444
249997 i49 j32 k49 l47 0.332731
249998 i49 j32 k49 l48 0.983800
249999 i49 j32 k49 l49 0.984322
[250000 rows x 5 columns]
Example #4 - Create a large 4D parameter with a random number seed
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Parameter(m, "a", [i, j, k, l])
a2 = gt.Parameter(m, "a2", [i, j, k, l])
# generate the records
a.generateRecords(density=0.05, seed=123)
a2.generateRecords(density=0.05)
In [1]: a.equals(a2, check_meta_data=False)
Out[1]: False
In [2]: a2.generateRecords(density=0.05, seed=123)
In [3]: a.equals(a2, check_meta_data=False)
Out[3]: True
Note
The seed is an int that will set the random number generator state (enables reproducible sequences of random numbers).

Variable

There are two different ways to create a GAMS variable and add it to a Container.

  1. Use Variable constructor
  2. Use the Container method addVariable (which internally calls the Variable constructor)

Constructor

Constructor Arguments
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
description str Description of symbol No ""
domain list, str, or Set/Alias List of domains given either as string (* for universe set) or as reference to a Set/Alias object, an empty domain list will create a scalar variable No []
domain_forwarding bool or list Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. No False
name str Name of symbol Yes -
records many Symbol records No None
type str Type of variable being created [binary, integer, positive, negative, free, sos1, sos2, semicont, semiint] No free
uels_on_axes bool Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object No False
Note
Variable records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, has the same type, and has the same domain_forwarding state. The symbol description will only be updated if new text is provided.

Properties

Property Description Type Special Setter Behavior
container reference to the Container that the symbol belongs to Container -
description description of symbol str -
dimension dimension of symbol int setting is a shorthand notation to create ["*"] * n domains in symbol
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list, str, or Set/Alias -
domain_forwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. bool or list no effect after records have been set
domain_labels column headings for the records DataFrame list of str will add a _<dimension> tag to user supplied column names (if not unique)
domain_names string version of domain names list of str -
domain_type none, relaxed or regular depending on state of domain links str -
is_scalar True if the len(self.domain) = 0 bool -
modified Flag that identifies if the Variable has been modified bool -
name name of symbol str sets the GAMS name of the symbol
number_records number of symbol records (i.e., returns len(self.records) if not None) int -
records the main symbol records pandas.DataFrame responsive to domain_forwarding state
shape a tuple describing the array dimensions if records were converted with .toDense() tuple -
summary output a dict of only the metadata dict -
type str type of variable str -

Methods

Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol dimensions. If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
None
capitalizeUELs will capitalize all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
casefoldUELs will casefold all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
countDomainViolations returns the count of how many records contain at least one domain violation - int
countDuplicateRecords returns the count of how many (case insensitive) duplicate records exist - int
countEps total number of SpecialValues.EPS across all columns columns="level" (str, list) int or None
countNA total number of SpecialValues.NA across all columns columns="level" (str, list) int or None
countNegInf total number of SpecialValues.NEGINF across all columns columns="level" (str, list) int or None
countPosInf total number of SpecialValues.POSINF across all columns columns="level" (str, list) int or None
countUndef total number of SpecialValues.UNDEF across all columns columns="level" (str, list) int or None
dropDefaults drop records that are set to GAMS default records (check .default_records property for values) - None
dropDomainViolations drop records from the symbol that contain a domain violation - None
dropDuplicateRecords drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) keep="first" (str, False) None
dropEps drop records from the symbol that are GAMS EPS (zero 0.0 records will be retained) - None
dropMissing drop records from the symbol that are NaN (includes both NA and Undef special values) - None
dropNA drop records from the symbol that are GAMS NA - None
dropUndef drop records from the symbol that are GAMS Undef - None
equals Used to compare the symbol to another symbol. The columns argument allows the user to numerically compare only specified variable attributes (default is to compare all). If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_meta_data=True then check that symbol name, description and variable type are the same, otherwise skip. rtol (relative tolerance) and atol (absolute tolerance) set equality tolerances; can be different tolerances for different variable attributes (if specified as a dict). If verbose=True will return an exception from the asserter describing the nature of the difference. columns=["level", "marginal", "lower", "upper", "scale"]
check_uels=True (bool)
check_meta_data=True (bool)
rtol=0.0 (int, float, None)
atol=0.0 (int, float, None)
verbose=False (bool)
bool
findDomainViolations get a view of the records DataFrame that contain any domain violations - pandas.DataFrame
findDuplicateRecords get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) keep="first" (str, False) pandas.DataFrame
findEps find positions of SpecialValues.EPS in column column="level" (str) pandas.DataFrame or None
findNA find positions of SpecialValues.NA in column column="level" (str) pandas.DataFrame or None
findNegInf find positions of SpecialValues.NEGINF in column column="level" (str) pandas.DataFrame or None
findPosInf find positions of SpecialValues.POSINF in column column="level" (str) pandas.DataFrame or None
findUndef find positions of SpecialValues.Undef in column column="level" (str) pandas.DataFrame or None
generateRecords convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1]. If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. density=1.0 (float, list)
func=numpy.random.uniform(0,1) (dict of callables)
seed=None (int, None)
None
getMaxValue get the maximum value across all columns columns="level" (str, list) float or None
getMinValue get the minimum value across all columns columns="level" (str, list) float or None
getMeanValue get the mean value across all columns columns="level" (str, list) float or None
getMaxAbsValue get the maximum absolute value across all columns columns="level" (str, list) float
getSparsity get the sparsity of the symbol w.r.t the cardinality - float
getUELs gets UELs from symbol dimensions. If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int; must specify a single dimension if passing codes. Returns only UELs in the data if ignore_unused=True, otherwise return all UELs. dimensions=None (int, list, None)
codes=None (int, list, None)
ignore_unused=False (bool)
list
hasDomainViolations returns True if there are domain violations in the records, returns False if not. - bool
hasDuplicateRecords returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. - bool
isValid checks if the symbol is in a valid format, throw exceptions if verbose=True, recheck a symbol if force=True verbose=False
force=True
bool
ljustUELs will left justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
lowerUELs will lowercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
lstripUELs will left strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
pivot Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1]. If columns is None then it is set to the last dimension. If value is None then the level values will be pivoted. Missing values in the pivot will take the value provided by fill_value index=None (str, list, None)
columns=None (str, list, None)
value (str)
fill_value=None (int, float, str)
pd.DataFrame
removeUELs removes UELs that appear in the symbol dimensions, If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. uels=None (str, list, None)
dimensions=None (int, list, None)
bool
renameUELs renames UELs (case-sensitive) that appear in the symbol dimensions. If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True, the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** uels (str, list, dict)
dimensions (int, list, None)
allow_merge=False (bool)
None
reorderUELs reorders the UELs in the symbol dimensions. If uels is None, reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. uels (str, list, dict, None)
dimensions (int, list, None)
None
rjustUELs will right justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
rstripUELs will right strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
setRecords main convenience method to set standard pandas.DataFrame records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). records (many types) None
setUELs set the UELs for symbol dimensions. If dimensions is None then set UELs for all dimensions. If rename=True, then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
rename=False (bool)
None
stripUELs will strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
titleUELs will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
toDense convert column to a dense numpy.array format column="level" (str) numpy.array or None
toDict convenience method to return symbol records as a Python dictionary. columns will control which attributes to include in the dict. orient can take values natural or columns and will control the shape of the dict. Must use orient="columns" if attempting to set symbol records with setRecords. columns="level" (str)
orient="natural" (str)
dict
toList convenience method to return symbol records as a Python list, the columns argument will control with attributes to include in the list columns="level" (str) list
toSparseCoo convert column to a sparse COOrdinate numpy.array format column="level" (str) sparse matrix format or None
toValue convenience method to return symbol records as a Python float. Only possible with scalar symbols. Attribute can be specified with column argument. column="level" (str) float
upperUELs will uppercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
whereMax find the domain entry of records with a maximum value (return first instance only) column="level" (str) list of str or None
whereMaxAbs find the domain entry of records with a maximum absolute value (return first instance only) column="level" (str) list of str or None
whereMin find the domain entry of records with a minimum value (return first instance only) column="level" (str) list of str or None

Adding Variable Records

Three possibilities exist to assign symbol records to a variable (roughly ordered in complexity):

  1. Setting the argument records in the set constructor/container method (internally calls setRecords) - creates a data copy
  2. Using the symbol method setRecords - creates a data copy
  3. Setting the property records directly - does not create a data copy

If the data is in a convenient format, a user may want to pass the records directly within the variable constructor. This is an optional keyword argument and internally the variable constructor will simply call the setRecords method. In contrast to the setRecords methods in in either the Set or Parameter classes the setRecords method for variables will only accept Pandas DataFrames and specially structured dict for creating records from matrices. This restriction is out of necessity because to properly set a record for a Variable the user must pass data for the level, marginal, lower, upper and scale attributes. That said, any missing attributes will be filled in with the GAMS default record values (see: Variable Types), default scale value is always 1, and the default level and marginal values are 0 for all variable types). We show a few examples of ways to create differently structured variables:

Example #1 - Create a GAMS scalar variable
import gams.transfer as gt
m = gt.Container()
pi = gt.Variable(m, "pi", records=pd.DataFrame(data=[3.14159], columns=["level"]))
# NOTE: the above syntax is equivalent to -
# pi = gt.Variable(m, "pi", "free")
# pi.setRecords(pd.DataFrame(data=[3.14159], columns=["level"]))
# NOTE: the above syntax is also equivalent to -
# m.addVariable("pi", "free", records=pd.DataFrame(data=[3.14159], columns=["level"]))
In [1]: pi.records
Out[1]:
level marginal lower upper scale
0 3.14159 0.0 -inf inf 1.0
Example #2 - Create a 1D variable (defined over *) from a list of tuples

In this example we only set the marginal values.

import gams.transfer as gt
m = gt.Container()
v = gt.Variable(
m,
"v",
"free",
domain=["*"],
records=pd.DataFrame(
data=[("i" + str(i), i) for i in range(5)], columns=["domain", "marginal"]
),
)
In [1]: v.records
Out[1]:
uni level marginal lower upper scale
0 i0 0.0 0.0 -inf inf 1.0
1 i1 0.0 1.0 -inf inf 1.0
2 i2 0.0 2.0 -inf inf 1.0
3 i3 0.0 3.0 -inf inf 1.0
4 i4 0.0 4.0 -inf inf 1.0
Example #3 - Create a 1D variable (defined over a set) from a list of tuples
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", ["*"], records=["i" + str(i) for i in range(5)])
v = gt.Variable(
m,
"v",
"free",
domain=i,
records=pd.DataFrame(
data=[("i" + str(i), i) for i in range(5)], columns=["domain", "marginal"]
),
)
In [1]: v.records
Out[1]:
i level marginal lower upper scale
0 i0 0.0 0.0 -inf inf 1.0
1 i1 0.0 1.0 -inf inf 1.0
2 i2 0.0 2.0 -inf inf 1.0
3 i3 0.0 3.0 -inf inf 1.0
4 i4 0.0 4.0 -inf inf 1.0
Example #4 - Create a 2D positive variable, specifying no numerical data
import gams.transfer as gt
import pandas as pd
m = gt.Container()
v = gt.Variable(
m,
"v",
"positive",
["*", "*"],
records=pd.DataFrame([("seattle", "san-diego"), ("chicago", "madison")]),
)
In [1]: v.records
Out[1]:
uni_0 uni_1 level marginal lower upper scale
0 seattle san-diego 0.0 0.0 0.0 inf 1.0
1 chicago madison 0.0 0.0 0.0 inf 1.0
Example #5 - Create a 2D variable (defined over a set) from a matrix
import gams.transfer as gt
import pandas as pd
import numpy as np
m = gt.Container()
i = gt.Set(m, "i", ["*"], records=["i" + str(i) for i in range(5)])
j = gt.Set(m, "j", ["*"], records=["j" + str(i) for i in range(5)])
a = gt.Parameter(
m,
"a",
[i, j],
records=[("i" + str(i), "j" + str(j), i + j) for i in range(5) for j in range(5)],
)
# create a free variable and set the level and marginal attributes from matricies
v = gt.Variable(
m, "v", domain=[i, j], records={"level": a.toDense(), "marginal": a.toDense()}
)
# if not specified, the toDense() method will convert the level values to a matrix
In [1]: v.toDense()
Out[1]:
array([[0., 1., 2., 3., 4.],
[1., 2., 3., 4., 5.],
[2., 3., 4., 5., 6.],
[3., 4., 5., 6., 7.],
[4., 5., 6., 7., 8.]])
Example #6 - Create a 1D variable from a pandas Series
import gams.transfer as gt
import pandas as pd
s = pd.Series(index=["a", "b", "c"], data=[i + 1 for i in range(3)])
m = gt.Container()
v = gt.Variable(m, "v", domain=["*"], records=s, uels_on_axes=True)
In [1]: v.records
Out[1]:
uni level marginal lower upper scale
0 a 1.0 0.0 -inf inf 1.0
1 b 2.0 0.0 -inf inf 1.0
2 c 3.0 0.0 -inf inf 1.0
Note
transfer will assume that the level value is being set if attributes cannot be found in the axes
Example #7 - Create a 1D variable from a pandas Series (set only the marginal)
import gams.transfer as gt
import pandas as pd
# NOTE: we include the "marginal" label in level 1 of the MultiIndex
s = pd.Series(
index=pd.MultiIndex.from_product([["a", "b", "c"], ["marginal"]]),
data=[i + 1 for i in range(3)],
)
m = gt.Container()
v = gt.Variable(m, "v", domain=["*"], records=s, uels_on_axes=True)
In [1]: v.records
Out[1]:
uni level marginal lower upper scale
0 a 0.0 1.0 -inf inf 1.0
1 b 0.0 2.0 -inf inf 1.0
2 c 0.0 3.0 -inf inf 1.0
Note
transfer will search the axes for any variable attributes and set those attributes provided. It is necessary to include all attributes into the same axes level (if a MultiIndex) without any other UELs – if other UELs are found in the same level then all element will be interpreted as UELs and could result in GAMS domain violations.
Example #8 - Create a 2D variable from a DataFrame (uels_on_axes=True)
import gams.transfer as gt
import pandas as pd
import numpy as np
dim1 = [f"d{i}" for i in range(2)]
dim2 = [f"e{i}" for i in range(2)]
dim3 = [f"f{i}" for i in range(2)]
dim4 = [f"g{i}" for i in range(2)]
rng = np.random.default_rng(seed=100)
df = pd.DataFrame(
data=rng.uniform(size=(4, 4)),
index=pd.MultiIndex.from_product([dim1, dim2]),
columns=pd.MultiIndex.from_product([dim3, dim4]),
)
In [1]: df
Out[1]:
f0 f1
g0 g1 g0 g1
d0 e0 0.834982 0.596554 0.288863 0.042952
e1 0.973654 0.596472 0.790263 0.910339
d1 e0 0.688154 0.189991 0.981479 0.284740
e1 0.629273 0.581036 0.599912 0.535248
m = gt.Container()
v = gt.Variable(m, "v", domain=["*"] * 4, records=df, uels_on_axes=True)
In [8]: v.records
Out[8]:
uni_0 uni_1 uni_2 uni_3 level marginal lower upper scale
0 d0 e0 f0 g0 0.834982 0.0 -inf inf 1.0
1 d0 e0 f0 g1 0.596554 0.0 -inf inf 1.0
2 d0 e0 f1 g0 0.288863 0.0 -inf inf 1.0
3 d0 e0 f1 g1 0.042952 0.0 -inf inf 1.0
4 d0 e1 f0 g0 0.973654 0.0 -inf inf 1.0
5 d0 e1 f0 g1 0.596472 0.0 -inf inf 1.0
6 d0 e1 f1 g0 0.790263 0.0 -inf inf 1.0
7 d0 e1 f1 g1 0.910339 0.0 -inf inf 1.0
8 d1 e0 f0 g0 0.688154 0.0 -inf inf 1.0
9 d1 e0 f0 g1 0.189991 0.0 -inf inf 1.0
10 d1 e0 f1 g0 0.981479 0.0 -inf inf 1.0
11 d1 e0 f1 g1 0.284740 0.0 -inf inf 1.0
12 d1 e1 f0 g0 0.629273 0.0 -inf inf 1.0
13 d1 e1 f0 g1 0.581036 0.0 -inf inf 1.0
14 d1 e1 f1 g0 0.599912 0.0 -inf inf 1.0
15 d1 e1 f1 g1 0.535248 0.0 -inf inf 1.0

Directly Set Records

As with sets, the primary advantage of the setRecords method is that transfer will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.

Example #1 - Correctly set records (directly)
import gams.transfer as gt
import pandas as pd
import numpy as np
df = pd.DataFrame(
data=[
("h" + str(h), "m" + str(m), "s" + str(s))
for h in range(8760)
for m in range(60)
for s in range(60)
],
columns=["h", "m", "s"],
)
# it is necessary to specify all variable attributes if setting records directly
# NOTE: all numeric data must be type float
df["level"] = np.random.uniform(0, 100, len(df))
df["marginal"] = 0.0
df["lower"] = gt.SpecialValues.NEGINF
df["upper"] = gt.SpecialValues.POSINF
df["scale"] = 1.0
m = gt.Container()
hrs = gt.Set(m, "h", records=df["h"].unique())
mins = gt.Set(m, "m", records=df["m"].unique())
secs = gt.Set(m, "s", records=df["s"].unique())
df["h"] = df["h"].astype(hrs.records["uni"].dtype)
df["m"] = df["m"].astype(mins.records["uni"].dtype)
df["s"] = df["s"].astype(secs.records["uni"].dtype)
a = gt.Variable(m, "a", domain=[hrs, mins, secs])
# set records
a.records = df
In [1]: a.isValid()
Out[1]: True
Attention
All numeric data in the records will need to be type float in order to maintain a valid symbol.

In this example we create a large variable (31,536,000 records and 8880 unique domain elements – we mimic data that is labeled for every second in one year) and assign it to a variable with a.records. transfer requires that all domain columns must be a categorical data type, furthermore this categorical must be ordered. The records setter function does very little work other than checking if the object being set is a DataFrame. This places more responsibility on the user to create a DataFrame that complies with the standard format. In Example #1 we take care to properly reference the categorical data types from the domain sets – and in the end a.isValid() = True. As with Set and Parameters, users can use the .isValid(verbose=True) method to debug any structural issues.

Generate Variable Records

Generating the initial pandas.DataFrame object could be difficult for Variable symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords convenience function. Internally, generateRecords computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords will only work on symbols where <symbol>.domain_type == "regular").

Example #1 - Create a large (dense) 4D variable
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Variable(m, "a", "free", [i, j, k, l])
# generate the records
a.generateRecords()
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l level marginal lower upper scale
0 i0 j0 k0 l0 0.470248 0.0 -inf inf 1.0
1 i0 j0 k0 l1 0.924286 0.0 -inf inf 1.0
2 i0 j0 k0 l2 0.347550 0.0 -inf inf 1.0
3 i0 j0 k0 l3 0.937009 0.0 -inf inf 1.0
4 i0 j0 k0 l4 0.050716 0.0 -inf inf 1.0
... ... ... ... ... ... ... ... ... ...
6249995 i49 j49 k49 l45 0.385032 0.0 -inf inf 1.0
6249996 i49 j49 k49 l46 0.029305 0.0 -inf inf 1.0
6249997 i49 j49 k49 l47 0.440716 0.0 -inf inf 1.0
6249998 i49 j49 k49 l48 0.432931 0.0 -inf inf 1.0
6249999 i49 j49 k49 l49 0.157107 0.0 -inf inf 1.0
[6250000 rows x 9 columns]
Note
In Example #1 a large 4D variable was generated – by default, only the level value of these records are randomly drawn from the interval [0,1] (uniform distribution). Other variable attributes take the default record value.

As with Parameters, it is possible to generate a sparse variable with the density argument to generateRecords. We extend this example by passing our own custom func argument that will control the behavior of the value columns. The func argument accepts a dict of callables (i.e., a reference to a function).

Example #2 - Create a large (sparse) 4D variable with normally distributed values
import gams.transfer as gt
import numpy as np
# create a custom function to pass to `generateRecords`
def level_dist(size):
return np.random.normal(loc=10.0, scale=2.3, size=size)
def marginal_dist(size):
return np.random.normal(loc=0.5, scale=0.1, size=size)
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Variable(m, "a", "free", [i, j, k, l])
# generate the records
a.generateRecords(density=0.05, func={"level":level_dist, "marginal":marginal_dist})
In [1]: a.isValid()
Out[1]: True
In [12]: a.records
Out[12]:
i j k l level marginal lower upper scale
0 i0 j0 k0 l36 11.105235 0.468989 -inf inf 1.0
1 i0 j0 k0 l40 5.697361 0.478019 -inf inf 1.0
2 i0 j0 k1 l17 11.900784 0.473814 -inf inf 1.0
3 i0 j0 k1 l24 10.105931 0.456925 -inf inf 1.0
4 i0 j0 k1 l31 8.444142 0.490966 -inf inf 1.0
... ... ... ... ... ... ... ... ... ...
312495 i49 j49 k47 l17 11.523186 0.508001 -inf inf 1.0
312496 i49 j49 k47 l20 9.341183 0.739237 -inf inf 1.0
312497 i49 j49 k47 l26 10.705808 0.581103 -inf inf 1.0
312498 i49 j49 k47 l32 7.910963 0.479655 -inf inf 1.0
312499 i49 j49 k49 l8 11.800414 0.628040 -inf inf 1.0
[312500 rows x 9 columns]
In [3]: a.records["level"].mean()
Out[3]: 10.004072307451391
In [4]: a.records["level"].std()
Out[4]: 2.292569938350144
In [5]: a.records["marginal"].mean()
Out[5]: 0.49970172269778
In [6]: a.records["marginal"].std()
Out[6]: 0.09998772109802055
Note
The custom callable function reference must expose a size argument. It might be tedious to know the exact number of the records that will be generated, especially if a fractional density is specified; therefore, the generateRecords method will pass in the correct size automatically. Users are encouraged to use the Numpy suite of random distributions when generating samples – custom functions have the potential to be computationally burdensome if a symbol has a large number of records.

Equation

There are two different ways to create a GAMS equation and add it to a Container.

  1. Use Equation constructor
  2. Use the Container method addEquation (which internally calls the Equation constructor)

Constructor

Constructor Arguments
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
description str Description of symbol No ""
domain list, str, or Set/Alias List of domains given either as string (* for universe set) or as reference to a Set/Alias object, an empty domain list will create a scalar equation No []
domain_forwarding bool or list Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. No False
name str Name of symbol Yes -
records many Symbol records No None
type str Type of equation being created [eq (or E/e), geq (or G/g), leq (or L/l), nonbinding (or N/n), external (or X/x)] Yes -
uels_on_axes bool Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object No False
Note
Equation records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, has the same type, and has the same domain_forwarding state. The symbol description will only be updated if new text is provided.

Properties

Property Description Type Special Setter Behavior
container reference to the Container that the symbol belongs to Container -
description description of symbol str -
dimension dimension of symbol int setting is a shorthand notation to create ["*"] * n domains in symbol
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list, str, or Set/Alias -
domain_forwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. bool or list no effect after records have been set
domain_labels column headings for the records DataFrame list of str will add a _<dimension> tag to user supplied column names (if not unique)
domain_names string version of domain names list of str -
domain_type none, relaxed or regular depending on state of domain links str -
is_scalar True if the len(self.domain) = 0 bool -
modified Flag that identifies if the Equation has been modified bool -
name name of symbol str sets the GAMS name of the symbol
number_records number of symbol records (i.e., returns len(self.records) if not None) int -
records the main symbol records pandas.DataFrame responsive to domain_forwarding state
shape a tuple describing the array dimensions if records were converted with .toDense() tuple -
summary output a dict of only the metadata dict -
type str type of variable str -

Methods

Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol dimensions. If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
None
capitalizeUELs will capitalize all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
casefoldUELs will casefold all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
countDomainViolations returns the count of how many records contain at least one domain violation - int
countDuplicateRecords returns the count of how many (case insensitive) duplicate records exist - int
countEps total number of SpecialValues.EPS across all columns columns="level" (str, list) int or None
countNA total number of SpecialValues.NA across all columns columns="level" (str, list) int or None
countNegInf total number of SpecialValues.NEGINF across all columns columns="level" (str, list) int or None
countPosInf total number of SpecialValues.POSINF across all columns columns="level" (str, list) int or None
countUndef total number of SpecialValues.UNDEF across all columns columns="level" (str, list) int or None
dropDefaults drop records that are set to GAMS default records (check .default_records property for values) - None
dropDomainViolations drop records from the symbol that contain a domain violation - None
dropDuplicateRecords drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) keep="first" (str, False) None
dropEps drop records from the symbol that are GAMS EPS (zero 0.0 records will be retained) - None
dropMissing drop records from the symbol that are NaN (includes both NA and Undef special values) - None
dropNA drop records from the symbol that are GAMS NA - None
dropUndef drop records from the symbol that are GAMS Undef - None
equals Used to compare the symbol to another symbol. The columns argument allows the user to numerically compare only specified equation attributes (default is to compare all). If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_meta_data=True then check that symbol name, description and equation type are the same, otherwise skip. rtol (relative tolerance) and atol (absolute tolerance) set equality tolerances; can be different tolerances for different equation attributes (if specified as a dict). If verbose=True will return an exception from the asserter describing the nature of the difference. columns=["level", "marginal", "lower", "upper", "scale"]
check_uels=True (bool)
check_meta_data=True (bool)
rtol=0.0 (int, float, None)
atol=0.0 (int, float, None)
verbose=False (bool)
bool
findDomainViolations get a view of the records DataFrame that contain any domain violations - pandas.DataFrame
findDuplicateRecords get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) keep="first" (str, False) pandas.DataFrame
findEps find positions of SpecialValues.EPS in column column="level" (str) pandas.DataFrame or None
findNA find positions of SpecialValues.NA in column column="level" (str) pandas.DataFrame or None
findNegInf find positions of SpecialValues.NEGINF in column column="level" (str) pandas.DataFrame or None
findPosInf find positions of SpecialValues.POSINF in column column="level" (str) pandas.DataFrame or None
findUndef find positions of SpecialValues.Undef in column column="level" (str) pandas.DataFrame or None
generateRecords convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1]. If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. density=1.0 (float, list)
func=numpy.random.uniform(0,1) (dict of callables)
seed=None (int, None)
None
getMaxAbsValue get the maximum absolute value across all columns columns="level" (str, list) float or None
getMaxValue get the maximum value across all columns columns="level" (str, list) float or None
getMeanValue get the mean value across all columns columns="level" (str, list) float or None
getMinValue get the minimum value across all columns columns="level" (str, list) float or None
getSparsity get the sparsity of the symbol w.r.t the cardinality - float
getUELs gets UELs from symbol dimensions. If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int; must specify a single dimension if passing codes. Returns only UELs in the data if ignore_unused=True, otherwise return all UELs. dimensions=None (int, list, None)
codes=None (int, list, None)
ignore_unused=False (bool)
list
hasDomainViolations returns True if there are domain violations in the records, returns False if not. - bool
hasDuplicateRecords returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. - bool
isValid checks if the symbol is in a valid format, throw exceptions if verbose=True, recheck a symbol if force=True verbose=False
force=True
bool
ljustUELs will left justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
lowerUELs will lowercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
lstripUELs will left strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
pivot Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1]. If columns is None then it is set to the last dimension. If value is None then the level values will be pivoted. Missing values in the pivot will take the value provided by fill_value index=None (str, list, None)
columns=None (str, list, None)
value (str)
fill_value=None (int, float, str)
pd.DataFrame
removeUELs removes UELs that appear in the symbol dimensions, If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. uels=None (str, list, None)
dimensions=None (int, list, None)
bool
renameUELs renames UELs (case-sensitive) that appear in the symbol dimensions. If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True, the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** uels (str, list, dict)
dimensions (int, list, None)
allow_merge=False (bool)
None
reorderUELs reorders the UELs in the symbol dimensions. If uels is None, reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. uels (str, list, dict, None)
dimensions (int, list, None)
None
rjustUELs will right justify all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
rstripUELs will right strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
setRecords main convenience method to set standard pandas.DataFrame records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). records (many types) None
setUELs set the UELs for symbol dimensions. If dimensions is None then set UELs for all dimensions. If rename=True, then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
rename=False (bool)
None
stripUELs will strip whitespace from all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
titleUELs will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
toDense convert column to a dense numpy.array format column="level" (str) numpy.array or None
toDict convenience method to return symbol records as a Python dictionary. columns will control which attributes to include in the dict. orient can take values natural or columns and will control the shape of the dict. Must use orient="columns" if attempting to set symbol records with setRecords. columns="level" (str)
orient="natural" (str)
dict
toList convenience method to return symbol records as a Python list, the columns argument will control with attributes to include in the list columns="level" (str) list
toSparseCoo convert column to a sparse COOrdinate numpy.array format column="level" (str) sparse matrix format or None
toValue convenience method to return symbol records as a Python float. Only possible with scalar symbols. Attribute can be specified with column argument. column="level" (str) float
upperUELs will uppercase all UELs in the symbol or a subset of specified dimensions, can be chained with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
whereMax find the domain entry of records with a maximum value (return first instance only) column="level" (str) list of str or None
whereMaxAbs find the domain entry of records with a maximum absolute value (return first instance only) column="level" (str) list of str or None
whereMin find the domain entry of records with a minimum value (return first instance only) column="level" (str) list of str or None

Adding Equation Records

Adding equation records mimics that of variables – three possibilities exist to assign symbol records to an equation (roughly ordered in complexity):

  1. Setting the argument records in the set constructor/container method (internally calls setRecords) - creates a data copy
  2. Using the symbol method setRecords - creates a data copy
  3. Setting the property records directly - does not create a data copy

Setting equation records require the user to be explicit with the type of equation that is being created; in contrast to setting variable records (where the default variable is considered to be free).

If the data is in a convenient format, a user may want to pass the records directly within the equation constructor. This is an optional keyword argument and internally the equation constructor will simply call the setRecords method. In contrast to the setRecords methods in in either the Set or Parameter classes the setRecords method for variables will only accept Pandas DataFrames and specially structured dict for creating records from matrices. This restriction is out of necessity because to properly set a record for an Equation the user must pass data for the level, marginal, lower, upper and scale attributes. That said, any missing attributes will be filled in with the GAMS default record values (level = 0.0, marginal = 0.0, lower = -inf, upper = inf, scale = 1.0). We show a few examples of ways to create differently structured variables:

Example #1 - Create a GAMS scalar equation
import gams.transfer as gt
m = gt.Container()
# here we create an equality (=E=) equation
z = gt.Equation(m, "z", "eq", records=pd.DataFrame(data=[3.14159], columns=["level"]))
# NOTE: the above syntax is equivalent to -
# pi = gt.Equation(m, "pi", "eq")
# pi.setRecords(pd.DataFrame(data=[3.14159], columns=["level"]))
# NOTE: the above syntax is also equivalent to -
# m.addEquation("pi", "eq", records=pd.DataFrame(data=[3.14159], columns=["level"]))
In [1]: pi.records
Out[1]:
level marginal lower upper scale
0 3.14159 0.0 -inf inf 1.0
Example #2 - Create a 1D Equation (defined over *) from a list of tuples

In this example we only set the marginal values.

import gams.transfer as gt
m = gt.Container()
# here we define a greater than or equal (=G=) equation
i = gt.Equation(
m,
"i",
"geq",
domain=["*"],
records=pd.DataFrame(
data=[("i" + str(i), i) for i in range(5)], columns=["domain", "marginal"]
),
)
In [1]: i.type
Out[1]: 'geq'
In [2]: i.records
Out[2]:
uni level marginal lower upper scale
0 i0 0.0 0.0 -inf inf 1.0
1 i1 0.0 1.0 -inf inf 1.0
2 i2 0.0 2.0 -inf inf 1.0
3 i3 0.0 3.0 -inf inf 1.0
4 i4 0.0 4.0 -inf inf 1.0
Example #3 - Create a 1D Equation (defined over a set) from a list of tuples
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", ["*"], records=["i" + str(i) for i in range(5)])
# here we define a less than or equal (=L=) equation
e = gt.Equation(
m,
"e",
"leq",
domain=i,
records=pd.DataFrame(
data=[("i" + str(i), i) for i in range(5)], columns=["domain", "marginal"]
),
)
In [1]: i.type
Out[1]: 'leq'
In [5]: e.records
Out[5]:
i level marginal lower upper scale
0 i0 0.0 0.0 -inf inf 1.0
1 i1 0.0 1.0 -inf inf 1.0
2 i2 0.0 2.0 -inf inf 1.0
3 i3 0.0 3.0 -inf inf 1.0
4 i4 0.0 4.0 -inf inf 1.0
Example #4 - Create a 2D equation, specifying no numerical data
import gams.transfer as gt
import pandas as pd
m = gt.Container()
e = gt.Equation(
m,
"e",
"eq",
["*", "*"],
records=pd.DataFrame([("seattle", "san-diego"), ("chicago", "madison")]),
)
In [1]: e.records
Out[1]:
uni_0 uni_1 level marginal lower upper scale
0 seattle san-diego 0.0 0.0 -inf inf 1.0
1 chicago madison 0.0 0.0 -inf inf 1.0
Example #5 - Create a 2D equation (defined over a set) from a matrix
import gams.transfer as gt
import pandas as pd
import numpy as np
m = gt.Container()
i = gt.Set(m, "i", ["*"], records=["i" + str(i) for i in range(5)])
j = gt.Set(m, "j", ["*"], records=["j" + str(i) for i in range(5)])
a = gt.Parameter(
m,
"a",
[i, j],
records=[("i" + str(i), "j" + str(j), i + j) for i in range(5) for j in range(5)],
)
# create a nonbinding (=N=) equation and set the level and marginal attributes from matricies
e = gt.Equation(
m, "e", "nonbinding", domain=[i, j], records={"level": a.toDense(), "marginal": a.toDense()}
)
In [1]: e.records
Out[1]:
i j level marginal lower upper scale
0 i0 j1 1.0 1.0 -inf inf 1.0
1 i0 j2 2.0 2.0 -inf inf 1.0
2 i0 j3 3.0 3.0 -inf inf 1.0
3 i0 j4 4.0 4.0 -inf inf 1.0
4 i1 j0 1.0 1.0 -inf inf 1.0
5 i1 j1 2.0 2.0 -inf inf 1.0
6 i1 j2 3.0 3.0 -inf inf 1.0
7 i1 j3 4.0 4.0 -inf inf 1.0
8 i1 j4 5.0 5.0 -inf inf 1.0
9 i2 j0 2.0 2.0 -inf inf 1.0
10 i2 j1 3.0 3.0 -inf inf 1.0
11 i2 j2 4.0 4.0 -inf inf 1.0
12 i2 j3 5.0 5.0 -inf inf 1.0
13 i2 j4 6.0 6.0 -inf inf 1.0
14 i3 j0 3.0 3.0 -inf inf 1.0
15 i3 j1 4.0 4.0 -inf inf 1.0
16 i3 j2 5.0 5.0 -inf inf 1.0
17 i3 j3 6.0 6.0 -inf inf 1.0
18 i3 j4 7.0 7.0 -inf inf 1.0
19 i4 j0 4.0 4.0 -inf inf 1.0
20 i4 j1 5.0 5.0 -inf inf 1.0
21 i4 j2 6.0 6.0 -inf inf 1.0
22 i4 j3 7.0 7.0 -inf inf 1.0
23 i4 j4 8.0 8.0 -inf inf 1.0
# if not specified, the toDense() method will convert the level values to a matrix
In [2]: e.toDense()
Out[2]:
array([[0., 1., 2., 3., 4.],
[1., 2., 3., 4., 5.],
[2., 3., 4., 5., 6.],
[3., 4., 5., 6., 7.],
[4., 5., 6., 7., 8.]])
Example #6 - Create a 1D equation from a pandas Series
import gams.transfer as gt
import pandas as pd
s = pd.Series(index=["a", "b", "c"], data=[i + 1 for i in range(3)])
m = gt.Container()
e = gt.Equation(m, "e", "eq", domain=["*"], records=s, uels_on_axes=True)
In [1]: e.records
Out[1]:
uni level marginal lower upper scale
0 a 1.0 0.0 -inf inf 1.0
1 b 2.0 0.0 -inf inf 1.0
2 c 3.0 0.0 -inf inf 1.0
Note
transfer will assume that the level value is being set if attributes cannot be found in the axes (same behavior when setting Variables)
Example #7 - Create a 1D equation from a pandas Series (set only the marginal)
import gams.transfer as gt
import pandas as pd
# NOTE: we include the "marginal" label in level 1 of the MultiIndex
s = pd.Series(
index=pd.MultiIndex.from_product([["a", "b", "c"], ["marginal"]]),
data=[i + 1 for i in range(3)],
)
m = gt.Container()
e = gt.Equation(m, "e", "eq", domain=["*"], records=s, uels_on_axes=True)
In [1]: v.records
Out[1]:
uni level marginal lower upper scale
0 a 0.0 1.0 -inf inf 1.0
1 b 0.0 2.0 -inf inf 1.0
2 c 0.0 3.0 -inf inf 1.0
Example #8 - Create a 2D equation from a DataFrame (uels_on_axes=True)
import gams.transfer as gt
import pandas as pd
import numpy as np
dim1 = [f"d{i}" for i in range(2)]
dim2 = [f"e{i}" for i in range(2)]
dim3 = [f"f{i}" for i in range(2)]
dim4 = [f"g{i}" for i in range(2)]
rng = np.random.default_rng(seed=100)
df = pd.DataFrame(
data=rng.uniform(size=(4, 4)),
index=pd.MultiIndex.from_product([dim1, dim2]),
columns=pd.MultiIndex.from_product([dim3, dim4]),
)
In [1]: df
Out[1]:
f0 f1
g0 g1 g0 g1
d0 e0 0.834982 0.596554 0.288863 0.042952
e1 0.973654 0.596472 0.790263 0.910339
d1 e0 0.688154 0.189991 0.981479 0.284740
e1 0.629273 0.581036 0.599912 0.535248
m = gt.Container()
e = gt.Equation(m, "e", "eq", domain=["*"] * 4, records=df, uels_on_axes=True)
In [8]: e.records
Out[8]:
uni_0 uni_1 uni_2 uni_3 level marginal lower upper scale
0 d0 e0 f0 g0 0.834982 0.0 -inf inf 1.0
1 d0 e0 f0 g1 0.596554 0.0 -inf inf 1.0
2 d0 e0 f1 g0 0.288863 0.0 -inf inf 1.0
3 d0 e0 f1 g1 0.042952 0.0 -inf inf 1.0
4 d0 e1 f0 g0 0.973654 0.0 -inf inf 1.0
5 d0 e1 f0 g1 0.596472 0.0 -inf inf 1.0
6 d0 e1 f1 g0 0.790263 0.0 -inf inf 1.0
7 d0 e1 f1 g1 0.910339 0.0 -inf inf 1.0
8 d1 e0 f0 g0 0.688154 0.0 -inf inf 1.0
9 d1 e0 f0 g1 0.189991 0.0 -inf inf 1.0
10 d1 e0 f1 g0 0.981479 0.0 -inf inf 1.0
11 d1 e0 f1 g1 0.284740 0.0 -inf inf 1.0
12 d1 e1 f0 g0 0.629273 0.0 -inf inf 1.0
13 d1 e1 f0 g1 0.581036 0.0 -inf inf 1.0
14 d1 e1 f1 g0 0.599912 0.0 -inf inf 1.0
15 d1 e1 f1 g1 0.535248 0.0 -inf inf 1.0

Directly Set Records

As with set, parameters and variables, the primary advantage of the setRecords method is that transfer will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.

Example #1 - Correctly set records (directly)
import gams.transfer as gt
import pandas as pd
import numpy as np
df = pd.DataFrame(
data=[
("h" + str(h), "m" + str(m), "s" + str(s))
for h in range(8760)
for m in range(60)
for s in range(60)
],
columns=["h", "m", "s"],
)
# it is necessary to specify all variable attributes if setting records directly
# NOTE: all numeric data must be type float
df["level"] = np.random.uniform(0, 100, len(df))
df["marginal"] = 0.0
df["lower"] = gt.SpecialValues.NEGINF
df["upper"] = gt.SpecialValues.POSINF
df["scale"] = 1.0
m = gt.Container()
hrs = gt.Set(m, "h", records=df["h"].unique())
mins = gt.Set(m, "m", records=df["m"].unique())
secs = gt.Set(m, "s", records=df["s"].unique())
df["h"] = df["h"].astype(hrs.records["uni"].dtype)
df["m"] = df["m"].astype(mins.records["uni"].dtype)
df["s"] = df["s"].astype(secs.records["uni"].dtype)
a = gt.Equation(m, "a", "eq", domain=[hrs, mins, secs])
# set records
a.records = df
In [1]: e.isValid()
Out[1]: True
Attention
All numeric data in the records will need to be type float in order to maintain a valid symbol.

In this example we create a large equation (31,536,000 records and 8880 unique domain elements) and assign it to a variable with a.records. transfer requires that all domain columns must be a categorical data type, furthermore this categorical must be ordered. The records setter function does very little work other than checking if the object being set is a DataFrame. This places more responsibility on the user to create a DataFrame that complies with the standard format. In Example #1 we take care to properly reference the categorical data types from the domain sets – and in the end a.isValid() = True. As with Set and Parameters, users can use the .isValid(verbose=True) method to debug any structural issues.

Generate Equation Records

Generating the initial pandas.DataFrame object could be difficult for Equation symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords convenience function. Internally, generateRecords computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords will only work on symbols where <symbol>.domain_type == "regular").

Example #1 - Create a large (dense) 4D equation
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Equation(m, "a", "eq", [i, j, k, l])
# generate the records
a.generateRecords()
In [1]: a.isValid()
Out[1]: True
In [2]: a.records
Out[2]:
i j k l level marginal lower upper scale
0 i0 j0 k0 l0 0.470248 0.0 -inf inf 1.0
1 i0 j0 k0 l1 0.924286 0.0 -inf inf 1.0
2 i0 j0 k0 l2 0.347550 0.0 -inf inf 1.0
3 i0 j0 k0 l3 0.937009 0.0 -inf inf 1.0
4 i0 j0 k0 l4 0.050716 0.0 -inf inf 1.0
... ... ... ... ... ... ... ... ... ...
6249995 i49 j49 k49 l45 0.385032 0.0 -inf inf 1.0
6249996 i49 j49 k49 l46 0.029305 0.0 -inf inf 1.0
6249997 i49 j49 k49 l47 0.440716 0.0 -inf inf 1.0
6249998 i49 j49 k49 l48 0.432931 0.0 -inf inf 1.0
6249999 i49 j49 k49 l49 0.157107 0.0 -inf inf 1.0
[6250000 rows x 9 columns]
Note
In Example #1 a large 4D equation was generated – by default, only the level value of these records are randomly drawn from the interval [0,1] (uniform distribution). Other variable attributes take the default record value.

As with Variables, it is possible to generate a sparse variable with the density argument to generateRecords. We extend this example by passing our own custom func argument that will control the behavior of the value columns. The func argument accepts a dict of callables (i.e., a reference to a function).

Example #2 - Create a large (sparse) 4D equation with normally distributed values
import gams.transfer as gt
import numpy as np
# create a custom function to pass to `generateRecords`
def level_dist(size):
return np.random.normal(loc=10.0, scale=2.3, size=size)
def marginal_dist(size):
return np.random.normal(loc=0.5, scale=0.1, size=size)
m = gt.Container()
i = gt.Set(m, "i", records=[f"i{i}" for i in range(50)])
j = gt.Set(m, "j", records=[f"j{i}" for i in range(50)])
k = gt.Set(m, "k", records=[f"k{i}" for i in range(50)])
l = gt.Set(m, "l", records=[f"l{i}" for i in range(50)])
# create and define the symbol `a` with `regular` domains
a = gt.Equation(m, "a", "eq", [i, j, k, l])
# generate the records
a.generateRecords(density=0.05, func={"level":level_dist, "marginal":marginal_dist})
In [1]: a.isValid()
Out[1]: True
In [12]: a.records
Out[12]:
i j k l level marginal lower upper scale
0 i0 j0 k0 l36 11.105235 0.468989 -inf inf 1.0
1 i0 j0 k0 l40 5.697361 0.478019 -inf inf 1.0
2 i0 j0 k1 l17 11.900784 0.473814 -inf inf 1.0
3 i0 j0 k1 l24 10.105931 0.456925 -inf inf 1.0
4 i0 j0 k1 l31 8.444142 0.490966 -inf inf 1.0
... ... ... ... ... ... ... ... ... ...
312495 i49 j49 k47 l17 11.523186 0.508001 -inf inf 1.0
312496 i49 j49 k47 l20 9.341183 0.739237 -inf inf 1.0
312497 i49 j49 k47 l26 10.705808 0.581103 -inf inf 1.0
312498 i49 j49 k47 l32 7.910963 0.479655 -inf inf 1.0
312499 i49 j49 k49 l8 11.800414 0.628040 -inf inf 1.0
[312500 rows x 9 columns]
In [3]: a.records["level"].mean()
Out[3]: 10.004072307451391
In [4]: a.records["level"].std()
Out[4]: 2.292569938350144
In [5]: a.records["marginal"].mean()
Out[5]: 0.49970172269778
In [6]: a.records["marginal"].std()
Out[6]: 0.09998772109802055
Note
The custom callable function reference must expose a size argument. It might be tedious to know the exact number of the records that will be generated, especially if a fractional density is specified; therefore, the generateRecords method will pass in the correct size automatically. Users are encouraged to use the Numpy suite of random distributions when generating samples – custom functions have the potential to be computationally burdensome if a symbol has a large number of records.

Alias

There are two different ways to create a GAMS alias and add it to a Container.

  1. Use Alias constructor
  2. Use the Container method addAlias (which internally calls the Alias constructor)

Constructor

Constructor Arguments
Argument Type Description Required Default
alias_with Set object set object from which to create an alias Yes -
container Container A reference to the Container object that the symbol is being added to Yes -
name str Name of symbol Yes -
Note
The Alias property alias_with can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container.
Example - Creating an alias from a set

transfer only stores the reference to the parent set as part of the alias structure – most properties that are called from an alias object simply point to the properties of the parent set (with the exception of container, name, and alias_with). It is possible to create an alias from another alias object. In this case a recursive search will be performed to find the root parent set – this is the set that will ultimately be stored as the alias_with property. We can see this behavior in the following example:

import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=["i" + str(i) for i in range(5)])
ip = gt.Alias(m, "ip", i)
ipp = gt.Alias(m, "ipp", ip)
In [1]: ip.alias_with.name
Out[1]: 'i'
In [2]: ipp.alias_with.name
Out[2]: 'i'

Properties

Property Description Type Special Setter Behavior
alias_with aliased object Set -
container reference to the Container that the symbol belongs to Container -
description description of symbol str -
dimension dimension of symbol int setting is a shorthand notation to create ["*"] * n domains in symbol
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list, str, or Set/Alias -
domain_forwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. bool or list no effect after records have been set
domain_labels column headings for the records DataFrame list of str will add a _<dimension> tag to user supplied column names (if not unique)
domain_names string version of domain names list of str -
domain_type none, relaxed or regular depending on state of domain links str -
is_singleton if symbol is a singleton set bool -
modified Flag that identifies if the Alias has been modified bool -
name name of symbol str sets the GAMS name of the symbol
number_records number of symbol records (i.e., returns len(self.records) if not None) int -
records the main symbol records pandas.DataFrame responsive to domain_forwarding state
summary output a dict of only the metadata dict -

Methods

Method Description Arguments/Defaults Returns
addUELs adds UELs to the parent set dimensions. If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
None
equals Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_element_text=True then check that all set elements have the same descriptive element text, otherwise skip. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. If verbose=True will return an exception from the asserter describing the nature of the difference. check_uels=True (bool)
check_element_text=True (bool)
check_meta_data=True (bool)
verbose=False (bool)
bool
capitalizeUELs will capitalize all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
casefoldUELs will casefold all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
countDomainViolations returns the count of how many records in the parent set contain at least one domain violation - int
countDuplicateRecords returns the count of how many (case insensitive) duplicate records exist in the parent set - int
dropDomainViolations drop records from the parent set that contain a domain violation - None
dropDuplicateRecords drop records with (case insensitive) duplicate domains from the parent set – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) keep="first" (str, False) None
findDomainViolations get a view of the records DataFrame that contain any domain violations - pandas.DataFrame
findDuplicateRecords get a view of the records DataFrame from the parent set that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) keep="first" (str, False) pandas.DataFrame
getDomainViolations returns a list of DomainViolation objects if any (None otherwise) - list or None
getSparsity get the sparsity of the symbol w.r.t the cardinality - float
getUELs gets UELs from the parent set dimensions. If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int; must specify a single dimension if passing codes. Returns only UELs in the data if ignore_unused=True, otherwise return all UELs. dimensions=None (int, list, None)
codes=None (int, list, None)
ignore_unused=False (bool)
list
hasDomainViolations returns True if there are domain violations in the records of the parent set, returns False if not. - bool
hasDuplicateRecords returns True if there are (case insensitive) duplicate records in the parent set, returns False if not. - bool
isValid checks if the symbol is in a valid format, throw exceptions if verbose=True, re-check a symbol if force=True verbose=False
force=True
bool
ljustUELs will left justify all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
lowerUELs will lowercase all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
lstripUELs will left strip whitespace from all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
pivot Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1]. If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value index=None (str, list, None)
columns=None (str, list, None)
fill_value=None (int, float, str)
pd.DataFrame
removeUELs removes UELs that appear in the parent set dimensions, If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. uels=None (str, list, None)
dimensions=None (int, list, None)
bool
renameUELs renames UELs (case-sensitive) that appear in the parent set dimensions. If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True, the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** uels (str, list, dict)
dimensions (int, list, None)
allow_merge=False (bool)
None
reorderUELs reorders the UELs in the parent set dimensions. If uels is None, reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the parent set. uels (str, list, dict, None)
dimensions (int, list, None)
None
rjustUELs will right justify all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations length (int)
fill_character=None - if None, assumes " "
dimensions=None (int, list, or None) - if None, assumes all symbols
self
rstripUELs will right strip whitespace from all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
setRecords main convenience method to set standard pandas.DataFrame formatted records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). records (many types) None
setUELs set the UELs for parent set dimensions. If dimensions is None then set UELs for all dimensions. If rename=True, then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** uels (str, list)
dimensions=None (int, list, None)
rename=False (bool)
None
stripUELs will strip whitespace from all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
titleUELs will title (capitalize all individual words) in all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self
toList convenience method to return symbol records as a Python list include_element_text=False (bool) list
upperUELs will uppercase all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations dimensions=None (int, list, or None) - if None, assumes all symbols self

Adding Alias Records

The linked structure of Aliases offers some unique opportunities to access some of the setter functionality of the parent set. Specifically, transfer allows the user to change the domain, description, dimension, and records of the underlying parent set as a shorthand notation. We can see this behavior if we look at a modified Example #1 from Adding Set Records.

Example - Creating set records through an alias link
import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i")
ip = gt.Alias(m, "ip",i)
ip.description = "adding new descriptive set text"
ip.domain = ["*", "*"]
ip.setRecords([("i" + str(i), "j" + str(j)) for i in range(3) for j in range(3)])
In [1]: i.description
Out[1]: 'adding new descriptive set text'
In [2]: i.domain
Out[2]: ['*', '*']
In [3]: i.records
Out[3]:
uni_0 uni_1 element_text
0 i0 j0
1 i0 j1
2 i0 j2
3 i1 j0
4 i1 j1
5 i1 j2
6 i2 j0
7 i2 j1
8 i2 j2
Note
An alias .isValid()=True when the underlying parent set is also valid – if the parent set is removed from the Container the alias will no longer be valid.

UniverseAlias

There are two different ways to create a GAMS UniverseAlias (an alias to the universe) and add it to a Container.

  1. Use UniverseAlias constructor
  2. Use the Container method addUniverseAlias (which internally calls the UniverseAlias constructor)

Constructor

Constructor Arguments
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name str Name of symbol Yes -
Example - Creating an alias to the universe

In GAMS it is possible to create aliases to the universe (i.e., the entire list of UELs) with the syntax:

set i / i1, i2 /;
alias(h,*);
set j / j1, j2 /;

In this small example, h would be associated with all four UELs (i1, i2, j1 and j2) even though set j was defined after the alias declaration. transfer mimics this behavior with the UniverseAlias class. Internally, the records attribute will always call the <Container>.getUELs() and build the Pandas DataFrame on the fly. The UniverseAlias class is fundamentally different from the Alias class because it does not point to a parent set at all; it is not possible to perform operations (like setRecords or findDomainViolations) on the parent set through a UniverseAlias (because there is no parent set). This means that a UniverseAlias can be created by only defining the symbol name. We can see this behavior in the following example:

import gams.transfer as gt
m = gt.Container()
i = gt.Set(m, "i", records=["i1", "i2"])
h = gt.UniverseAlias(m, "h")
j = gt.Set(m, "j", records=["j1", "j2"])
# -- alternative syntax --
# m = gt.Container()
# m.addSet("i", records=["i1", "i2"])
# m.addUniverseAlias("h")
# m.addSet("j", records=["j1", "j2"])
In [1]: m.data
Out[1]:
{'i': <Set `i` (0x7f9598a3bd90)>,
'h': <UniverseAlias `h` (0x7f9598a61690)>,
'j': <Set `j` (0x7f95b9359cc0)>}
In [2]: h.records
Out[2]:
uni
0 i1
1 i2
2 j1
3 j2
Note
Unlike other sets, the universe does not hold on to set element_text, thus the returned DataFrame for the UniverseAlias will only have 1 column.

Properties

Property Description Type Special Setter Behavior
alias_with always * str -
container reference to the Container that the symbol belongs to Container -
description always Aliased with * str -
dimension always 1 int -
domain always ["*"] list of str -
domain_labels always ["*"] list of str -
domain_names always ["*"] list of str -
domain_type always none str -
is_singleton always False bool -
modified flag that identifies if the UniverseAlias has been modified bool -
name name of symbol str sets the GAMS name of the symbol
number_records number of symbol records (i.e., returns len(records) if not None) int -
records the main symbol records pandas.DataFrame -
summary output a dict of only the metadata dict -

Methods

Method Description Arguments/Defaults Returns
equals Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_element_text=True then check that all set elements have the same descriptive element text, otherwise skip. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. If verbose=True will return an exception from the asserter describing the nature of the difference. check_uels=True (bool)
check_element_text=True (bool)
check_meta_data=True (bool)
verbose=False (bool)
bool
getSparsity always 0.0 - float
getUELs gets UELs from the Container. Returns only UELs in the data if ignore_unused=True, otherwise return all UELs. ignore_unused=False (bool) list
isValid checks if the symbol is in a valid format, throw exceptions if verbose=True, re-check a symbol if force=True verbose=False
force=True
bool
pivot Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1]. If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value index=None (str, list, None)
columns=None (str, list, None)
fill_value=None (int, float, str)
pd.DataFrame
toList convenience method to return symbol records as a Python list - list

DomainViolation

DomainViolation objects are convenient containers that store information about the location of domain violations in a symbol. These objects are computed dynamically with the getDomainViolations method and should not be instantiated by the user (they are read-only, to the extent that this is possible in Python). However, the user may be interested in some of the information that they contain.

Constructor

Constructor Arguments/Properties
Argument Type Description Required Default
symbol _Symbol A reference to the _Symbol object that has a domain violation Yes -
dimension int An index to the dimension of the symbol where the domain violation exists Yes -
domain Set, Alias or UniverseAlias A reference to the symbol domain that is the source of the domain violation Yes -
violations list A list of all the domain elements that are causing violations Yes -