Table of Contents
Container
The main object class within transfer
is called Container
. The Container
is the vessel that allows symbols to be linked together (through their domain definitions), it enables implicit set definitions, it enables structural manipulations of the data (matrix generation), and it allows the user to perform different read/write operations.
Constructor
- Constructor Arguments
Argument | Type | Description | Required | Default |
---|---|---|---|---|
load_from | str or PathLike object, GMD Object Handle, GamsDatabase object, Container | Points to the source of the data being read into the Container | No | None |
system_directory | str | Absolute path to GAMS system directory | No | Attempts to find the GAMS installation by creating a GamsWorkspace object and loading the system_directory attribute. |
Creating a Container
is a simple matter of initializing an object. For example:
This new Container
object, here called m
, contains a number of convenient properties and methods that allow the user to interact with the symbols that are in the Container
. Some of these methods are used to filter out different types of symbols, other methods are used to numerically characterize the data within each symbol.
Properties
Property | Description | Type | Special Setter Behavior |
---|---|---|---|
data | main dictionary that is used to store all symbol data (case preserving) | CasePreservingDict | - |
modified | Flag that identifies if the Container has been modified in some way. Container.modifed=False will reset this flag for all symbols in the container as well as the container itself. | bool | - |
summary | returns a brief summary of the Container | dict | - |
Symbols are organized in the Container
under the data
Container
attribute. The dot notation (m.data
) is used to access the underlying dictionary. Symbols in this dictionary can then be retrieved with the standard bracket notation (m.data[<symbol_name>]
). The Container
is also subscriptable (i.e., m["i"]
will return the i
Set object just as if the user called m.data["i"]
). The behavior of the data
dictionary is has been customized to be case-insensitive (which mimics the behavior of GAMS) – m["i"]
and m["I"]
will return the same object.
Symbol existence in the Container
can be tested with an overloaded Python in
operator. The following (case-insensitive) syntax is possible:
- Note
- The final example assumes the existence of a separate symbol object called
i
.
The Container
can also iterate through the symbols (will return a tuple of (symbol_name, symbol_object)
using a for-loop
as in the following example. The list
and dict
methods can be useful in creating lists and dictionaries of all symbols in the Container
.
- Note
- The
len
(length) function can be used to quickly find out how many symbols exist in theContainer
(len(m)
).
Methods
Method | Description | Arguments/Defaults | Returns |
---|---|---|---|
addAlias | Container method to add an Alias | name (str ) alias_with (Set , Alias ) | Alias object |
addEquation | Container method to add an Equation | name (str ) type (str ) domain=[] (str , list ) records=None (pandas.DataFrame , numpy.ndarry , None ) domain_forwarding=False (bool ) description="" (str ) uels_on_axes=False (bool ) | Equation object |
addParameter | Container method to add a Parameter | name (str ) domain=None (str , list , None ) records=None (pandas.DataFrame , numpy.ndarry , None ) domain_forwarding=False (bool ) description="" (str ) uels_on_axes=False (bool ) | Parameter object |
addSet | Container method to add a Set | name (str ) domain=None (str , list , None ) is_singleton=False (bool ) records=None (pandas.DataFrame , numpy.ndarry , None ) domain_forwarding=False (bool ) description="" (str ) uels_on_axes=False (bool ) | Set object |
addUniverseAlias | Container method to add a UniverseAlias | name (str ) | UniverseAlias object |
addVariable | Container method to add an Variable | name (str ) type="free" (str ) domain=[] (str , list ) records=None (pandas.DataFrame , numpy.ndarry , None ) domain_forwarding=False (bool ) description="" (str ) uels_on_axes=False (bool ) | Variable object |
capitalizeUELs | will capitalize all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
casefoldUELs | will casefold all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
countDomainViolations | get the count of how many records contain at least one domain violation for symbols (if symbols=None assume all symbols) | symbols=None (str , list , None ) | dict |
countDuplicateRecords | returns the count of how many duplicate records exist for symbols (if symbols=None assume all symbols) | symbols=None (str , list , None ) | dict |
describeAliases | create a summary table with descriptive statistics for Aliases | symbols=None (None , str , list ) - if None , assumes all aliases | pandas.DataFrame |
describEquations | create a summary table with descriptive statistics for Equations | symbols=None (None , str , list ) - if None , assumes all equations | pandas.DataFrame |
describeParameters | create a summary table with descriptive statistics for Parameters | symbols=None (None , str , list ) - if None , assumes all parameters | pandas.DataFrame |
describeSets | create a summary table with descriptive statistics for Sets | symbols=None (None , str , list ) - if None , assumes all sets | pandas.DataFrame |
describeVariables | create a summary table with descriptive statistics for Variables | symbols=None (None , str , list ) - if None , assumes all variables | pandas.DataFrame |
dropDomainViolations | drop records that have domain violations for symbols (if symbols=None assume all symbols) | symbols=None (str , list , None ) | None |
dropDuplicateRecords | drop records with duplicate domains from symbols in the Container – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) | symbols=None (str , list , None ) - if None , assumes all symbols keep="first" (str, False ) | None |
getAliases | return all alias objects in the container (is_valid=None ), return all valid alias objects (is_valid=True ), return all invalid alias objects (is_valid=False ) in the container | is_valid=None (bool , None ) | list |
getDomainViolations | gets domain violations that exist in the data for symbols (if symbols=None assume all symbols); returns a list of DomainViolation objects (or None if no violations) | symbols=None (str , list , None ) | list or None |
getEquations | return all equation objects (is_valid=None ), return all valid equation objects (is_valid=True ), return all invalid equation objects (is_valid=False ) in the container | is_valid=None (bool , None ) types=None (list of equation types) - if None , assumes all types | list |
getParameters | return all parameter objects in the container (is_valid=None ), return all valid parameter objects (is_valid=True ), return all invalid parameter objects (is_valid=False ) in the container | is_valid=None (bool , None ) | list |
getSets | return all set objects in the container (is_valid=None ), return all valid set objects (is_valid=True ), return all invalid set objects (is_valid=False ) in the container | is_valid=None (bool , None ) | list |
getSymbols | returns a list of object references for symbols | symbols (str , list ) | list |
getUELs | gets UELs from all symbols . If symbols=None and ignore_unused=False , return the full universe set. If symbols=None and ignore_unused=True , return a universe set that contains UELs that only appear in data. | symbols=None (str , list , None ) ignore_unused=False (bool ) | list |
getVariables | return all variable objects (is_valid=None ), return all valid variable objects (is_valid=True ), return all invalid variable objects (is_valid=False ) in the container | is_valid=None (bool , None ) types=None (list of variable types) - if None , assumes all types | list |
hasDomainViolations | returns True if there are domain violations in the records for symbols (if symbols=None assume all symbols), returns False if not. | symbols=None (str , list , None ) | bool |
hasDuplicateRecords | returns True if there are duplicate records for symbols (if symbols=None assume all symbols), False if not. | symbols=None (str , list , None ) | bool |
isValid | True if symbols in the Container are valid (if symbols=None assume all symbols) | symbols=None (str , list , None ) | bool |
ljustUELs | will left justify all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " symbols=None (int , list , or None ) - if None , assumes all symbols | self |
listAliases | list all aliases (is_valid=None ), list all valid aliases (is_valid=True ), list all invalid aliases (is_valid=False ) in the container | is_valid=None (bool , None ) | list |
listEquations | list all equations (is_valid=None ), list all valid equations (is_valid=True ), list all invalid equations (is_valid=False ) in the container | is_valid=None (bool , None ) types=None (list of equation types) - if None , assumes all types | list |
listParameters | list all parameters (is_valid=None ), list all valid parameters (is_valid=True ), list all invalid parameters (is_valid=False ) in the container | is_valid=None (bool , None ) | list |
listSets | list all sets (is_valid=None ), list all valid sets (is_valid=True ), list all invalid sets (is_valid=False ) in the container | is_valid=None (bool , None ) | list |
listSymbols | list all symbols (is_valid=None ), list all valid symbols (is_valid=True ), list all invalid symbols (is_valid=False ) in the container | is_valid=None (bool , None ) | list |
listVariables | list all variables (is_valid=None ), list all valid variables (is_valid=True ), list all invalid variables (is_valid=False ) in the container | is_valid=None (bool , None ) types=None (list of variable types) - if None , assumes all types | list |
lowerUELs | will lowercase all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
lstripUELs | will left strip whitespace from all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
read | main method to read load_from , can be provided with a list of symbols to read in subsets, records controls if symbol records are loaded or just metadata, mode controls if data is first read into categorical data structures or object type structures, encoding specifies the original file encoding and to properly decode special characters | load_from (str or PathLike object, GMD Object Handle, GamsDatabase object, Container ) symbols="all" (str , list ) records=True (bool ) mode="category" (category or string ) encoding=None (str or None ) | None |
removeSymbols | symbols to remove from the Container, also sets the symbols container to None. If symbols=None , will remove all symbols. | symbols=None (str , list , None ) | None |
removeUELs | removes UELs from all symbols in all dimensions. If uels is None only unused UELs will be removed. If symbols is None UELs will be removed from all symbols. | uels (str , list , None ) symbols=None (str , list , None ) | None |
renameSymbol | rename a symbol in the Container | old_name (str ), new_name (str ) | None |
renameUELs | renames UELs (case-sensitive) that appear in symbols (for all dimensions). If symbols=None , rename UELs in all symbols. If allow_merge=True , the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** | uels (dict ) symbols=None (str , list , None ) allow_merge=False (bool ) | None |
reorderSymbols | reorder symbols in order to avoid domain violations | - | None |
rjustUELs | will right justify all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " symbols=None (int , list , or None ) - if None , assumes all symbols | self |
rstripUELs | will right strip whitespace from all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
stripUELs | will strip whitespace from all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
titleUELs | will title (capitalize all individual words) in all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
upperUELs | will uppercase all UELs in the Container or a subset of specified symbols , can be chained with other *UELs string operations | symbols=None (str , list , None ) - if None , assumes all symbols | self |
write | main bulk write method to a write_to target | write_to (str , PathLike object, GamsDatabase object, GMD Object) symbols=None (None , str , list ) - if None , assumes all symbols compress=False (bool ) uel_priority=None (str , list ) merge_symbols=None (None , str , list ) mode=None (None , string , category ) - if None , assumes string mode writing eps_to_zero=True (bool ) - default behavior will convert all -0.0 (EPS ) values into 0.0 (drop the sign bit) | None |
Set
There are two different ways to create a GAMS set and add it to a Container
.
- Use
Set
constructor - Use the
Container
methodaddSet
(which internally calls theSet
constructor)
Constructor
Argument | Type | Description | Required | Default |
---|---|---|---|---|
container | Container | A reference to the Container object that the symbol is being added to | Yes | - |
description | str | Description of symbol | No | "" |
domain | list , str , or Set/Alias | List of domains given either as string ('*' for universe set) or as reference to a Set/Alias object | No | ["*"] |
domain_forwarding | bool or list | Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | No | False |
is_singleton | bool | Indicates if set is a singleton set (True ) or not (False ) | No | False |
name | str | Name of symbol | Yes | - |
records | many | Symbol records | No | None |
uels_on_axes | bool | Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object | No | False |
- Note
- Set records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, and has the same
is_singleton
anddomain_forwarding
state. The symbol description will only be updated if new text is provided.
Properties
Property | Description | Type | Special Setter Behavior |
---|---|---|---|
container | reference to the Container that the symbol belongs to | Container | - |
description | description of symbol | str | - |
dimension | dimension of symbol | int | setting is a shorthand notation to create ["*"] * n domains in symbol |
domain | list of domains given either as string (* for universe set) or as reference to the Set/Alias object | list , str , or Set/Alias | - |
domain_forwarding | flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | bool or list | no effect after records have been set |
domain_labels | column headings for the records DataFrame | list of str | will add a _<dimension> tag to user supplied column names (if not unique) |
domain_names | string version of domain names | list of str | - |
domain_type | none , relaxed or regular depending on state of domain links | str | - |
is_singleton | bool if symbol is a singleton set | bool | - |
modified | Flag that identifies if the Set has been modified | bool | - |
name | name of symbol | str | sets the GAMS name of the symbol |
number_records | number of symbol records (i.e., returns len(self.records) if not None ) | int | - |
records | the main symbol records | pandas.DataFrame | responsive to domain_forwarding state |
summary | output a dict of only the metadata | dict | - |
Methods
Method | Description | Arguments/Defaults | Returns |
---|---|---|---|
addUELs | adds UELs to the symbol dimensions . If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) | None |
capitalizeUELs | will capitalize all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
casefoldUELs | will casefold all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
countDomainViolations | returns the count of how many records contain at least one domain violation | - | int |
countDuplicateRecords | returns the count of how many (case insensitive) duplicate records exist | - | int |
dropDomainViolations | drop records from the symbol that contain a domain violation | - | None |
dropDuplicateRecords | drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) | keep="first" (str, False ) | None |
equals | Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_element_text=True then check that all set elements have the same descriptive element text, otherwise skip. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. If verbose=True will return an exception from the asserter describing the nature of the difference. | check_uels=True (bool ) check_element_text=True (bool ) check_meta_data=True (bool ) verbose=False (bool ) | bool |
findDomainViolations | get a view of the records DataFrame that contain any domain violations | - | pandas.DataFrame |
findDuplicateRecords | get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) | keep="first" (str, False ) | pandas.DataFrame |
generateRecords | convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1] . If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. | density=1.0 (float , list ) seed=None (int , None ) | None |
getDomainViolations | returns a list of DomainViolation objects if any (None otherwise) | - | list or None |
getSparsity | get the sparsity of the symbol w.r.t the cardinality | - | float |
getUELs | gets UELs from symbol dimensions . If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int ; must specify a single dimension if passing codes . Returns only UELs in the data if ignore_unused=True , otherwise return all UELs. | dimensions=None (int , list , None ) codes=None (int , list , None ) ignore_unused=False (bool ) | list |
hasDomainViolations | returns True if there are domain violations in the records, returns False if not. | - | bool |
hasDuplicateRecords | returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. | - | bool |
isValid | checks if the symbol is in a valid format, throw exceptions if verbose=True , recheck a symbol if force=True | verbose=False force=True | bool |
ljustUELs | will left justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lowerUELs | will lowercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lstripUELs | will left strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
pivot | Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1] . If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value | index=None (str , list , None ) columns=None (str , list , None ) fill_value=None (int , float , str ) | pd.DataFrame |
renameUELs | renames UELs (case-sensitive) that appear in the symbol dimensions . If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True , the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** | uels (str , list , dict ) dimensions (int , list , None ) allow_merge=False (bool ) | None |
reorderUELs | reorders the UELs in the symbol dimensions . If uels is None , reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. | uels (str , list , dict , None ) dimensions (int , list , None ) | None |
removeUELs | removes UELs that appear in the symbol dimensions , If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. | uels=None (str , list , None ) dimensions=None (int , list , None ) | bool |
rjustUELs | will right justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
rstripUELs | will right strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
setRecords | main convenience method to set standard pandas.DataFrame formatted records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). | records (many types) | None |
setUELs | set the UELs for symbol dimensions . If dimensions is None then set UELs for all dimensions. If rename=True , then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) rename=False (bool ) | None |
stripUELs | will strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
titleUELs | will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
toList | convenience method to return symbol records as a Python list | include_element_text=False (bool) | list |
upperUELs | will uppercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
Adding Set Records
Three possibilities exist to assign symbol records to a set (roughly ordered in complexity):
- Setting the argument
records
in the set constructor/container method (internally callssetRecords
) - creates a data copy - Using the symbol method
setRecords
- creates a data copy - Setting the property
records
directly - does not create a data copy
If the data is in a convenient format, a user may want to pass the records directly within the set constructor. This is an optional keyword argument and internally the set constructor will simply call the setRecords
method. The symbol method setRecords
is a convenience method that transforms the given data into an approved Pandas DataFrame format (see Standard Data Formats). Many native Python data types can be easily transformed into DataFrames, so the setRecords
method for Set
objects will accept a number of different types for input. The setRecords
method is called internally on any data structure that is passed through the records
argument. We show a few examples of ways to create differently structured sets:
- Example #1 - Create a 1D set from a list
- Example #2 - Create a 1D set from a tuple
- Example #3 - Create a 2D set from a list of tuples
- Example #4 - Create a 1D set from a DataFrame slice + .unique()
- Note
- The
.unique()
method preserves the order of appearance, unlikeset()
.
Set element text is very handy when labeling specific set elements within a set. A user can add a set element text directly with a set element. Note that it is not required to label all set elements, as can be seen in the following example.
- Example #5 - Add set element text
- Example #6 - Create a 1D set from a Pandas Series
- Example #7 - Create a 2D set from a Pandas Series
- Attention
- The order of the set element could be surprising depending on how the pandas MultiIndex is created. Users should take care that the order of the elements returned from the
getUELs
method is correct.
- Example #8 - Create a 2D set from a Pandas DataFrame with uels_on_axes=True
- Note
- It not possible to set
element_text
whenuels_on_axes=True
Directly Set Records
The primary advantage of the setRecords
method is that transfer
will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container
a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.
- Example #1 - Directly set records (1D set)
Stepping through this example we take the following steps:
- Create an empty
Container
- Create a GAMS set
i
in the Container, but do not set therecords
- Create a Pandas DataFrame (manually, in this example) taking care to follow the standard format
- The DataFrame has the right shape and column labels so we can proceed to set the records.
- We need to cast the
uni
column as acategorical
data type, so we create a custom ordered categorty type usingpandas.CategoricalDtype
- Finally, we set the records directly by passing a reference to
df_i
into the symbol records attribute. The setter function of.records
checks that a DataFrame is being set, but does not check validity. Thus, as a final step we call the.isValid()
method to verify that the symbol is valid.
- Attention
- Users can debug their DataFrames by running
<symbol_name>.isValid(verbose=True)
to get feedback about their data.
- Example #2 - Directly set records (1D subset)
This example is more subtle in that we want to create a set j
that is a subset of i
. We create the set i
using the setRecords
method but then set the records directly for j
. There are two important details to note: 1) the column labels in df_j
now reflect the standard format for a symbol with a domain set (as opposed to the universe) and 2) we create the categorical dtype by referencing the parent set (i
) for the categories (instead of referencing itself).
Generate Set Records
Generating the initial pandas.DataFrame
object could be difficult for Set
symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords
convenience function. Internally, generateRecords
computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords
will only work on symbols where <symbol>.domain_type == "regular"
).
- Example #1 - Create a large (dense) 4D set
It is also possible to generate a sparse set (randomly selected rows are removed from the dense dataframe) with the density
argument to generateRecords
.
- Example #2 - Create a large (sparse) 4D set
- Example #3 - Create a large 4D set w/ only 1 sparse dimension
Parameter
There are two different ways to create a GAMS parameter and add it to a Container
.
- Use
Parameter
constructor - Use the
Container
methodaddParameter
(which internally calls theParameter
constructor)
Constructor
- Constructor Arguments
Argument | Type | Description | Required | Default |
---|---|---|---|---|
container | Container | A reference to the Container object that the symbol is being added to | Yes | - |
description | str | Description of symbol | No | "" |
domain | list , str , or Set/Alias | List of domains given either as string ('*' for universe set) or as reference to a Set/Alias object, an empty domain list will create a scalar parameter | No | [] |
domain_forwarding | bool or list | Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | No | False |
name | str | Name of symbol | Yes | - |
records | many | Symbol records | No | None |
uels_on_axes | bool | Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object | No | False |
- Note
- Parameter records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, and has the same
domain_forwarding
state. The symbol description will only be updated if new text is provided.
Properties
Property | Description | Type | Special Setter Behavior |
---|---|---|---|
container | reference to the Container that the symbol belongs to | Container | - |
description | description of symbol | str | - |
dimension | dimension of symbol | int | setting is a shorthand notation to create ["*"] * n domains in symbol |
domain | list of domains given either as string (* for universe set) or as reference to the Set/Alias object | list , str , or Set/Alias | - |
domain_forwarding | flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | bool or list | no effect after records have been set |
domain_labels | column headings for the records DataFrame | list of str | will add a _<dimension> tag to user supplied column names (if not unique) |
domain_names | string version of domain names | list of str | - |
domain_type | none , relaxed or regular depending on state of domain links | str | - |
is_scalar | True if the len(self.domain) = 0 | bool | - |
modified | Flag that identifies if the Parameter has been modified | bool | - |
name | name of symbol | str | sets the GAMS name of the symbol |
number_records | number of symbol records (i.e., returns len(self.records) if not None ) | int | - |
records | the main symbol records | pandas.DataFrame | responsive to domain_forwarding state |
shape | a tuple describing the array dimensions if records were converted with .toDense() | tuple | - |
summary | output a dict of only the metadata | dict | - |
Methods
Method | Description | Arguments/Defaults | Returns |
---|---|---|---|
addUELs | adds UELs to the symbol dimensions . If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) | None |
capitalizeUELs | will capitalize all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
casefoldUELs | will casefold all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
countDomainViolations | returns the count of how many records contain at least one domain violation | - | int |
countDuplicateRecords | returns the count of how many (case insensitive) duplicate records exist | - | int |
countEps | total number of SpecialValues.EPS in value column | - | int or None |
countNA | total number of SpecialValues.NA in value column | - | int or None |
countNegInf | total number of SpecialValues.NEGINF in value column | - | int or None |
countPosInf | total number of SpecialValues.POSINF in value column | - | int or None |
countUndef | total number of SpecialValues.UNDEF in value column | - | int or None |
dropDefaults | an alias to dropZeros | - | None |
dropDomainViolations | drop records from the symbol that contain a domain violation | - | None |
dropDuplicateRecords | drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) | keep="first" (str, False ) | None |
dropEps | drop records from the symbol that are GAMS EPS (zero 0.0 records will be retained) | - | None |
dropMissing | drop records from the symbol that are NaN (includes both NA and Undef special values) | - | None |
dropNA | drop records from the symbol that are GAMS NA | - | None |
dropUndef | drop records from the symbol that are GAMS Undef | - | None |
dropZeros | drop records from the symbol that are zero (GAMS EPS (-0.0 ) will not be dropped) | - | None |
equals | Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. rtol (relative tolerance) and atol (absolute tolerance) set equality tolerances. If verbose=True will return an exception from the asserter describing the nature of the difference. | check_uels=True (bool ) check_meta_data=True (bool ) rtol=0.0 (float , None ) atol=0.0 (float , None ) verbose=False (bool ) | bool |
findDomainViolations | get a view of the records DataFrame that contain any domain violations | - | pandas.DataFrame |
findDuplicateRecords | get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) | keep="first" (str, False ) | pandas.DataFrame |
findEps | find positions of SpecialValues.EPS in value column | - | pandas.DataFrame or None |
findNA | find positions of SpecialValues.NA in value column | - | pandas.DataFrame or None |
findNegInf | find positions of SpecialValues.NEGINF in value column | - | pandas.DataFrame or None |
findPosInf | find positions of SpecialValues.POSINF in value column | - | pandas.DataFrame or None |
findUndef | find positions of SpecialValues.Undef in value column | - | pandas.DataFrame or None |
generateRecords | convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1] . If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. | density=1.0 (float , list ) func=numpy.random.uniform(0,1) (callable) seed=None (int , None ) | None |
getMaxAbsValue | get the maximum absolute value in value column | - | float or None |
getMaxValue | get the maximum value in value column | - | float or None |
getMeanValue | get the mean value in value column | - | float or None |
getMinValue | get the minimum value in value column | - | float or None |
getSparsity | get the sparsity of the symbol w.r.t the cardinality | - | float |
getUELs | gets UELs from symbol dimensions . If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int ; must specify a single dimension if passing codes . Returns only UELs in the data if ignore_unused=True , otherwise return all UELs. | dimensions=None (int , list , None ) codes=None (int , list , None ) ignore_unused=False (bool ) | list |
hasDomainViolations | returns True if there are domain violations in the records, returns False if not. | - | bool |
hasDuplicateRecords | returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. | - | bool |
isValid | checks if the symbol is in a valid format, throw exceptions if verbose=True , recheck a symbol if force=True | verbose=False force=True | bool |
ljustUELs | will left justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lowerUELs | will lowercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lstripUELs | will left strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
pivot | Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1] . If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value | index=None (str , list , None ) columns=None (str , list , None ) fill_value=None (int , float , str ) | pd.DataFrame |
removeUELs | removes UELs that appear in the symbol dimensions , If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. | uels=None (str , list , None ) dimensions=None (int , list , None ) | bool |
renameUELs | renames UELs (case-sensitive) that appear in the symbol dimensions . If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True , the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** | uels (str , list , dict ) dimensions (int , list , None ) allow_merge=False (bool ) | None |
reorderUELs | reorders the UELs in the symbol dimensions . If uels is None , reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. | uels (str , list , dict , None ) dimensions (int , list , None ) | None |
rjustUELs | will right justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
rstripUELs | will right strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
setRecords | main convenience method to set standard pandas.DataFrame records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). | records (many types). | None |
setUELs | set the UELs for symbol dimensions . If dimensions is None then set UELs for all dimensions. If rename=True , then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) rename=False (bool ) | None |
stripUELs | will strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
titleUELs | will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
toDense | convert symbol to a dense numpy.array format | - | numpy.array or None |
toDict | convenience method to return symbol records as a Python dictionary, orient can take values natural or columns and will control the shape of the dict . Must use orient="columns" if attempting to set symbol records with setRecords | orient="natural" | dict |
toList | convenience method to return symbol records as a Python list | - | list |
toSparseCoo | convert symbol to a sparse COOrdinate numpy.array format | - | sparse matrix format or None |
toValue | convenience method to return symbol records as a Python float. Only possible with scalar symbols. | - | float |
upperUELs | will uppercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
whereMax | find the domain entry of records with a maximum value (return first instance only) | - | list of str or None |
whereMaxAbs | find the domain entry of records with a maximum absolute value (return first instance only) | - | list of str or None |
whereMin | find the domain entry of records with a minimum value (return first instance only) | - | list of str or None |
Adding Parameter Records
Three possibilities exist to assign symbol records to a parameter (roughly ordered in complexity):
- Setting the argument
records
in the set constructor/container method (internally callssetRecords
) - creates a data copy - Using the symbol method
setRecords
- creates a data copy - Setting the property
records
directly - does not create a data copy
If the data is in a convenient format, a user may want to pass the records directly within the parameter constructor. This is an optional keyword argument and internally the parameter constructor will simply call the setRecords
method. The symbol method setRecords
is a convenience method that transforms the given data into an approved Pandas DataFrame format (see Standard Data Formats). Many native Python data types can be easily transformed into DataFrames, so the setRecords
method for Set
objects will accept a number of different types for input. The setRecords
method is called internally on any data structure that is passed through the records
argument. We show a few examples of ways to create differently structured parameters:
- Example #1 - Create a GAMS scalar
- Note
transfer
will still convert scalar values to a standard format (i.e., a Pandas DataFrame with a single row and column).
- Example #2 - Create a 1D parameter (defined over
*
) from a list of tuples
- Example #3 - Create a 1D parameter (defined over a set) from a list of tuples
- Example #4 - Create a 2D parameter (defined over a set) from a DataFrame slice
- Note
- The original indexing is preserved when a user slices rows out of a reference dataframe.
- Example #5 - Create a 2D parameter (defined over a set) from a matrix
- Example #6 - Create a 2D parameter from an array using setRecords
- Example #7 - Create a 1D parameter from a pandas Series
- Example #8 - Create a 2D parameter from a pandas Series
- Example #9 - Create a 2D parameter from a DataFrame (uels_on_axes=True)
Directly Set Records
As with sets, the primary advantage of the setRecords
method is that transfer
will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container
a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.
- Example #1 - Correctly set records (directly)
In this example we create a large parameter (31,536,000 records and 8880 unique domain elements – we mimic data that is labeled for every second in one year) and assign it to a parameter with a.records
. transfer
requires that all domain columns must be a categorical data type, furthermore, this categorical must be ordered. The records
setter function does very little work other than checking if the object being set is a DataFrame. This places more responsibility on the user to create a DataFrame that complies with the standard format. In Example #1 we take care to properly reference the categorical data types from the domain sets – and in the end a.isValid() = True
.
Users will need to use the .isValid(verbose=True)
method to debug any structural issues. As an example we incorrectly generate categorical data types by passing the DataFrame constructor the generic dtype="category"
argument. This creates categorical column types but they are not ordered and they do not reference the underlying domain set. These errors result in a
being invalid.
- Example #2 - Incorrectly set records (directly)
Generate Parameter Records
Generating the initial pandas.DataFrame
object could be difficult for Parameter
symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords
convenience function. Internally, generateRecords
computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords
will only work on symbols where <symbol>.domain_type == "regular"
).
- Example #1 - Create a large (dense) 4D parameter
- Note
- In Example #1 a large 4D parameter was generated – by default, the value of these records are randomly drawn numbers from the interval
[0,1]
(uniform distribution).
As with Sets
, it is possible to generate a sparse parameter with the density
argument to generateRecords
. We extend this example by passing our own custom func
argument that will control the behavior of the value
columns. The func
argument accepts a callable
(i.e., a reference to a function).
- Example #2 - Create a large (sparse) 4D parameter with normally distributed values
- Note
- The custom
callable
function reference must expose asize
argument. It might be tedious to know the exact number of the records that will be generated, especially if a fractional density is specified; therefore, thegenerateRecords
method will pass in the correct size automatically. Users are encouraged to use the Numpy suite of random distributions when generating samples – custom functions have the potential to be computationally burdensome if a symbol has a large number of records.
- Example #3 - Create a large 4D parameter with 1 sparse dimension
- Example #4 - Create a large 4D parameter with a random number seed
- Note
- The
seed
is anint
that will set the random number generator state (enables reproducible sequences of random numbers).
Variable
There are two different ways to create a GAMS variable and add it to a Container
.
- Use
Variable
constructor - Use the
Container
methodaddVariable
(which internally calls theVariable
constructor)
Constructor
- Constructor Arguments
Argument | Type | Description | Required | Default |
---|---|---|---|---|
container | Container | A reference to the Container object that the symbol is being added to | Yes | - |
description | str | Description of symbol | No | "" |
domain | list , str , or Set/Alias | List of domains given either as string (* for universe set) or as reference to a Set/Alias object, an empty domain list will create a scalar variable | No | [] |
domain_forwarding | bool or list | Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | No | False |
name | str | Name of symbol | Yes | - |
records | many | Symbol records | No | None |
type | str | Type of variable being created [binary , integer , positive , negative , free , sos1 , sos2 , semicont , semiint ] | No | free |
uels_on_axes | bool | Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object | No | False |
- Note
- Variable records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, has the same
type
, and has the samedomain_forwarding
state. The symbol description will only be updated if new text is provided.
Properties
Property | Description | Type | Special Setter Behavior |
---|---|---|---|
container | reference to the Container that the symbol belongs to | Container | - |
description | description of symbol | str | - |
dimension | dimension of symbol | int | setting is a shorthand notation to create ["*"] * n domains in symbol |
domain | list of domains given either as string (* for universe set) or as reference to the Set/Alias object | list , str , or Set/Alias | - |
domain_forwarding | flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | bool or list | no effect after records have been set |
domain_labels | column headings for the records DataFrame | list of str | will add a _<dimension> tag to user supplied column names (if not unique) |
domain_names | string version of domain names | list of str | - |
domain_type | none , relaxed or regular depending on state of domain links | str | - |
is_scalar | True if the len(self.domain) = 0 | bool | - |
modified | Flag that identifies if the Variable has been modified | bool | - |
name | name of symbol | str | sets the GAMS name of the symbol |
number_records | number of symbol records (i.e., returns len(self.records) if not None ) | int | - |
records | the main symbol records | pandas.DataFrame | responsive to domain_forwarding state |
shape | a tuple describing the array dimensions if records were converted with .toDense() | tuple | - |
summary | output a dict of only the metadata | dict | - |
type | str type of variable | str | - |
Methods
Method | Description | Arguments/Defaults | Returns |
---|---|---|---|
addUELs | adds UELs to the symbol dimensions . If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) | None |
capitalizeUELs | will capitalize all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
casefoldUELs | will casefold all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
countDomainViolations | returns the count of how many records contain at least one domain violation | - | int |
countDuplicateRecords | returns the count of how many (case insensitive) duplicate records exist | - | int |
countEps | total number of SpecialValues.EPS across all columns | columns="level" (str , list ) | int or None |
countNA | total number of SpecialValues.NA across all columns | columns="level" (str , list ) | int or None |
countNegInf | total number of SpecialValues.NEGINF across all columns | columns="level" (str , list ) | int or None |
countPosInf | total number of SpecialValues.POSINF across all columns | columns="level" (str , list ) | int or None |
countUndef | total number of SpecialValues.UNDEF across all columns | columns="level" (str , list ) | int or None |
dropDefaults | drop records that are set to GAMS default records (check .default_records property for values) | - | None |
dropDomainViolations | drop records from the symbol that contain a domain violation | - | None |
dropDuplicateRecords | drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) | keep="first" (str, False ) | None |
dropEps | drop records from the symbol that are GAMS EPS (zero 0.0 records will be retained) | - | None |
dropMissing | drop records from the symbol that are NaN (includes both NA and Undef special values) | - | None |
dropNA | drop records from the symbol that are GAMS NA | - | None |
dropUndef | drop records from the symbol that are GAMS Undef | - | None |
equals | Used to compare the symbol to another symbol. The columns argument allows the user to numerically compare only specified variable attributes (default is to compare all). If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_meta_data=True then check that symbol name, description and variable type are the same, otherwise skip. rtol (relative tolerance) and atol (absolute tolerance) set equality tolerances; can be different tolerances for different variable attributes (if specified as a dict ). If verbose=True will return an exception from the asserter describing the nature of the difference. | columns=["level", "marginal", "lower", "upper", "scale"] check_uels=True (bool ) check_meta_data=True (bool ) rtol=0.0 (int , float , None ) atol=0.0 (int , float , None ) verbose=False (bool ) | bool |
findDomainViolations | get a view of the records DataFrame that contain any domain violations | - | pandas.DataFrame |
findDuplicateRecords | get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) | keep="first" (str, False ) | pandas.DataFrame |
findEps | find positions of SpecialValues.EPS in column | column="level" (str ) | pandas.DataFrame or None |
findNA | find positions of SpecialValues.NA in column | column="level" (str ) | pandas.DataFrame or None |
findNegInf | find positions of SpecialValues.NEGINF in column | column="level" (str ) | pandas.DataFrame or None |
findPosInf | find positions of SpecialValues.POSINF in column | column="level" (str ) | pandas.DataFrame or None |
findUndef | find positions of SpecialValues.Undef in column | column="level" (str ) | pandas.DataFrame or None |
generateRecords | convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1] . If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. | density=1.0 (float , list ) func=numpy.random.uniform(0,1) (dict of callables ) seed=None (int , None ) | None |
getMaxValue | get the maximum value across all columns | columns="level" (str , list ) | float or None |
getMinValue | get the minimum value across all columns | columns="level" (str , list ) | float or None |
getMeanValue | get the mean value across all columns | columns="level" (str , list ) | float or None |
getMaxAbsValue | get the maximum absolute value across all columns | columns="level" (str , list ) | float |
getSparsity | get the sparsity of the symbol w.r.t the cardinality | - | float |
getUELs | gets UELs from symbol dimensions . If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int ; must specify a single dimension if passing codes . Returns only UELs in the data if ignore_unused=True , otherwise return all UELs. | dimensions=None (int , list , None ) codes=None (int , list , None ) ignore_unused=False (bool ) | list |
hasDomainViolations | returns True if there are domain violations in the records, returns False if not. | - | bool |
hasDuplicateRecords | returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. | - | bool |
isValid | checks if the symbol is in a valid format, throw exceptions if verbose=True , recheck a symbol if force=True | verbose=False force=True | bool |
ljustUELs | will left justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lowerUELs | will lowercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lstripUELs | will left strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
pivot | Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1] . If columns is None then it is set to the last dimension. If value is None then the level values will be pivoted. Missing values in the pivot will take the value provided by fill_value | index=None (str , list , None ) columns=None (str , list , None ) value (str ) fill_value=None (int , float , str ) | pd.DataFrame |
removeUELs | removes UELs that appear in the symbol dimensions , If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. | uels=None (str , list , None ) dimensions=None (int , list , None ) | bool |
renameUELs | renames UELs (case-sensitive) that appear in the symbol dimensions . If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True , the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** | uels (str , list , dict ) dimensions (int , list , None ) allow_merge=False (bool ) | None |
reorderUELs | reorders the UELs in the symbol dimensions . If uels is None , reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. | uels (str , list , dict , None ) dimensions (int , list , None ) | None |
rjustUELs | will right justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
rstripUELs | will right strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
setRecords | main convenience method to set standard pandas.DataFrame records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). | records (many types) | None |
setUELs | set the UELs for symbol dimensions . If dimensions is None then set UELs for all dimensions. If rename=True , then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) rename=False (bool ) | None |
stripUELs | will strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
titleUELs | will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
toDense | convert column to a dense numpy.array format | column="level" (str ) | numpy.array or None |
toDict | convenience method to return symbol records as a Python dictionary. columns will control which attributes to include in the dict . orient can take values natural or columns and will control the shape of the dict . Must use orient="columns" if attempting to set symbol records with setRecords . | columns="level" (str) orient="natural" (str) | dict |
toList | convenience method to return symbol records as a Python list, the columns argument will control with attributes to include in the list | columns="level" (str) | list |
toSparseCoo | convert column to a sparse COOrdinate numpy.array format | column="level" (str ) | sparse matrix format or None |
toValue | convenience method to return symbol records as a Python float. Only possible with scalar symbols. Attribute can be specified with column argument. | column="level" (str) | float |
upperUELs | will uppercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
whereMax | find the domain entry of records with a maximum value (return first instance only) | column="level" (str ) | list of str or None |
whereMaxAbs | find the domain entry of records with a maximum absolute value (return first instance only) | column="level" (str ) | list of str or None |
whereMin | find the domain entry of records with a minimum value (return first instance only) | column="level" (str ) | list of str or None |
Adding Variable Records
Three possibilities exist to assign symbol records to a variable (roughly ordered in complexity):
- Setting the argument
records
in the set constructor/container method (internally callssetRecords
) - creates a data copy - Using the symbol method
setRecords
- creates a data copy - Setting the property
records
directly - does not create a data copy
If the data is in a convenient format, a user may want to pass the records directly within the variable constructor. This is an optional keyword argument and internally the variable constructor will simply call the setRecords
method. In contrast to the setRecords
methods in in either the Set or Parameter classes the setRecords
method for variables will only accept Pandas DataFrames and specially structured dict
for creating records from matrices. This restriction is out of necessity because to properly set a record for a Variable the user must pass data for the level
, marginal
, lower
, upper
and scale
attributes. That said, any missing attributes will be filled in with the GAMS default record values (see: Variable Types), default scale
value is always 1, and the default level
and marginal
values are 0 for all variable types). We show a few examples of ways to create differently structured variables:
- Example #1 - Create a GAMS scalar variable
- Example #2 - Create a 1D variable (defined over
*
) from a list of tuples
In this example we only set the marginal
values.
- Example #3 - Create a 1D variable (defined over a set) from a list of tuples
- Example #4 - Create a 2D positive variable, specifying no numerical data
- Example #5 - Create a 2D variable (defined over a set) from a matrix
- Example #6 - Create a 1D variable from a pandas Series
- Note
transfer
will assume that thelevel
value is being set if attributes cannot be found in the axes
- Example #7 - Create a 1D variable from a pandas Series (set only the marginal)
- Note
transfer
will search the axes for any variable attributes and set those attributes provided. It is necessary to include all attributes into the same axes level (if a MultiIndex) without any other UELs – if other UELs are found in the same level then all element will be interpreted as UELs and could result in GAMS domain violations.
- Example #8 - Create a 2D variable from a DataFrame (uels_on_axes=True)
Directly Set Records
As with sets, the primary advantage of the setRecords
method is that transfer
will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container
a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.
- Example #1 - Correctly set records (directly)
- Attention
- All numeric data in the records will need to be type
float
in order to maintain a valid symbol.
In this example we create a large variable (31,536,000 records and 8880 unique domain elements – we mimic data that is labeled for every second in one year) and assign it to a variable with a.records
. transfer
requires that all domain columns must be a categorical data type, furthermore this categorical must be ordered. The records
setter function does very little work other than checking if the object being set is a DataFrame. This places more responsibility on the user to create a DataFrame that complies with the standard format. In Example #1 we take care to properly reference the categorical data types from the domain sets – and in the end a.isValid() = True
. As with Set and Parameters, users can use the .isValid(verbose=True)
method to debug any structural issues.
Generate Variable Records
Generating the initial pandas.DataFrame
object could be difficult for Variable
symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords
convenience function. Internally, generateRecords
computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords
will only work on symbols where <symbol>.domain_type == "regular"
).
- Example #1 - Create a large (dense) 4D variable
- Note
- In Example #1 a large 4D variable was generated – by default, only the
level
value of these records are randomly drawn from the interval[0,1]
(uniform distribution). Other variable attributes take the default record value.
As with Parameters
, it is possible to generate a sparse variable with the density
argument to generateRecords
. We extend this example by passing our own custom func
argument that will control the behavior of the value
columns. The func
argument accepts a dict
of callables
(i.e., a reference to a function).
- Example #2 - Create a large (sparse) 4D variable with normally distributed values
- Note
- The custom
callable
function reference must expose asize
argument. It might be tedious to know the exact number of the records that will be generated, especially if a fractional density is specified; therefore, thegenerateRecords
method will pass in the correct size automatically. Users are encouraged to use the Numpy suite of random distributions when generating samples – custom functions have the potential to be computationally burdensome if a symbol has a large number of records.
Equation
There are two different ways to create a GAMS equation and add it to a Container
.
- Use
Equation
constructor - Use the
Container
methodaddEquation
(which internally calls theEquation
constructor)
Constructor
- Constructor Arguments
Argument | Type | Description | Required | Default |
---|---|---|---|---|
container | Container | A reference to the Container object that the symbol is being added to | Yes | - |
description | str | Description of symbol | No | "" |
domain | list , str , or Set/Alias | List of domains given either as string (* for universe set) or as reference to a Set/Alias object, an empty domain list will create a scalar equation | No | [] |
domain_forwarding | bool or list | Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | No | False |
name | str | Name of symbol | Yes | - |
records | many | Symbol records | No | None |
type | str | Type of equation being created [eq (or E /e ), geq (or G /g ), leq (or L /l ), nonbinding (or N /n ), external (or X /x )] | Yes | - |
uels_on_axes | bool | Instructs setRecords to assume symbol domain information is contained in the axes of the Pandas object | No | False |
- Note
- Equation records can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container, has the same domain, has the same
type
, and has the samedomain_forwarding
state. The symbol description will only be updated if new text is provided.
Properties
Property | Description | Type | Special Setter Behavior |
---|---|---|---|
container | reference to the Container that the symbol belongs to | Container | - |
description | description of symbol | str | - |
dimension | dimension of symbol | int | setting is a shorthand notation to create ["*"] * n domains in symbol |
domain | list of domains given either as string (* for universe set) or as reference to the Set/Alias object | list , str , or Set/Alias | - |
domain_forwarding | flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | bool or list | no effect after records have been set |
domain_labels | column headings for the records DataFrame | list of str | will add a _<dimension> tag to user supplied column names (if not unique) |
domain_names | string version of domain names | list of str | - |
domain_type | none , relaxed or regular depending on state of domain links | str | - |
is_scalar | True if the len(self.domain) = 0 | bool | - |
modified | Flag that identifies if the Equation has been modified | bool | - |
name | name of symbol | str | sets the GAMS name of the symbol |
number_records | number of symbol records (i.e., returns len(self.records) if not None ) | int | - |
records | the main symbol records | pandas.DataFrame | responsive to domain_forwarding state |
shape | a tuple describing the array dimensions if records were converted with .toDense() | tuple | - |
summary | output a dict of only the metadata | dict | - |
type | str type of variable | str | - |
Methods
Method | Description | Arguments/Defaults | Returns |
---|---|---|---|
addUELs | adds UELs to the symbol dimensions . If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) | None |
capitalizeUELs | will capitalize all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
casefoldUELs | will casefold all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
countDomainViolations | returns the count of how many records contain at least one domain violation | - | int |
countDuplicateRecords | returns the count of how many (case insensitive) duplicate records exist | - | int |
countEps | total number of SpecialValues.EPS across all columns | columns="level" (str , list ) | int or None |
countNA | total number of SpecialValues.NA across all columns | columns="level" (str , list ) | int or None |
countNegInf | total number of SpecialValues.NEGINF across all columns | columns="level" (str , list ) | int or None |
countPosInf | total number of SpecialValues.POSINF across all columns | columns="level" (str , list ) | int or None |
countUndef | total number of SpecialValues.UNDEF across all columns | columns="level" (str , list ) | int or None |
dropDefaults | drop records that are set to GAMS default records (check .default_records property for values) | - | None |
dropDomainViolations | drop records from the symbol that contain a domain violation | - | None |
dropDuplicateRecords | drop records with (case insensitive) duplicate domains from the symbol – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) | keep="first" (str, False ) | None |
dropEps | drop records from the symbol that are GAMS EPS (zero 0.0 records will be retained) | - | None |
dropMissing | drop records from the symbol that are NaN (includes both NA and Undef special values) | - | None |
dropNA | drop records from the symbol that are GAMS NA | - | None |
dropUndef | drop records from the symbol that are GAMS Undef | - | None |
equals | Used to compare the symbol to another symbol. The columns argument allows the user to numerically compare only specified equation attributes (default is to compare all). If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_meta_data=True then check that symbol name, description and equation type are the same, otherwise skip. rtol (relative tolerance) and atol (absolute tolerance) set equality tolerances; can be different tolerances for different equation attributes (if specified as a dict ). If verbose=True will return an exception from the asserter describing the nature of the difference. | columns=["level", "marginal", "lower", "upper", "scale"] check_uels=True (bool ) check_meta_data=True (bool ) rtol=0.0 (int , float , None ) atol=0.0 (int , float , None ) verbose=False (bool ) | bool |
findDomainViolations | get a view of the records DataFrame that contain any domain violations | - | pandas.DataFrame |
findDuplicateRecords | get a view of the records DataFrame that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) | keep="first" (str, False ) | pandas.DataFrame |
findEps | find positions of SpecialValues.EPS in column | column="level" (str ) | pandas.DataFrame or None |
findNA | find positions of SpecialValues.NA in column | column="level" (str ) | pandas.DataFrame or None |
findNegInf | find positions of SpecialValues.NEGINF in column | column="level" (str ) | pandas.DataFrame or None |
findPosInf | find positions of SpecialValues.POSINF in column | column="level" (str ) | pandas.DataFrame or None |
findUndef | find positions of SpecialValues.Undef in column | column="level" (str ) | pandas.DataFrame or None |
generateRecords | convenience method to set standard pandas.DataFrame formatted records given domain set information. Will generate records with the Cartesian product of all domain sets. The density argument can take any value on the interval [0,1] . If density is <1 then randomly selected records will be removed. `density` will accept a `list` of length `dimension` -- allows users to specify a density per symbol dimension. Random number state can be set with `seed` argument. | density=1.0 (float , list ) func=numpy.random.uniform(0,1) (dict of callables ) seed=None (int , None ) | None |
getMaxAbsValue | get the maximum absolute value across all columns | columns="level" (str , list ) | float or None |
getMaxValue | get the maximum value across all columns | columns="level" (str , list ) | float or None |
getMeanValue | get the mean value across all columns | columns="level" (str , list ) | float or None |
getMinValue | get the minimum value across all columns | columns="level" (str , list ) | float or None |
getSparsity | get the sparsity of the symbol w.r.t the cardinality | - | float |
getUELs | gets UELs from symbol dimensions . If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int ; must specify a single dimension if passing codes . Returns only UELs in the data if ignore_unused=True , otherwise return all UELs. | dimensions=None (int , list , None ) codes=None (int , list , None ) ignore_unused=False (bool ) | list |
hasDomainViolations | returns True if there are domain violations in the records, returns False if not. | - | bool |
hasDuplicateRecords | returns True if there are (case insensitive) duplicate records in the symbol, returns False if not. | - | bool |
isValid | checks if the symbol is in a valid format, throw exceptions if verbose=True , recheck a symbol if force=True | verbose=False force=True | bool |
ljustUELs | will left justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lowerUELs | will lowercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lstripUELs | will left strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
pivot | Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1] . If columns is None then it is set to the last dimension. If value is None then the level values will be pivoted. Missing values in the pivot will take the value provided by fill_value | index=None (str , list , None ) columns=None (str , list , None ) value (str ) fill_value=None (int , float , str ) | pd.DataFrame |
removeUELs | removes UELs that appear in the symbol dimensions , If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. | uels=None (str , list , None ) dimensions=None (int , list , None ) | bool |
renameUELs | renames UELs (case-sensitive) that appear in the symbol dimensions . If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True , the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** | uels (str , list , dict ) dimensions (int , list , None ) allow_merge=False (bool ) | None |
reorderUELs | reorders the UELs in the symbol dimensions . If uels is None , reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the symbol. | uels (str , list , dict , None ) dimensions (int , list , None ) | None |
rjustUELs | will right justify all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
rstripUELs | will right strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
setRecords | main convenience method to set standard pandas.DataFrame records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). | records (many types) | None |
setUELs | set the UELs for symbol dimensions . If dimensions is None then set UELs for all dimensions. If rename=True , then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) rename=False (bool ) | None |
stripUELs | will strip whitespace from all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
titleUELs | will title (capitalize all individual words) in all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
toDense | convert column to a dense numpy.array format | column="level" (str ) | numpy.array or None |
toDict | convenience method to return symbol records as a Python dictionary. columns will control which attributes to include in the dict . orient can take values natural or columns and will control the shape of the dict . Must use orient="columns" if attempting to set symbol records with setRecords . | columns="level" (str) orient="natural" (str) | dict |
toList | convenience method to return symbol records as a Python list, the columns argument will control with attributes to include in the list | columns="level" (str) | list |
toSparseCoo | convert column to a sparse COOrdinate numpy.array format | column="level" (str ) | sparse matrix format or None |
toValue | convenience method to return symbol records as a Python float. Only possible with scalar symbols. Attribute can be specified with column argument. | column="level" (str) | float |
upperUELs | will uppercase all UELs in the symbol or a subset of specified dimensions , can be chained with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
whereMax | find the domain entry of records with a maximum value (return first instance only) | column="level" (str ) | list of str or None |
whereMaxAbs | find the domain entry of records with a maximum absolute value (return first instance only) | column="level" (str ) | list of str or None |
whereMin | find the domain entry of records with a minimum value (return first instance only) | column="level" (str ) | list of str or None |
Adding Equation Records
Adding equation records mimics that of variables – three possibilities exist to assign symbol records to an equation (roughly ordered in complexity):
- Setting the argument
records
in the set constructor/container method (internally callssetRecords
) - creates a data copy - Using the symbol method
setRecords
- creates a data copy - Setting the property
records
directly - does not create a data copy
Setting equation records require the user to be explicit with the type of equation that is being created; in contrast to setting variable records (where the default variable is considered to be free
).
If the data is in a convenient format, a user may want to pass the records directly within the equation constructor. This is an optional keyword argument and internally the equation constructor will simply call the setRecords
method. In contrast to the setRecords
methods in in either the Set or Parameter classes the setRecords
method for variables will only accept Pandas DataFrames and specially structured dict
for creating records from matrices. This restriction is out of necessity because to properly set a record for an Equation the user must pass data for the level
, marginal
, lower
, upper
and scale
attributes. That said, any missing attributes will be filled in with the GAMS default record values (level = 0.0
, marginal = 0.0
, lower = -inf
, upper = inf
, scale = 1.0
). We show a few examples of ways to create differently structured variables:
- Example #1 - Create a GAMS scalar equation
- Example #2 - Create a 1D Equation (defined over
*
) from a list of tuples
In this example we only set the marginal
values.
- Example #3 - Create a 1D Equation (defined over a set) from a list of tuples
- Example #4 - Create a 2D equation, specifying no numerical data
- Example #5 - Create a 2D equation (defined over a set) from a matrix
- Example #6 - Create a 1D equation from a pandas Series
- Note
transfer
will assume that thelevel
value is being set if attributes cannot be found in the axes (same behavior when settingVariables
)
- Example #7 - Create a 1D equation from a pandas Series (set only the marginal)
- Example #8 - Create a 2D equation from a DataFrame (uels_on_axes=True)
Directly Set Records
As with set, parameters and variables, the primary advantage of the setRecords
method is that transfer
will convert many different (and convenient) data types into the standard data format (a Pandas DataFrame). Users that require higher performance will want to directly pass the Container
a reference to a valid Pandas DataFrame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process and it avoids making a copy of the data in memory. In this section we walk the user through an example of how to set records directly.
- Example #1 - Correctly set records (directly)
- Attention
- All numeric data in the records will need to be type
float
in order to maintain a valid symbol.
In this example we create a large equation (31,536,000 records and 8880 unique domain elements) and assign it to a variable with a.records
. transfer
requires that all domain columns must be a categorical data type, furthermore this categorical must be ordered. The records
setter function does very little work other than checking if the object being set is a DataFrame. This places more responsibility on the user to create a DataFrame that complies with the standard format. In Example #1 we take care to properly reference the categorical data types from the domain sets – and in the end a.isValid() = True
. As with Set and Parameters, users can use the .isValid(verbose=True)
method to debug any structural issues.
Generate Equation Records
Generating the initial pandas.DataFrame
object could be difficult for Equation
symbols that have a large number of records and a small number of UELs – these higher dimensional symbols will benefit from the generateRecords
convenience function. Internally, generateRecords
computes the dense Cartesian product of all the domain sets that define a symbol (generateRecords
will only work on symbols where <symbol>.domain_type == "regular"
).
- Example #1 - Create a large (dense) 4D equation
- Note
- In Example #1 a large 4D equation was generated – by default, only the
level
value of these records are randomly drawn from the interval[0,1]
(uniform distribution). Other variable attributes take the default record value.
As with Variables
, it is possible to generate a sparse variable with the density
argument to generateRecords
. We extend this example by passing our own custom func
argument that will control the behavior of the value
columns. The func
argument accepts a dict
of callables
(i.e., a reference to a function).
- Example #2 - Create a large (sparse) 4D equation with normally distributed values
- Note
- The custom
callable
function reference must expose asize
argument. It might be tedious to know the exact number of the records that will be generated, especially if a fractional density is specified; therefore, thegenerateRecords
method will pass in the correct size automatically. Users are encouraged to use the Numpy suite of random distributions when generating samples – custom functions have the potential to be computationally burdensome if a symbol has a large number of records.
Alias
There are two different ways to create a GAMS alias and add it to a Container
.
- Use
Alias
constructor - Use the
Container
methodaddAlias
(which internally calls theAlias
constructor)
Constructor
- Constructor Arguments
Argument | Type | Description | Required | Default |
---|---|---|---|---|
alias_with | Set object | set object from which to create an alias | Yes | - |
container | Container | A reference to the Container object that the symbol is being added to | Yes | - |
name | str | Name of symbol | Yes | - |
- Note
- The Alias property
alias_with
can be updated through the object constructor (a new object will not be created) if a symbol of the same name already exists in the container.
- Example - Creating an alias from a set
transfer
only stores the reference to the parent set as part of the alias structure – most properties that are called from an alias object simply point to the properties of the parent set (with the exception of container
, name
, and alias_with
). It is possible to create an alias from another alias object. In this case a recursive search will be performed to find the root parent set – this is the set that will ultimately be stored as the alias_with
property. We can see this behavior in the following example:
Properties
Property | Description | Type | Special Setter Behavior |
---|---|---|---|
alias_with | aliased object | Set | - |
container | reference to the Container that the symbol belongs to | Container | - |
description | description of symbol | str | - |
dimension | dimension of symbol | int | setting is a shorthand notation to create ["*"] * n domains in symbol |
domain | list of domains given either as string (* for universe set) or as reference to the Set/Alias object | list , str , or Set/Alias | - |
domain_forwarding | flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth). Can pass as a list of bool to control which domains to forward. | bool or list | no effect after records have been set |
domain_labels | column headings for the records DataFrame | list of str | will add a _<dimension> tag to user supplied column names (if not unique) |
domain_names | string version of domain names | list of str | - |
domain_type | none , relaxed or regular depending on state of domain links | str | - |
is_singleton | if symbol is a singleton set | bool | - |
modified | Flag that identifies if the Alias has been modified | bool | - |
name | name of symbol | str | sets the GAMS name of the symbol |
number_records | number of symbol records (i.e., returns len(self.records) if not None ) | int | - |
records | the main symbol records | pandas.DataFrame | responsive to domain_forwarding state |
summary | output a dict of only the metadata | dict | - |
Methods
Method | Description | Arguments/Defaults | Returns |
---|---|---|---|
addUELs | adds UELs to the parent set dimensions . If dimensions is None then add UELs to all dimensions. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) | None |
equals | Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_element_text=True then check that all set elements have the same descriptive element text, otherwise skip. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. If verbose=True will return an exception from the asserter describing the nature of the difference. | check_uels=True (bool ) check_element_text=True (bool ) check_meta_data=True (bool ) verbose=False (bool ) | bool |
capitalizeUELs | will capitalize all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
casefoldUELs | will casefold all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
countDomainViolations | returns the count of how many records in the parent set contain at least one domain violation | - | int |
countDuplicateRecords | returns the count of how many (case insensitive) duplicate records exist in the parent set | - | int |
dropDomainViolations | drop records from the parent set that contain a domain violation | - | None |
dropDuplicateRecords | drop records with (case insensitive) duplicate domains from the parent set – keep argument can take values of "first" (keeps the first instance of a duplicate record), "last" (keeps the last instance of a record), or False (drops all duplicates including the first and last) | keep="first" (str, False ) | None |
findDomainViolations | get a view of the records DataFrame that contain any domain violations | - | pandas.DataFrame |
findDuplicateRecords | get a view of the records DataFrame from the parent set that contain any (case insensitive) duplicate domains – keep argument can take values of "first" (finds all duplicates while keeping the first instance as unique), "last" (finds all duplicates while keeping the last instance as unique), or False (finds all duplicates) | keep="first" (str, False ) | pandas.DataFrame |
getDomainViolations | returns a list of DomainViolation objects if any (None otherwise) | - | list or None |
getSparsity | get the sparsity of the symbol w.r.t the cardinality | - | float |
getUELs | gets UELs from the parent set dimensions . If dimensions is None then get UELs from all dimensions (maintains order). The argument codes accepts a list of str UELs and will return the corresponding int ; must specify a single dimension if passing codes . Returns only UELs in the data if ignore_unused=True , otherwise return all UELs. | dimensions=None (int , list , None ) codes=None (int , list , None ) ignore_unused=False (bool ) | list |
hasDomainViolations | returns True if there are domain violations in the records of the parent set, returns False if not. | - | bool |
hasDuplicateRecords | returns True if there are (case insensitive) duplicate records in the parent set, returns False if not. | - | bool |
isValid | checks if the symbol is in a valid format, throw exceptions if verbose=True , re-check a symbol if force=True | verbose=False force=True | bool |
ljustUELs | will left justify all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lowerUELs | will lowercase all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
lstripUELs | will left strip whitespace from all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
pivot | Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1] . If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value | index=None (str , list , None ) columns=None (str , list , None ) fill_value=None (int , float , str ) | pd.DataFrame |
removeUELs | removes UELs that appear in the parent set dimensions , If uels is None then remove all unused UELs (categories). If dimensions is None then operate on all dimensions. | uels=None (str , list , None ) dimensions=None (int , list , None ) | bool |
renameUELs | renames UELs (case-sensitive) that appear in the parent set dimensions . If dimensions is None then operate on all dimensions of the symbol. If allow_merge=True , the categorical object will be re-created to offer additional data flexibility. ** All trailing whitespace is trimmed ** | uels (str , list , dict ) dimensions (int , list , None ) allow_merge=False (bool ) | None |
reorderUELs | reorders the UELs in the parent set dimensions . If uels is None , reorder UELs to data order and append any unused categories. If dimensions is None then reorder UELs in all dimensions of the parent set. | uels (str , list , dict , None ) dimensions (int , list , None ) | None |
rjustUELs | will right justify all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | length (int ) fill_character=None - if None , assumes " " dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
rstripUELs | will right strip whitespace from all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
setRecords | main convenience method to set standard pandas.DataFrame formatted records. If uels_on_axes=True setRecords will assume that all domain information is contained in the axes of the pandas object – data will be flattened (if necessary). | records (many types) | None |
setUELs | set the UELs for parent set dimensions . If dimensions is None then set UELs for all dimensions. If rename=True , then the old UEL names will be renamed with the new UEL names. ** All trailing whitespace is trimmed ** | uels (str , list ) dimensions=None (int , list , None ) rename=False (bool ) | None |
stripUELs | will strip whitespace from all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
titleUELs | will title (capitalize all individual words) in all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
toList | convenience method to return symbol records as a Python list | include_element_text=False (bool) | list |
upperUELs | will uppercase all UELs in the parent set or a subset of specified dimensions in the parent set, can be chain with other *UELs string operations | dimensions=None (int , list , or None ) - if None , assumes all symbols | self |
Adding Alias Records
The linked structure of Aliases offers some unique opportunities to access some of the setter functionality of the parent set. Specifically, transfer
allows the user to change the domain
, description
, dimension
, and records
of the underlying parent set as a shorthand notation. We can see this behavior if we look at a modified Example #1 from Adding Set Records.
- Example - Creating set records through an alias link
- Note
- An alias
.isValid()=True
when the underlying parent set is also valid – if the parent set is removed from the Container the alias will no longer be valid.
UniverseAlias
There are two different ways to create a GAMS UniverseAlias (an alias to the universe) and add it to a Container
.
- Use
UniverseAlias
constructor - Use the
Container
methodaddUniverseAlias
(which internally calls theUniverseAlias
constructor)
Constructor
- Constructor Arguments
Argument | Type | Description | Required | Default |
---|---|---|---|---|
container | Container | A reference to the Container object that the symbol is being added to | Yes | - |
name | str | Name of symbol | Yes | - |
- Example - Creating an alias to the universe
In GAMS it is possible to create aliases to the universe (i.e., the entire list of UELs) with the syntax:
set i / i1, i2 /;
alias(h,*);
set j / j1, j2 /;
In this small example, h
would be associated with all four UELs (i1
, i2
, j1
and j2
) even though set j
was defined after the alias declaration. transfer
mimics this behavior with the UniverseAlias
class. Internally, the records
attribute will always call the <Container>.getUELs()
and build the Pandas DataFrame
on the fly. The UniverseAlias
class is fundamentally different from the Alias
class because it does not point to a parent set at all; it is not possible to perform operations (like setRecords
or findDomainViolations
) on the parent set through a UniverseAlias
(because there is no parent set). This means that a UniverseAlias
can be created by only defining the symbol name. We can see this behavior in the following example:
- Note
- Unlike other sets, the universe does not hold on to set
element_text
, thus the returnedDataFrame
for theUniverseAlias
will only have 1 column.
Properties
Property | Description | Type | Special Setter Behavior |
---|---|---|---|
alias_with | always * | str | - |
container | reference to the Container that the symbol belongs to | Container | - |
description | always Aliased with * | str | - |
dimension | always 1 | int | - |
domain | always ["*"] | list of str | - |
domain_labels | always ["*"] | list of str | - |
domain_names | always ["*"] | list of str | - |
domain_type | always none | str | - |
is_singleton | always False | bool | - |
modified | flag that identifies if the UniverseAlias has been modified | bool | - |
name | name of symbol | str | sets the GAMS name of the symbol |
number_records | number of symbol records (i.e., returns len(records) if not None ) | int | - |
records | the main symbol records | pandas.DataFrame | - |
summary | output a dict of only the metadata | dict | - |
Methods
Method | Description | Arguments/Defaults | Returns |
---|---|---|---|
equals | Used to compare the symbol to another symbol. If check_uels=True then check both used and unused UELs and confirm same order, otherwise only check used UELs in data and do not check UEL order. If check_element_text=True then check that all set elements have the same descriptive element text, otherwise skip. If check_meta_data=True then check that symbol name and description are the same, otherwise skip. If verbose=True will return an exception from the asserter describing the nature of the difference. | check_uels=True (bool ) check_element_text=True (bool ) check_meta_data=True (bool ) verbose=False (bool ) | bool |
getSparsity | always 0.0 | - | float |
getUELs | gets UELs from the Container . Returns only UELs in the data if ignore_unused=True , otherwise return all UELs. | ignore_unused=False (bool ) | list |
isValid | checks if the symbol is in a valid format, throw exceptions if verbose=True , re-check a symbol if force=True | verbose=False force=True | bool |
pivot | Convenience function to pivot records into a new shape (only symbols with >1D can be pivoted). If index is None then it is set to dimensions [0..dimension-1] . If columns is None then it is set to the last dimension. Missing values in the pivot will take the value provided by fill_value | index=None (str , list , None ) columns=None (str , list , None ) fill_value=None (int , float , str ) | pd.DataFrame |
toList | convenience method to return symbol records as a Python list | - | list |
DomainViolation
DomainViolation
objects are convenient containers that store information about the location of domain violations in a symbol. These objects are computed dynamically with the getDomainViolations
method and should not be instantiated by the user (they are read-only, to the extent that this is possible in Python). However, the user may be interested in some of the information that they contain.
Constructor
- Constructor Arguments/Properties
Argument | Type | Description | Required | Default |
---|---|---|---|---|
symbol | _Symbol | A reference to the _Symbol object that has a domain violation | Yes | - |
dimension | int | An index to the dimension of the symbol where the domain violation exists | Yes | - |
domain | Set , Alias or UniverseAlias | A reference to the symbol domain that is the source of the domain violation | Yes | - |
violations | list | A list of all the domain elements that are causing violations | Yes | - |