GAMS Transfer R

GAMS Transfer is a package to maintain GAMS data outside a GAMS script in a programming language like Python, Matlab, or R. It allows the user to add GAMS symbols (Sets, Aliases, Parameters, Variables and Equations), to manipulate GAMS symbols, as well as read/write symbols to different data endpoints. GAMS Transfer's main focus is the highly efficient transfer of data between GAMS and the target programming language, while keeping those operations as simple as possible for the user. In order to achieve this, symbol records - the actual and potentially large-scale data sets - are stored in native data structures of the corresponding programming languages. The benefits of this approach are threefold: (1) The user is usually very familiar with these data structures, (2) these data structures come with a large tool box for various data operations, and (3) optimized methods for reading from and writing to GAMS can transfer the data as a bulk - resulting in the high performance of this package.

Getting Started

Install

The user must download and install the latest version of GAMS in order to install GAMS Transfer R. GAMS Transfer R can then be installed from either the source package or from the binary package.

The installation from the source will install gamstransfer and all of its dependencies.

install.packages("[PathToGAMS]/apifiles/R/gamstransfer/source/gamstransfer_r.tar.gz", dependencies=TRUE)

The users can also install gamstransfer without compiling the package source code using gamstransfer binary package. The binary packages are platform dependent and the instructions for each supported platform are shown below.

Windows

install.packages("[PathToGAMS]/apifiles/R/binary/gamstransfer.zip", type="binary")

Linux

install.packages("[PathToGAMS]/apifiles/R/binary/gamstransfer.tar.gz")

macOS

install.packages("[PathToGAMS]/apifiles/R/binary/gamstransfer.tgz", type="binary")

GAMS Transfer R depends on packages R6, R.utils, and Rcpp.

Examples

GDX Read

Reading in all symbols can be accomplished with one line of code (we reference data from the trnsport.gms example).

m = Container$new("trnsport.gdx")

All symbol data is organized in the data field (a named list) which can be accessed via m$data. The records are stored as data frames and can be accessed via m$data$<symbol_name>$records or m$data[[symbol_name]]$records.

GDX Write

There are five symbol classes within GAMS Transfer R: Sets, Parameters, Variables, Equations, and Aliases. For purposes of this quick start, we show how to recreate the distance data structure from the trnsport.gms model (the parameter d). This brief example shows how users can achieve "GAMS-like" functionality, but within an R environment. GAMS Transfer R leverages the object oriented programming to simplify the syntax.

m = Container$new()
# create the sets i, j
i = Set$new(m, "i", records = c("seattle", "san-diego"), description = "supply")
j = Set$new(m, "j", records = c("new-york", "chicago", "topeka"), description = "markets")
# add "d" parameter -- domain linked to set objects i and j
d = Parameter$new(m, "d", c(i, j), description = "distance in thousands of miles")
# create some data as a generic data frame
dist = data.frame(
from = c("seattle", "seattle", "seattle",
"san-diego", "san-diego", "san-diego"),
to = c("new-york", "chicago", "topeka",
"new-york", "chicago", "topeka"),
thousand_miles = c(2.5, 1.7, 1.8, 2.5, 1.8, 1.4)
)
# setRecords will automatically convert the dist data frame into
# a standard data frame format
d$setRecords(dist)
# write the GDX
m$write("out.gdx")

This example shows a few fundamental features of GAMS Transfer R:

  1. A Container is analogous to a GDX file
  2. Symbols will always be linked to a Container (notice that we always pass the Container reference m to the symbol constructor)
  3. Records can be added to a symbol with the setRecords() method, through the records constructor argument (internally calls setRecords()), or through directly setting the records field. GAMS Transfer R will convert many common R data structures into a standard format.
  4. Domain linking is possible by passing domain set objects to other symbols.
  5. Writing a GDX file can be accomplished in one line with the write() method.

Full Example

It is possible to use GAMS Transfer R to recreate the trnsport.gms results in GDX form. As part of this example, we also introduce the write() method (and generate new.gdx). We will discuss it in more detail in the following section: Data Exchange with GDX.

# create an empty Container object
m = Container$new()
# add sets
i = Set$new(m, "i", records=c("seattle", "san-diego"), description="supply")
j = Set$new(m, "j", records=c("new-york", "chicago", "topeka"), description="markets")
# add parameters
a = Parameter$new(m, "a", c("*"), description="capacity of plant i in cases")
b = Parameter$new(m, "b", j, description="demand at market j in cases")
d = Parameter$new(m, "d", c(i, j), description="distance in thousands of miles")
f = Parameter$new(
m, "f", records=90, description="freight in dollars per case per thousand miles"
)
c = Parameter$new(
m, "c", c(i, j), description="transport cost in thousands of dollars per case"
)
# set parameter records
cap = data.frame(plant = c("seattle", "san-diego"), n_cases = c(350, 600))
a$setRecords(cap)
dem = data.frame(market = c("new-york","chicago", "topeka"), n_cases = c(325, 300, 275))
b$setRecords(dem)
dist = data.frame(
from = c("seattle", "seattle", "seattle",
"san-diego", "san-diego", "san-diego"),
to = c("new-york", "chicago", "topeka",
"new-york", "chicago", "topeka"),
thousand_miles = c(2.5, 1.7, 1.8, 2.5, 1.8, 1.4)
)
d$setRecords(dist)
# c(i,j) = f * d(i,j) / 1000;
cost = d$records
cost$value = f$records$value * cost$value/1000
c$setRecords(cost)
# add variables
q = data.frame(
from = c("seattle", "seattle", "seattle",
"san-diego", "san-diego", "san-diego"),
to = c("new-york", "chicago", "topeka",
"new-york", "chicago", "topeka"),
level = c(50, 300, 0, 275, 0, 275),
marginal = c(0, 0, 0.036, 0, 0.009, 0)
)
x = Variable$new(m, "x", "positive", c(i, j), records=q, description="shipment quantities in cases")
z = Variable$new(
m,
"z",
records=data.frame(level = 153.675),
description="total transportation costs in thousands of dollars"
)
# add equations
cost = Equation$new(m, "cost", "eq", description="define objective function")
supply = Equation$new(m, "supply", "leq", i, description="observe supply limit at plant i")
demand = Equation$new(m, "demand", "geq", j, description="satisfy demand at market j")
# set equation records
cost$setRecords(data.frame(level = 0, marginal = 1, lower = 0, upper = 0))
supplies = data.frame(
from = c("seattle", "san-diego"),
level = c(350, 550),
marginal = c(SpecialValues$EPS, 0),
lower = c(SpecialValues$NEGINF, SpecialValues$NEGINF),
upper = c(350, 600)
)
supply$setRecords(supplies)
demands = data.frame(
from = c("new-york", "chicago", "topeka"),
level = c(325, 300, 275),
marginal = c(0.225, 0.153, 0.126),
lower = c(325, 300, 275)
)
demand$setRecords(demands)
m$write("new.gdx")

It can be observed from the above example that a typical work flow for writing using GAMS Transfer R is creating a container, filling it with symbols (Sets, Parameters, Variables, Equations, and Aliases), and write it to a GDX file. To read a GDX file, a Container can simply be initialized with the GDX file name as an argument.

These examples introduced the reader to the GAMS Transfer R syntax. In the remaining sections, we will present details about the core functionality and dig further into the syntax.

Manual

Container

Storing, manipulating, and transforming sparse data requires that it lives within an environment. This data can then be linked together to enable various operations. In GAMS Transfer R, we refer to this "environment" as the Container, it is the main repository for storing our data. GAMS Transfer R offers two types of containers: Container and ConstContainer. By default, the user should choose Container. For even higher efficiency, one can choose a ConstContainer. However, it is important to know that it is a constant container and the symbols cannot be updated once added.

Container

In GAMS Transfer R, the main object class for storing and linking our data is called Container. Linking data enables data operations such as implicit set growth, domain checking, and data format transformations (to dense/sparse matrix formats). A Container also enables different read/write operations.

Creating a Container is a simple matter of initializing an object. For example:

m = Container$new()

This new Container object, here called m, contains a number of convenient fields and methods that allow the user to interact with the symbols that are in the Container. Some of these methods are simply used to filter out different types of symbols, other methods are used to numerically characterize the data within each symbol.

The Container constructor arguments are:

Argument Type Description Required Default
loadFrom string Name of the GDX file being read into the Container No NULL
systemDirectory string Absolute path to GAMS system directory No Attempts to find the GAMS installation by checking path system variable and looking for the GAMS executable.

Container fields
Field Description Type Special Setter Behavior
data main list that is used to store all symbol data list -

Symbols are organized in the Container under the data field. The $ notation (m$data) is used to access the underlying named list; symbols in this named list can then be retrieved with the standard bracket (or double bracket) notation (m$data[<symbol_name>]).

Container methods
Method Description Arguments/Defaults Returns
addAlias add an Alias name (string)
aliasWith (Set,Alias)
Alias object
addUniverseAlias add a UniverseAlias name (string)
UniverseAlias object
addEquation add an Equation name (string)
type (string)
domain=NULL (string,list)
records=NULL (data frame,vector,NULL)
domainForwarding=FALSE (logical)
description="" (string)
Equation object
addParameter add a Parameter name (string)
domain=NULL (string,list)
records=NULL (data frame,array, matrix)
domainForwarding=FALSE (logical)
description="" (string)
Parameter object
addSet add a Set name (string)
domain="*" (string,list)
isSingleton=FALSE (logical)
records=NULL (data frame,array,matrix)
domainForwarding=FALSE (logical)
description="" (string)
Set object
addVariable add a Variable name (string)
type="free" (string)
domain=NULL (string,list)
records=NULL (data frame,array,matrix)
domainForwarding=FALSE (logical)
description="" (string)
Variable object
countDomainViolations returns a named list containing number of domain violations symbols with non-zero domain violations - list
countDuplicateRecords returns a named list containing number of duplicate records for symbols with non-zero duplicate records - list
describeAliases create a summary table with descriptive statistics for Aliases symbols=NULL (string,list) - if NULL, assumes all Aliases data frame
describeEquations create a summary table with descriptive statistics for Equations symbols=NULL (string,list) - if NULL, assumes all equations data frame
describeParameters create a summary table with descriptive statistics for Parameters symbols=NULL (string,list) - if NULL, assumes all parameters data frame
describeSets create a summary table with descriptive statistics for Sets symbols=NULL (string,list) - if NULL, assumes all sets data frame
describeVariables create a summary table with descriptive statistics for Variables symbols=NULL (string,list) - if NULL, assumes all variables data frame
dropDomainViolations drops domain violations for symbols in the Container - -
dropDuplicateRecords drops duplicate records for symbols in the Container keep= first/last -
getDomainViolations gets domain violations for symbols in the Container - list of DomainViolation
getSymbolNames returns the original symbol names for a vector names of any case names (character) character
getSymbols returns a list of object references for symbols symbols (character, vector, list) Symbol (for one symbol) or a list
getUELs returns the UELs used in the Container symbols=NULL (character), ignoreUnused = FALSE character vector
getUniverseSet provides a universe for all symbols - list
hasDomainViolations returns TRUE if any symbol in the container has domain violations, FALSE otherwise - logical
hasDuplicateRecords returns TRUE if any symbol in the container has duplicate records, FALSE otherwise - logical
hasSymbols checks if symbol names for a vector names exists in the refContainer names (character) logical
isValid TRUE if all symbols in the Container are valid verbose=FALSE (logical)
force=FALSE (logical)
logical
listAliases list all aliases (isValid=NULL), list all valid aliases (isValid=TRUE), list all invalid aliases (isValid=FALSE) in the container isValid=NULL (logical) vector
listEquations list all equations (isValid=NULL), list all valid equations (isValid=TRUE), list all invalid equations (isValid=FALSE) in the container isValid=NULL (logical)
types=NULL (list or vector of eqaution types) - if NULL, assumes all types
vector
listParameters list all parameters (isValid=NULL), list all valid parameters (isValid=TRUE), list all invalid parameters (isValid=FALSE) in the container isValid=NULL (logical) vector
listSets list all sets (isValid=NULL), list all valid sets (isValid=TRUE), list all invalid sets (isValid=FALSE) in the container isValid=NULL (logical) vector
listSymbols list all symbols (isValid=NULL), list all valid symbols (isValid=TRUE), list all invalid symbols (isValid=FALSE) in the container isValid=NULL (logical) vector
listVariables list all variables (isValid=NULL), list all valid variables (isValid=TRUE), list all invalid variables (isValid=FALSE) in the container isValid=NULL (logical)
types=NULL (list or vector of variable types) - if NULL, assumes all types
vector
read main method to read loadFrom, can be provided with a list of symbols to read in subsets, records controls if symbol records are loaded or just metadata loadFrom (string, Container, ConstContainer)
symbols="all" (string, list)
records=TRUE (logical)
NULL
removeSymbols symbols to remove from the Container, also sets the symbols' refContainer to NULL symbols (string,list) NULL
removeUELs removes uels from all symbols the Container uels=NULL (character) NULL
renameSymbol rename a symbol oldName in the Container oldName (string), newName (string) NULL
renameUELs renames uels in the Container uels (named character), allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL
reorderSymbols reorder symbols in order to avoid domain violations - NULL
write main bulk write method to a writeTo target writeTo (string)
compress=FALSE (logical)
uelPriority=NULL (string,list)
NULL

ConstContainer

While many GAMS Transfer users can rely on the Container for building their data pipeline, some users may only be interested in post-processing data from a GAMS model run. This one-directional flow of data means that these users do not need some of the advanced Container features such as domain linking, matrix generation, domain checking, etc. The ConstContainer (i.e., a Constant Container) object class is a data-focused read-only object that will provide a snapshot of the data target (a GDX file) being read.

The ConstContainer shares many of the same methods and attributes that are in the Container Container class, which makes moving between the ConstContainer and the Container easy. Some important differences should be kept in mind:

  • The ConstContainer does not link any symbol data
  • The ConstContainer can only read from one source at a time. Every new call of $read() will clear the data list.
  • The ConstContainer constructor does not read in any symbol records. This enables users to browse an unknown data source quickly (similar behavior to gdxdump).
  • The ConstContainer does not have a $write() method. A ConstContainer can be read by a Container to enable data writing (however, a copy of the data will be generated).
  • The user will never need to instantiate a symbol object and add it to the ConstContainer. The ConstContainer internally generates its own set of (simplified) symbol classes and holds them in the $data field.

All of these differences were inspired by users that want to read the data as fast as possible and probe unknown data files without worrying about memory issues. ConstContainer provides users with a high level view of the data quickly.

Creating a ConstContainer is a simple matter of initializing an object. For example:

h = ConstContainer$new("out.gdx")

This new ConstContainer object, here called h, will load all the symbol data from out.gdx but it will not load any of the records. To load records, users must use the $read() method.

The ConstContainer constructor arguments are:

Argument Type Description Required Default
loadFrom string Name of the GDX file being read into the Container No NULL
systemDirectory string Absolute path to GAMS system directory No Attempts to find the GAMS installation by checking path system variable and looking for the GAMS executable.

ConstContainer fields
Field Description Type Special Setter Behavior
data main list that is used to store all symbol data list -

Symbols are organized in the Container under the data field. The $ notation (m$data) is used to access the underlying named list; symbols in this named list can then be retrieved with the standard bracket (or double bracket) notation (m$data[<symbol_name>]).

ConstContainer methods
Method Description Arguments/Defaults Returns
describeAliases create a summary table with descriptive statistics for Aliases symbols=NULL (string,list) - if NULL, assumes all Aliases data frame
describeEquations create a summary table with descriptive statistics for Equations symbols=NULL (string,list) - if NULL, assumes all equations data frame
describeParameters create a summary table with descriptive statistics for Parameters symbols=NULL (string,list) - if NULL, assumes all parameters data frame
describeSets create a summary table with descriptive statistics for Sets symbols=NULL (string,list) - if NULL, assumes all sets data frame
describeVariables create a summary table with descriptive statistics for Variables symbols=NULL (string,list) - if NULL, assumes all variables data frame
getSymbolNames returns the original symbol names for a vector names of any case names (character) character
hasSymbols checks (case insensitive) if symbol names for a vector names exists in the refContainer names (character) logical
listAliases list all aliases in the container - vector
listEquations list all equations in the container types=NULL (list or vector of equation types) - if NULL, assumes all types vector
listParameters list all parameters in the container - vector
listSets list all sets in the container - vector
listSymbols list all symbols in the container - vector
listVariables list all variables in the container types=NULL (list or vector of variable types) - if NULL, assumes all types vector
read main method to read loadFrom, can be provided with a list of symbols to read in subsets, records controls if symbol records are loaded or just metadata loadFrom (string)
symbols="all" (string, list)
records=TRUE (logical)
NULL

The structure of the DataFrames that are returned from the describe* methods mirrors that in the Container; the user should reference Describing Data for detailed descriptions of the columns.

Symbols

In GAMS Transfer R, a symbol is either a GAMS Set, Parameter, Variable, Equation, Alias. In GAMS Transfer R, a symbol cannot live on its own, but is always part of a Container or a ConstContainer.

Create a symbol

There are two different ways to create a GAMS set and add it to a Container.

  1. Use symbol constructor for Set, Parameter, Variable, Equation, Alias
    m = Container$new()
    p = Parameter$new(m, "p")
  2. Use the Container methods addSet, addParameter, addVariable, addEquation, addAlias (which internally calls the symbol constructor)
    m = Container$new()
    p = m$addParameter("p")

Add Symbol Records

Three possibilities exist to assign symbol records:

  1. Setting the argument records in the symbol constructor/Container method (internally calls setRecords)
  2. Using the symbol method setRecords
  3. Setting the field records directly

If the data is in a convenient format, a user may want to pass the records directly within the set constructor. This is an optional keyword argument and internally the symbol constructor will simply call the setRecords method. The symbol method setRecords is a convenience method that transforms the given data into an approved data frame format (see Standard Data Formats). Many native R data types can be easily transformed into data frames, so the setRecords method will accept a number of different types for input. The setRecords method is called internally on any data structure that is passed through the records argument.

The examples of setting records using the above-mentioned methods can be found in the respective documentation for each symbol ( Set, Parameter, Variable, Equation, and Alias ).

Set

Set Constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name string Name of symbol Yes -
domain list String, List of domains given either as a string ("*" for universe set) or as a reference to a Set object No "*"
isSingleton logical Indicates if set is a singleton set (TRUE) or not (FALSE) No FALSE
records string vector, data frame Symbol records No NULL
domainForwarding logical Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No FALSE
description string Description of symbol No ""

Set Fields
Field Description Type
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list
domainForwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) logical
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
isSingleton logical if symbol is a singleton set logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
refContainer reference to the Container that the symbol belongs to Container
summary output a list of only the metadata list

Set Methods
Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol uels (character), dimension=NULL NULL
countDomainViolations returns the number of domain violations for the symbol - numeric
countDuplicateRecords returns the number of duplicate records for the symbol - numeric
findDomainViolations get the index of records that contain any domain violations - data.frame
getCardinality get the full cartesian product of the domain - integer
dropDomainViolations drops domain violations for the symbol - -
dropDuplicateRecords drops duplicate records for the Symbol keep= first/last -
getDomainViolations gets domain violations for the symbol - list of DomainViolation
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
getUELs returns the UELs used in the Symbol dimension = NULL, codes = NULL, ignoreUnused=FALSE character vector
hasDomainViolations returns TRUE if the symbol contains domain violations, FALSE otherwise - logical
hasDuplicateRecords returns TRUE if the symbol contains duplicate records, FALSE otherwise - logical
isValid checks if the symbol is in a valid format, throw exceptions if verbose=TRUE, recheck a symbol if force=TRUE verbose=FALSE
force=FALSE
logical
removeUELs removes UELs from the symbol uels=NULL (character), dimension=NULL NULL
renameUELs renames UELs in the symbol uels(character of same length as current UELs or named character vector), dimension=NULL, allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL
setRecords main convenience method to set standard data frame formatted records records (string vector, list, data frame) NULL
setUELs sets UELs for the Symbol uels(character), dimension, rename=FALSE NULL

Adding Set Records

Three possibilities exist to assign symbol records to a set: We show a few examples of ways to create differently structured sets.

Example #1 - Create a 1D set from a vector
m = Container$new()
i = Set$new(m, "i", records = c("seattle", "san-diego"))
# NOTE: the above syntax is equivalent to -
# i = Set$new(m, "i")
# i$setRecords(c("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# m$addSet("i", records= c("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# i = m$addSet("i")
# i$setRecords(c("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# m$addSet("i")
# m$data[["i"]]$setRecords(c("seattle", "san-diego"))
> i$records
uni_1 element_text
1 seattle
2 san-diego
Example #2 - Create a 2D set from a list
m = Container$new()
k = Set$new(m, "k", c("*", "*"), records=list("seattle", "san-diego"))
# NOTE: the above syntax is equivalent to -
# k = Set$new(m, "k", c("*", "*"))
# k$setRecords(list("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# m$addSet("k", c("*","*"), records=list("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# k = m$addSet("k", c("*","*"))
# k$setRecords(list("seattle", "san-diego"))
# NOTE: the above syntax is also equivalent to -
# m$addSet("k", c("*", "*"))
# m$data[["k"]]$setRecords(list("seattle", "san-diego"))
> k$records
uni_1 uni_2 element_text
1 seattle san-diego
Example #3 - Create a 1D set from a data frame slice
m = Container$new()
dist = data.frame(
from = c("seattle", "seattle", "seattle",
"san-diego", "san-diego", "san-diego"),
to = c("new-york", "chicago", "topeka",
"new-york", "chicago", "topeka"),
thousand_miles = c(2.5, 1.7, 1.8, 2.5, 1.8, 1.4)
)
l = Set$new(m, "l", records = unique(dist[["from"]]))
# NOTE: the above syntax is equivalent to -
# l = Set$new(m, "l")
# l$setRecords(unique(dist[["from"]]))
# NOTE: the above syntax is also equivalent to -
# m$addSet("l", records=unique(dist[["from"]]))
# NOTE: the above syntax is also equivalent to -
# l = m$addSet("l")
# l$setRecords(unique(dist[["from"]]))
# NOTE: the above syntax is also equivalent to -
# m$addSet("l")
# m$data[["l"]]$setRecords(unique(dist[["from"]]))
> l$records
uni_1 element_text
1 seattle
2 san-diego

Set element text is very handy when labeling specific set elements within a set. A user can add a set element text directly with a set element. Note that it is not required to label all set elements, as can be seen in the following example.

Example #4 - Add set element text
m = Container$new()
i = Set$new(m, "i",
records = data.frame(c("seattle", "san-diego", "washington_dc"),
c("home of sub pop records", "", "former gams hq")))
# NOTE: the above syntax is equivalent to -
#
# i = Set$new(m, "i")
# i_recs = data.frame(c("seattle", "san-diego", "washington_dc"),
# c("home of sub pop records", "", "former gams hq"))
#
# i$setRecords(i_recs)
# NOTE: the above syntax is also equivalent to -
# m$addSet("i", records=i_recs)
# NOTE: the above syntax is also equivalent to -
# i = m$addSet("i")
# i$setRecords(i_recs)
# NOTE: the above syntax is also equivalent to -
# m$addSet("i")
# m$data[["i"]]$setRecords(i_recs)
> i$records
uni_1 element_text
1 seattle home of sub pop records
2 san-diego
3 washington_dc former gams hq

The primary advantage of the setRecords method is that GAMS Transfer R will convert many different (and convenient) data types into the standard data format (a data frame). Users that require higher performance will want to directly pass the Container a reference to a valid data frame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process.
In this section, we walk the user through an example of how to set records directly.

Example #5 - Directly set records (1D set)
m = Container$new()
i = Set$new(m, "i", description = "supply")
# create a standard format data frame
df_i = data.frame(uni_1 = c("seattle", "san-diego"),
element_text = c("", ""))
# need to create categorical column type, referencing elements already in df_i
df_i$uni_1 = factor(df_i$uni_1, ordered = TRUE)
# set the records directly
i$records = df_i
> i$isValid()
[1] TRUE

Stepping through this example we take the following steps:

  1. Create an empty Container
  2. Create a GAMS set i in the Container, but do not set the records
  3. Create a data frame (manually, in this example) taking care to follow the standard format
  4. The data frame has the right shape and column labels so we can proceed to set the records.
  5. We need to cast the uni_1 column as a factor, so we create a custom ordered category type using factor
  6. Finally, we set the records directly by passing a reference to df_i into the symbol records attribute. The setter function of records checks that a data frame is being set, but does not check validity. Thus, as a final step, we call the $isValid() method to verify that the symbol is valid.
Note
Users can debug their data frames by running <symbol_name>$isValid(verbose=TRUE) to get feedback about their data.
Example #6 - Directly set records (1D subset)
m = Container$new()
i = Set$new(m, "i", records=c("seattle", "san-diego"), description="supply")
j = Set$new(m, "j", i, description="supply")
# create a standard format data frame
df_j = data.frame(i_1 = c("seattle"), "element_text" = c(""))
# create the categorical column type
df_j$i_1 = factor(df_j$i_1, levels = i$records[, 1], ordered = TRUE)
# set the records
j$records = df_j
> j$isValid()
[1] TRUE

This example is more subtle in that we want to create a set j that is a subset of i. We create the set i using the setRecords method but then set the records directly for j. There are two important details to note: 1) the column labels in df_j now reflect the standard format for a symbol with a domain set (as opposed to the universe) and 2) we create the factors by referencing the parent set (i) for the levels (instead of referencing itself).

Parameter

Parameter Constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name string Name of symbol Yes -
domain list List of domains given either as a string ("*" for universe set) or as a reference to a Set object, an empty domain list will create a scalar parameter No NULL
records many Symbol records No NULL
domainForwarding logical Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No FALSE
description string Description of symbol No ""

Parameter Fields
Field Description Type
description description of symbol character
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" numeric,integer
domain list of domains given either as a string ("*" for universe set) or as a reference to the Set/Alias object list
domainForwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) logical
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
isScalar TRUE if the length(self$domain) = 0 logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
refContainer reference to the Container that the symbol belongs to Container
shape a vector describing the array dimensions if records were converted with $toDense() vector
summary output a list of only the metadata list

Parameter Methods
Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol uels (character), dimension=NULL NULL
countDomainViolations returns the number of domain violations for the symbol - numeric
countDuplicateRecords returns the number of duplicate records for the symbol - numeric
countEps total number of SpecialValues$EPS in value column - integer
countNA total number of SpecialValues[["NA"]] in value column - integer
countNegInf total number of SpecialValues$NEGINF in value column - integer
countPosInf total number of SpecialValues$POSINF in value column - integer
countUndef total number of SpecialValues$UNDEF in value column - integer
dropDomainViolations drops domain violations for the symbol - -
dropDuplicateRecords drops duplicate records for the Symbol keep= first/last -
findDomainViolations get the index of records that contain any domain violations - data.frame
getCardinality get the full cartesian product of the domain - integer
getDomainViolations gets domain violations for the symbol - list of DomainViolation
getMaxValue get the maximum value in value column - numeric
getMaxAbsValue get the maximum absolute value in value column - numeric
getMeanValue get the mean value in value column - numeric
getMinValue get the minimum value in value column - numeric
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
getUELs returns the UELs used in the Symbol dimension = NULL, codes = NULL, ignoreUnused=FALSE character vector
hasDomainViolations returns TRUE if the symbol contains domain violations, FALSE otherwise - logical
hasDuplicateRecords returns TRUE if the symbol contains duplicate records, FALSE otherwise - logical
isValid checks if the symbol is in a valid format, throw exceptions if verbose=TRUE, recheck a symbol if force=TRUE verbose=FALSE
force=FALSE
logical
removeUELs removes UELs from the symbol uels=NULL (character), dimension=NULL NULL
renameUELs renames UELs in the symbol uels(character of same length as current UELs or named character vector), dimension=NULL, allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL
setRecords main convenience method to set standard data frame records records (many types) NULL
setUELs sets UELs for the Symbol uels(character), dimension, rename=FALSE NULL
toDense convert value column of a symbol to a dense matrix or array format - array or matrix
whereMax find the row number in records data frame with a maximum value (return the first instance only) - integer
whereMaxAbs find the row number in records data frame with a maximum absolute value (return the first instance only) - integer
whereMin find the row number in records data frame with a minimum value (return the first instance only) - integer

Adding Parameter Records

Three possibilities exist to assign symbol records to a parameter: We show a few examples of ways to create differently structured parameters:

Example #1 - Create a GAMS scalar
m = Container$new()
pi = Parameter$new(m, "pi", records = 3.14159)
# NOTE: the above syntax is equivalent to -
# pi = Parameter$new(m, "pi")
# pi$setRecords(3.14159)
# NOTE: the above syntax is also equivalent to -
# m$addParameter("pi", records=3.14159)
# NOTE: the above syntax is also equivalent to -
# pi = m$addParameter("pi")
# pi$setRecords(3.14159)
# NOTE: the above syntax is also equivalent to -
# m$addParameter("pi")
# m$data[["pi"]]$setRecords(3.14159)
> pi$records
value
1 3.14159
Note
GAMS Transfer R will still convert scalar values to a standard format (i.e., a data frame with a single row and column).
Example #2 - Create a 2D parameter (defined over a set) from a

data frame slice

dist = data.frame(
from = c("seattle", "seattle", "seattle",
"san-diego", "san-diego", "san-diego"),
to = c("new-york", "chicago", "topeka",
"new-york", "chicago", "topeka"),
thousand_miles = c(2.5, 1.7, 1.8, 2.5, 1.8, 1.4)
)
m = Container$new()
i = Set$new(m, "i", "*", records = unique(dist$from))
j = Set$new(m, "j", "*", records = unique(dist$to))
a = Parameter$new(m, "a", c(i, j), records = dist)
> a$toDense()
[,1] [,2] [,3]
[1,] 2.5 1.7 1.8
[2,] 2.5 1.8 1.4
# use a$toDense() to create a new (and identicial) parameter a2
a2 = Parameter$new(m, "a2", c(i, j), records = a$toDense())
> a2$records
i_1 j_2 value
1 seattle new-york 2.5
2 seattle chicago 1.7
3 seattle topeka 1.8
4 san-diego new-york 2.5
5 san-diego chicago 1.8
6 san-diego topeka 1.4
Example #3 - Create a 2D parameter from an array using setRecords
m = Container$new()
i = Set$new(m, "i", records=paste0("i_", 1:5))
j = Set$new(m, "j", records=paste0("j_", 1:5))
# create the parameter with linked domains (these will control the
# $shape of the symbol)
a = Parameter$new(m, "a", c(i, j))
# here we use the $shape field to easily generate a dense random array
a$setRecords(array(runif(prod(a$shape()), min = 1, max = 10),
dim = a$shape() ))
> a$toDense()
[,1] [,2] [,3] [,4] [,5]
[1,] 3.837345 3.632743 9.003275 4.097475 8.608477
[2,] 7.217257 2.465452 3.286330 2.366017 8.822535
[3,] 8.421044 8.546226 5.403918 2.286660 6.319740
[4,] 3.960100 8.538932 2.210829 2.437113 5.324722
[5,] 1.333846 4.508688 7.411279 5.653044 7.248775

As with Sets, the primary advantage of the setRecords method is that GAMS Transfer will convert many different (and convenient) data types into the standard data format (data frame). Users that require higher performance will want to directly pass the Container a reference to a valid data frame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process. In this section, we walk the user through an example of how to set records directly.

Example #4 - Correctly set records (directly)
df = data.frame(h_1 = paste0("h", 1:8760), m_2 = paste0("m", 1:60),
s_3 = paste0("s", 1:60))
df$value = runif(nrow(df), min = 0, max = 100)
m = Container$new()
hrs = Set$new(m, "h", records = unique(df$h_1))
mins = Set$new(m, "m", records = unique(df$m_2))
secs = Set$new(m, "s", records = unique(df$s_3))
df$h_1 = factor(df$h_1, ordered = TRUE)
df$m_2 = factor(df$m_2, ordered = TRUE)
df$s_3 = factor(df$s_3, ordered = TRUE)
a = Parameter$new(m, "a", c(hrs, mins, secs))
# set records
a$records = df
> a$isValid()
[1] TRUE

In this example, we create a large parameter (31,536,000 records and 8880 unique domain elements. We mimic data that is labeled for every second in one year) and assign it to a parameter with a$records. GAMS Transfer R requires that all domain columns must be ordered factors. The records setter function does very little work other than checking if the object being set is a data frame. This places more responsibility on the user to create a data frame that complies with the standard format. In Example #1, we take care to properly reference the factor from the domain sets, and ensure that the symbol a is valid with a$isValid() = TRUE.

Users will need to use the $isValid(verbose=TRUE) method to debug any structural issues. As an example, we incorrectly generate categorical data types by passing the data frame constructor the generic factor argument. This creates factor columns, but they are not ordered and they do not reference the underlying domain set. These errors result in a being invalid.

Example #5 - Incorrectly set records (directly)
df = data.frame(h_1 = paste0("h", 1:8760), m_2 = paste0("m", 1:60),
s_3 = paste0("s", 1:60))
df$value = runif(nrow(df), min = 0, max = 100)
m = Container$new()
hrs = Set$new(m, "h", records = unique(df$h_1))
mins = Set$new(m, "m", records = unique(df$m_2))
secs = Set$new(m, "s", records = unique(df$s_3))
df$h_1 = factor(df$h_1)
df$m_2 = factor(df$m_2)
df$s_3 = factor(df$s_3)
a = Parameter$new(m, "a", c(hrs, mins, secs))
# set records
a$records = df
> a$isValid()
[1] FALSE
> a$isValid(verbose=TRUE)
Domain information in column h_1 must be an ORDERED factor
[1] FALSE

Variable

Variable Constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name string Name of symbol Yes -
type string Type of variable being created [binary, integer, positive, negative, free, sos1, sos2, semicont, semiint] No free
domain list, vector List, vector of domains given either as a string ("*" for universe set) or as a reference to a Set object, an empty domain list will create a scalar variable No NULL
records many Symbol records No NULL
domainForwarding logical Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No FALSE
description string Description of symbol No ""

Variable Fields
Field Description Type
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list
domainForwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) logical
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
refContainer reference to the Container that the symbol belongs to Container
shape a vector describing the array dimensions if records were converted with $toDense() vector
summary output a list of only the metadata list
type string type of variable list

Variable Methods
Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol uels (character), dimension=NULL NULL
countDomainViolations returns the number of domain violations for the symbol - numeric
countDuplicateRecords returns the number of duplicate records for the symbol - numeric
countEps total number of SpecialValues$EPS across all columns columns = "level" integer
countNA total number of SpecialValues[["NA"]] across all columns columns = "level" integer
countNegInf total number of SpecialValues$NEGINF across all columns columns = "level" integer
countPosInf total number of SpecialValues$POSINF across all columns columns = "level" integer
countUndef total number of SpecialValues$UNDEF across all columns columns = "level" integer
dropDomainViolations drops domain violations for the symbol - -
dropDuplicateRecords drops duplicate records for the Symbol keep= first/last -
findDomainViolations get the index of records that contain any domain violations - data.frame
getCardinality get the full cartesian product of the domain - integer
getDomainViolations gets domain violations for the symbol - list of DomainViolation
getMaxValue get the maximum value across all columns columns = "level" numeric
getMaxAbsValue get the maximum absolute value across all columns columns = "level" numeric
getMinValue get the minimum value across all columns columns = "level" numeric
getMeanValue get the mean value across all columns columns = "level" numeric
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
getUELs returns the UELs used in the Symbol dimension = NULL, codes = NULL, ignoreUnused=FALSE character vector
hasDomainViolations returns TRUE if the symbol contains domain violations, FALSE otherwise - logical
hasDuplicateRecords returns TRUE if the symbol contains duplicate records, FALSE otherwise - logical
isValid checks if the symbol is in a valid format, throw exceptions if verbose=TRUE, recheck a symbol if force=TRUE verbose=FALSE
force=FALSE
logical
removeUELs removes UELs from the symbol uels=NULL (character), dimension=NULL NULL
renameUELs renames UELs in the symbol uels(character of same length as current UELs or named character vector), dimension=NULL, allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL
setRecords main convenience method to set standard data frame records records (many types) NULL
setUELs sets UELs for the Symbol uels(character), dimension, rename=FALSE NULL
toDense convert symbol to a dense matrix or array format column = level array or matrix
whereMax find the row number in records data frame with a maximum value (return the first instance only) column = "level" integer
whereMaxAbs find the row number in records data frame with a maximum absolute value (return the first instance only) column = "level" integer
whereMin find the row number in records data frame with a minimum value (return the first instance only) column = "level" integer

Adding Variable Records

Three possibilities exist to assign symbol records to a variable: We show a few examples of ways to create differently structured variables:

Example #1 - Create a GAMS scalar variable
m = Container$new()
pi = Variable$new(m, "pi", records = data.frame(level = 3.14159))
# NOTE: the above syntax is equivalent to -
# pi = Variable$new(m, "pi", "free")
# pi$setRecords(data.frame(level = 3.14159))
# NOTE: the above syntax is also equivalent to -
# m$addVariable("pi", "free", records=data.frame(level = 3.14159))
> pi$records
level marginal lower upper scale
1 3.14159 0 -Inf Inf 1
Example #2 - Create a 2D positive variable, specifying no numerical data
m = Container$new()
v = Variable$new(m, "v", "positive", c("*", "*"), records =
data.frame(c("seattle", "chicago"), c("san-diego", c("madison"))))
> v$records
uni_1 uni_2 level marginal lower upper scale
1 seattle san-diego 0 0 0 Inf 1
2 chicago madison 0 0 0 Inf 1
Example #3 - Create a 2D variable (defined over a set) from a matrix
m = Container$new()
i = Set$new(m, "i", "*", records = paste0("i", 1:5))
j = Set$new(m, "j", "*", records = paste0("j", 1:5))
# creating records for parameter a
ij = list(i_1 = paste0("i", 1:5), j_2 = paste0("j", 1:5))
df = rev(expand.grid(rev(ij)))
df$value = 1:25
a = Parameter$new(m, "a", c(i, j), records = df)
# create a free variable and set the level and marginal attributes from matricies
v = Variable$new(m, "v", domain = c(i, j), records = list(level = a$toDense(), marginal = a$toDense()))
# if not specified, the toDense() method will convert the level values to a matrix
> v$toDense()
[,1] [,2] [,3] [,4] [,5]
[1,] 1 2 3 4 5
[2,] 6 7 8 9 10
[3,] 11 12 13 14 15
[4,] 16 17 18 19 20
[5,] 21 22 23 24 25

As with Sets, the primary advantage of the setRecords method is that GAMS Transfer will convert many different (and convenient) data types into the standard data format (data frame). Users that require higher performance will want to directly pass the Container a reference to a valid data frame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process. In this section, we walk the user through an example of how to set records directly.

Example #4 - Correctly set records (directly)
df = data.frame(h_1 = paste0("h", 1:8760), m_2 = paste0("m", 1:60),
s_3 = paste0("s", 1:60))
df$level = runif(nrow(df), min = 0, max = 100)
df$marginal = 0.0
df$lower = SpecialValues$NEGINF
df$upper = SpecialValues$POSINF
df$scale = 1.0
m = Container$new()
hrs = Set$new(m, "h", records = unique(df$h_1))
mins = Set$new(m, "m", records = unique(df$m_2))
secs = Set$new(m, "s", records = unique(df$s_3))
df$h_1 = factor(df$h_1, ordered = TRUE)
df$m_2 = factor(df$m_2, ordered = TRUE)
df$s_3 = factor(df$s_3, ordered = TRUE)
a = Variable$new(m, "a", domain = c(hrs, mins, secs))
# set records
a$records = df
> a$isValid()
[1] TRUE

In this example, we create a large variable (31,536,000 records and 8880 unique domain elements. We mimic data that is labeled for every second in one year) and assign it to a variable with a$records. GAMS Transfer R requires that all domain columns must be ordered factors. The records setter function does very little work other than checking if the object being set is a data frame. This places more responsibility on the user to create a data frame that complies with the standard format. In Example #1, we take care to properly reference the factor from the domain sets and ensure that the symbol a is valid with a$isValid() = TRUE.

As with Set and Parameters, users can use the $isValid(verbose=TRUE) method to debug any structural issues.

Equation

Equation Constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name string Name of symbol Yes -
type string Type of equation being created [eq (or E/e), geq (or G/g), leq (or L/l), nonbinding (or N/n), external (or X/x)] No free
domain list, vector List, vector of domains given either as a string ("*" for universe set) or as a reference to a Set object, an empty domain list will create a scalar equation No NULL
records many Symbol records No NULL
domainForwarding logical Flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) No FALSE
description string Description of symbol No ""

Equation Fields
Field Description Type
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domain list of domains given either as string (* for universe set) or as reference to the Set/Alias object list
domainForwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) logical
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
refContainer reference to the Container that the symbol belongs to Container
shape a vector describing the array dimensions if records were converted with $toDense() vector
summary output a list of only the metadata list
type string type of equation list

Equation Methods
Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol uels (character), dimension=NULL NULL
countDomainViolations returns the number of domain violations for the symbol - numeric
countDuplicateRecords returns the number of duplicate records for the symbol - numeric
countEps total number of SpecialValues$EPS across all columns columns = "level" integer
countNA total number of SpecialValues[["NA"]] across all columns columns = "level" integer
countNegInf total number of SpecialValues$NEGINF across all columns columns = "level" integer
countPosInf total number of SpecialValues$POSINF across all columns columns = "level" integer
countUndef total number of SpecialValues$UNDEF across all columns columns = "level" integer
dropDomainViolations drops domain violations for the symbol - -
dropDuplicateRecords drops duplicate records for the Symbol keep= first/last -
findDomainViolations get the index of records that contain any domain violations - data.frame
getCardinality get the full cartesian product of the domain - integer
getDomainViolations gets domain violations for the symbol - list of DomainViolation
getMaxValue get the maximum value across all columns columns = "level" numeric
getMaxAbsValue get the maximum absolute value across all columns columns = "level" numeric
getMinValue get the minimum value iacross all columns columns = "level" numeric
getMeanValue get the mean value across all columns columns = "level" numeric
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
getUELs returns the UELs used in the Symbol dimension = NULL, codes = NULL, ignoreUnused=FALSE character vector
hasDomainViolations returns TRUE if the symbol contains domain violations, FALSE otherwise - logical
hasDuplicateRecords returns TRUE if the symbol contains duplicate records, FALSE otherwise - logical
isValid checks if the symbol is in a valid format, throw exceptions if verbose=TRUE, recheck a symbol if force=TRUE verbose=FALSE
force=FALSE
logical
removeUELs removes UELs from the symbol uels=NULL (character), dimension=NULL NULL
renameUELs renames UELs in the symbol uels(character of same length as current UELs or named character vector), dimension=NULL, allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL
setRecords main convenience method to set standard data frame records records (many types) NULL
setUELs sets UELs for the Symbol uels(character), dimension, rename=FALSE NULL
toDense convert symbol to a dense matrix or array format column = "value" array or matrix
whereMax find the row number in records data frame with a maximum value (return the first instance only) column="level" integer
whereMaxAbs find the row number in records data frame with a maximum absolute value (return the first instance only) column="level" integer
whereMin find the row number in records data frame with a minimum value (return the first instance only) column="level" integer

Adding Equation Records

Three possibilities exist to assign symbol records to an equation: We show a few examples of ways to create differently structured equations:

Example #1 - Create a GAMS scalar equation
m = Container$new()
pi = Equation$new(m, "pi", records = data.frame(level = 3.14159))
# NOTE: the above syntax is equivalent to -
# pi = Equation$new(m, "pi", "free")
# pi$setRecords(data.frame(level = 3.14159))
# NOTE: the above syntax is also equivalent to -
# m$addEquation("pi", "free", records=data.frame(level = 3.14159))
> pi$records
level marginal lower upper scale
1 3.14159 0 -Inf Inf 1
Example #2 - Create a 2D positive equation, specifying no numerical data
m = Container$new()
e = Equation$new(m, "e", "eq", c("*", "*"), records =
data.frame(c("seattle", "chicago"), c("san-diego", c("madison"))))
> e$records
uni_1 uni_2 level marginal lower upper scale
1 seattle san-diego 0 0 -Inf Inf 1
2 chicago madison 0 0 -Inf Inf 1
Example #3 - Create a 2D equation (defined over a set) from a matrix
m = Container$new()
i = Set$new(m, "i", "*", records = paste0("i", 1:5))
j = Set$new(m, "j", "*", records = paste0("j", 1:5))
# creating records for parameter a
ij = list(i_1 = paste0("i", 1:5), j_2 = paste0("j", 1:5))
df = rev(expand.grid(rev(ij)))
df$value = 1:25
a = Parameter$new(m, "a", c(i, j), records = df)
# create a free variable and set the level and marginal attributes from matrices
e = Equation$new(m, "e", "nonbinding", domain = c(i, j), records = list(level = a$toDense(), marginal = a$toDense()))
> e$records
i_1 j_2 level marginal lower upper scale
1 i1 j1 1 1 -Inf Inf 1
2 i2 j1 6 6 -Inf Inf 1
3 i3 j1 11 11 -Inf Inf 1
4 i4 j1 16 16 -Inf Inf 1
5 i5 j1 21 21 -Inf Inf 1
6 i1 j2 2 2 -Inf Inf 1
7 i2 j2 7 7 -Inf Inf 1
8 i3 j2 12 12 -Inf Inf 1
9 i4 j2 17 17 -Inf Inf 1
10 i5 j2 22 22 -Inf Inf 1
11 i1 j3 3 3 -Inf Inf 1
12 i2 j3 8 8 -Inf Inf 1
13 i3 j3 13 13 -Inf Inf 1
14 i4 j3 18 18 -Inf Inf 1
15 i5 j3 23 23 -Inf Inf 1
16 i1 j4 4 4 -Inf Inf 1
17 i2 j4 9 9 -Inf Inf 1
18 i3 j4 14 14 -Inf Inf 1
19 i4 j4 19 19 -Inf Inf 1
20 i5 j4 24 24 -Inf Inf 1
21 i1 j5 5 5 -Inf Inf 1
22 i2 j5 10 10 -Inf Inf 1
23 i3 j5 15 15 -Inf Inf 1
24 i4 j5 20 20 -Inf Inf 1
25 i5 j5 25 25 -Inf Inf 1
# if not specified, the toDense() method will convert the level values to a matrix
> e$toDense()
[,1] [,2] [,3] [,4] [,5]
[1,] 1 2 3 4 5
[2,] 6 7 8 9 10
[3,] 11 12 13 14 15
[4,] 16 17 18 19 20
[5,] 21 22 23 24 25

As with sets, parameters, and variables the primary advantage of the setRecords method is that GAMS Transfer will convert many different (and convenient) data types into the standard data format (data frame). Users that require higher performance will want to directly pass the Container a reference to a valid data frame, thereby skipping some of these computational steps. This places more burden on the user to pass the data in a valid standard form, but it speeds the records setting process. In this section, we walk the user through an example of how to set records directly.

Example #4 - Correctly set records (directly)
df = data.frame(h_1 = paste0("h", 1:8760), m_2 = paste0("m", 1:60),
s_3 = paste0("s", 1:60))
df$level = runif(nrow(df), min = 0, max = 100)
df$marginal = 0.0
df$lower = SpecialValues$NEGINF
df$upper = SpecialValues$POSINF
df$scale = 1.0
m = Container$new()
hrs = Set$new(m, "h", records = unique(df$h_1))
mins = Set$new(m, "m", records = unique(df$m_2))
secs = Set$new(m, "s", records = unique(df$s_3))
df$h_1 = factor(df$h_1, ordered = TRUE)
df$m_2 = factor(df$m_2, ordered = TRUE)
df$s_3 = factor(df$s_3, ordered = TRUE)
a = Equation$new(m, "a", "eq", domain = c(hrs, mins, secs))
# set records
a$records = df
> a$isValid()
[1] TRUE

In this example, we create a large equation (31,536,000 records and 8880 unique domain elements. We mimic data that is labeled for every second in one year) and assign it to an equation with a$records. GAMS Transfer R requires that all domain columns must be ordered factors. The records setter function does very little work other than checking if the object being set is a data frame. This places more responsibility on the user to create a data frame that complies with the standard format. In Example #1, we take care to properly reference the factor from the domain sets and ensure that the symbol a is valid in the end with a$isValid() = TRUE.

As with sets, parameters, and variables, users can use the $isValid(verbose=TRUE) method to debug any structural issues.

Alias

Alias Constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name string Name of symbol Yes -
aliasWith Set object set object to create an alias for Yes -
Example - Creating an alias from a set

GAMS Transfer R only stores the reference to the parent set as part of the alias structure. Most properties that are called from an alias object simply point to the properties of the parent set (with the exception of refContainer, name, and aliasWith). It is possible to create an alias from another alias object. In this case, a recursive search will be performed to find the root parent set. This is the set that will ultimately be stored as the aliasWith field. We can see this behavior in the following example:

m = Container$new()
i = Set$new(m, "i", records = paste0("i", 1:5))
ip = Alias$new(m, "ip", i)
ipp = Alias$new(m, "ipp", ip)
> ip$aliasWith$name
[1] "i"
> ipp$aliasWith$name
[1] "i"

Alias Fields
Field Description Type
aliasWith aliased object Set
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domainForwarding flag that forces set elements to be recursively included in all parent sets (i.e., implicit set growth) logical
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
isSingleton logical if symbol is a singleton set logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
refContainer reference to the Container that the symbol belongs to Container
summary output a list of only the metadata list

Alias Methods
Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol uels (character), dimension=NULL NULL
countDomainViolations returns the number of domain violations for the symbol - numeric
countDuplicateRecords returns the number of duplicate records for the symbol - numeric
dropDomainViolations drops domain violations for the symbol - -
dropDuplicateRecords drops duplicate records for the Symbol keep= first/last -
findDomainViolations get the index of records that contain any domain violations - data.frame
getCardinality get the full cartesian product of the domain - integer
getDomainViolations gets domain violations for the symbol - list of DomainViolation
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
getUELs returns the UELs used in the Symbol dimension = NULL, codes = NULL, ignoreUnused=FALSE character vector
hasDomainViolations returns TRUE if the symbol contains domain violations, FALSE otherwise - logical
hasDuplicateRecords returns TRUE if the symbol contains duplicate records, FALSE otherwise - logical
isValid checks if the symbol is in a valid format, throw exceptions if verbose=TRUE, recheck a symbol if force=TRUE verbose=FALSE
force=FALSE
logical
removeUELs removes UELs from the symbol uels=NULL (character), dimension=NULL NULL
renameUELs renames UELs in the symbol uels(character of same length as current UELs or named character vector), dimension=NULL, allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL
setRecords main convenience method to set standard data frame formatted records records (string vector, list, data frame) NULL
setUELs sets UELs for the Symbol uels(character), dimension, rename=FALSE NULL

Adding Alias Records

The linked structure of Aliases offers some unique opportunities to access some of the setter functionality of the parent set. Specifically, GAMS Transfer allows the user to change the domain, description, dimension, and records of the underlying parent set as a shorthand notation. We can see this behavior if we look at a modified Example #1 from adding set records.

Example - Creating set records through an alias link
m = Container$new()
i = Set$new(m, "i")
ip = Alias$new(m, "ip", i)
ip$description = "adding new descriptive set text"
ip$domain = c("*", "*")
ij = list(paste0("i", 1:3), paste0("j", 1:3))
ip$setRecords(rev(expand.grid(rev(ij))))
> i$domain
[[1]]
[1] "*"
[[2]]
[1] "*"
> i$records
uni_1 uni_2 element_text
1 i1 j1
2 i1 j2
3 i1 j3
4 i2 j1
5 i2 j2
6 i2 j3
7 i3 j1
8 i3 j2
9 i3 j3
Note
An alias $isValid()=TRUE when the underlying parent set is also valid. If the parent set is removed from the Container the alias will no longer be valid.

UniverseAlias

UniverseAlias Constructor
Argument Type Description Required Default
container Container A reference to the Container object that the symbol is being added to Yes -
name string Name of symbol Yes -
Example - Creating an alias for the Universe
In GAMS it is possible to create aliases to the universe (i.e., the entire list of UELs) with the syntax:
set i / i1, i2 /;
alias(h,*);
set j / j1, j2 /;

GAMS Transfer R allows creating aliases to the universe set.

In this small example, h would be associated with all four UELs (i1, i2, j1 and j2) even though set j was defined after the alias declaration. GAMS Transfer mimics this behavior with the UniverseAlias class. Internally, the records attribute will always call the <Container>$getUELs(). The UniverseAlias class is fundamentally different from the Alias class because it does not point to a parent set object. It is not possible to perform operations (like setRecords or findDomainViolations) on the parent set through a UniverseAlias (because there is no parent set object). This means that a UniverseAlias can be created by only defining the symbol name. We can see this behavior in the following example:

m = Container$new()
i = Set$new(m, "i", records = c("i1", "i2"))
h = UniverseAlias$new(m, "h")
j = Set$new(m, "j", records = c("j1","j2"))
> m$data
$i
GAMS Transfer: R6 object of class Set. Use i$summary for details
$h
GAMS Transfer: R6 object of class UniverseAlias.
Use h$summary for details
$j
GAMS Transfer: R6 object of class Set. Use j$summary for details
> h$records
1 i1
2 i2
3 j1
4 j2
Note
Unlike other sets, the universe does not hold on to set element_text, thus the returned data frame for the UniverseAlias will only have one column.

UniverseAlias Fields
Field Description Type
aliasWith always "*" string
description always Aliased with * string
dimension always 1 integer
domainLabels always "*" character vector
domainNames always "*" character vector
domainType always none string
isSingleton always FALSE logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
refContainer reference to the Container that the symbol belongs to Container
summary output a list of only the metadata list

UniverseAlias Methods
Method Description Arguments/Defaults Returns
getCardinality returns nrow(self$records) - integer
getSparsity always 0 - numeric
getUELs returns the UELs from the Container. Returns only UELs in the data if ignoreUnused=TRUE, otherwise returns all UELs ignoreUnused=FALSE character vector
isValid checks if the symbol is in a valid format, throw exceptions if verbose=TRUE, recheck a symbol if force=TRUE verbose=FALSE
force=FALSE
logical

ConstContainer Symbols

The symbols in the ConstContainer do not contain a reference to any other symbols or container and fields of the symbols in the ConstContainer cannot be modified. This results in several key differences between container symbols and ConstContainer symbols. Unlike container symbols, user should never instantiate ConstContainer symbol objects directly.

ConstContainer Set Fields
Field Description Type
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
isSingleton logical if symbol is a singleton set logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
summary output a list of only the metadata list

ConstContainer Set Methods
Method Description Arguments/Defaults Returns
getCardinality get the full cartesian product of the domain - integer
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric

ConstContainer Parameter Fields
Field Description Type
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
isScalar TRUE if the length(self$domain) = 0 logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
summary output a list of only the metadata list

ConstContainer Parameter Methods
Method Description Arguments/Defaults Returns
countEps total number of SpecialValues$EPS in value column - integer
countNA total number of SpecialValues[["NA"]] in value column - integer
countNegInf total number of SpecialValues$NEGINF in value column - integer
countPosInf total number of SpecialValues$POSINF in value column - integer
countUndef total number of SpecialValues$UNDEF in value column - integer
getCardinality get the full cartesian product of the domain - integer
getMaxValue get the maximum value in value column - numeric
getMaxAbsValue get the maximum absolute value in value column - numeric
getMeanValue get the mean value in value column - numeric
getMinValue get the minimum value in value column - numeric
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
whereMax find the row number in records data frame with a maximum value (return the first instance only) - integer
whereMaxAbs find the row number in records data frame with a maximum absolute value (return the first instance only) - integer
whereMin find the row number in records data frame with a minimum value (return the first instance only) - integer

ConstContainer Variable Fields
Field Description Type
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
summary output a list of only the metadata list
type string type of variable list

ConstContainer Variable Methods
Method Description Arguments/Defaults Returns
countEps total number of SpecialValues$EPS across all columns columns = "level" integer
countNA total number of SpecialValues[["NA"]] across all columns columns = "level" integer
countNegInf total number of SpecialValues$NEGINF across all columns columns = "level" integer
countPosInf total number of SpecialValues$POSINF across all columns columns = "level" integer
countUndef total number of SpecialValues$UNDEF across all columns columns = "level" integer
getCardinality get the full cartesian product of the domain - integer
getMaxValue get the maximum value across all columns columns = "level" numeric
getMaxAbsValue get the maximum absolute value across all columns columns = "level" numeric
getMeanValue get the mean value across all columns columns = "level" numeric
getMinValue get the minimum value across all columns columns = "level" numeric
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
whereMax find the row number in records data frame with a maximum value (return the first instance only) column="level" integer
whereMaxAbs find the row number in records data frame with a maximum absolute value (return the first instance only) column="level" integer
whereMin find the row number in records data frame with a minimum value (return the first instance only) column="level" integer

ConstContainer Equation Fields
Field Description Type
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
summary output a list of only the metadata list
type string type of equation list

ConstContainer Equation Methods
Method Description Arguments/Defaults Returns
countEps total number of SpecialValues$EPS across all columns columns = "level" integer
countNA total number of SpecialValues[["NA"]] across all columns columns = "level" integer
countNegInf total number of SpecialValues$NEGINF across all columns columns = "level" integer
countPosInf total number of SpecialValues$POSINF across all columns columns = "level" integer
countUndef total number of SpecialValues$UNDEF across all columns columns = "level" integer
getCardinality get the full cartesian product of the domain - integer
getMaxValue get the maximum value across all columns columns = "level" numeric
getMaxAbsValue get the maximum absolute value across all columns columns = "level" numeric
getMeanValue get the mean value across all columns columns = "level" numeric
getMinValue get the minimum value iacross all columns columns = "level" numeric
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric
whereMax find the row number in records data frame with a maximum value (return the first instance only) column="level" integer
whereMaxAbs find the row number in records data frame with a maximum absolute value (return the first instance only) column="level" integer
whereMin find the row number in records data frame with a minimum value (return the first instance only) column="level" integer

ConstContainer Alias Fields
Field Description Type
aliasWith aliased object or name Set or string
description description of symbol string
dimension dimension of symbol, setting dimension is a shorthand notation to set domain to a list of size n containing "*" integer
domainLabels column headings for the records data frame list of string
domainNames string version of domain names list of string
domainType none, relaxed or regular depending on state of domain links string
isSingleton logical if symbol is a singleton set logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
summary output a list of only the metadata list

ConstContainer Alias Methods
Method Description Arguments/Defaults Returns
getCardinality get the full cartesian product of the domain - integer
getSparsity get the sparsity of the symbol w.r.t the cardinality - numeric

ConstContainer UniverseAlias Fields
Field Description Type
aliasWith always "*" string
description always Aliased with * string
dimension always 1 integer
domainLabels always "*" character vector
domainNames always "*" character vector
domainType always none string
isSingleton always FALSE logical
name name of symbol string
numberRecords number of symbol records (i.e., returns nrow(self$records) if not NULL) integer
records the main symbol records data frame
summary output a list of only the metadata list

ConstContainer UniverseAlias Methods
Method Description Arguments/Defaults Returns
getCardinality returns nrow(self$records) - integer
getSparsity always 0 - numeric

Additional Features

Validating Data

GAMS Transfer R requires that the records for all symbols exist in a standard format (Standard Data Formats) in order for them to be understood by the Container. It is possible that the data could end up in a state that is inconsistent with the standard format (especially if setting symbol attributes directly). GAMS Transfer R includes the $isValid() method in order to determine if a symbol is valid. This method returns a logical. This method does not guarantee that a symbol will be successfully written to GDX. Other data errors (duplicate records, long UEL names, or domain violations) could exist that are not tested in $isValid(). For example, we create two valid sets and then check them with $isValid() to be sure.

It is possible to run $isValid() on both the Container as well as the symbol object. The Container method $isValid() will also return FALSE if there are any invalid symbols in the Container object. In addition, the Container$isValid() method also detects broken Container references in symbols and inconsistent symbol naming between the Container$data field and the symbol objects.

Example (valid data)
m = Container$new()
i = Set$new(m, "i", records = c("seattle", "san-diego", "washington_dc"))
j = Set$new(m, "j", i, records = c("san-diego", "washington_dc"))
> i$isValid()
[1] TRUE
> j$isValid()
[1] TRUE
> m$isValid()
[1] TRUE

Now we create some data that is invalid due to incorrect column names in the records for set j.

m = Container$new()
i = Set$new(m, "i", records = c("seattle", "san-diego", "washington_dc"))
j = Set$new(m, "j", i)
j$records = data.frame(cities=c("grayslake", "washington_dc"))
> i$isValid()
[1] TRUE
> j$isValid()
[1] FALSE
> m$isValid()
[1] FALSE

In this example, we use records field of the symbol j to set the records. As mentioned in setting the records, when setting records directly, users must adhere to Standard Data Formats. In this example, symbol j does not have the correct number of columns. Moreover, the column headings do not follow the convention (should be "i_1"). The user can get more detailed error reporting if the verbose argument is set to TRUE. For example:

> j$isValid(verbose=TRUE)
Symbol 'records' does not have the correct number of columns (<symbol dimension> + 1)
[1] FALSE

The $isValid() method checks:

  1. If the symbol belongs to a Container
  2. If all domain set symbols exist in the Container
  3. If all domain set symbols objects are valid
  4. If records are a data frame (or None)
  5. The shape of the records is congruent with the dimensionality of the symbol
  6. If records column headings are in standard format
  7. That all data columns are type numerical
  8. To make sure that all domain categories are type string
Note
Calling $isValid() too often may have a significant impact on the performance.

Domain Forwarding

GAMS includes the ability to define sets directly from data using the implicit set notation (see: Implicit Set Definition (or: Domain Defining Symbol Declarations)). This notation has an analogue in GAMS Transfer R called domainForwarding.

Note
It is possible to recursively update a subset tree in GAMS Transfer R.

Domain forwarding is available as an argument to all symbol object constructors; the user would simply need to pass domainForwarding=TRUE.

In this example, we have raw data that is in the dist data frame, and we want to send the domain information into the i and j sets. We take care to pass the set objects as the domain for parameter c.

m = Container$new()
i = Set$new(m, "i")
j = Set$new(m, "j")
dist = data.frame(
from = c("seattle", "seattle", "seattle",
"san-diego", "san-diego", "san-diego"),
to = c("new-york", "chicago", "topeka",
"new-york", "chicago", "topeka"),
thousand_miles = c(2.5, 1.7, 1.8, 2.5, 1.8, 1.4)
)
c = Parameter$new(m, "c", c(i, j), records = dist, domainForwarding = TRUE)
> i$records
uni_1 element_text
1 seattle
4 san-diego
> j$records
uni_1 element_text
1 new-york
2 chicago
3 topeka
> c$records
i_1 j_2 value
1 seattle new-york 2.5
2 seattle chicago 1.7
3 seattle topeka 1.8
4 san-diego new-york 2.5
5 san-diego chicago 1.8
6 san-diego topeka 1.4
Note
The element order in the sets i and j mirrors that in the raw data.

In this example, we show that domain forwarding will also work recursively to update the entire set lineage. The domain forwarding occurs at the creation of every symbol object. The correct order of elements in set i is (z, a, b, c) because the records from j are forwarded first, and then the records from k are propagated through (back to i).

m = Container$new()
i = Set$new(m, "i")
j = Set$new(m, "j", i, records = c("z"), domainForwarding = TRUE)
k = Set$new(m, "k", j, records = c("a", "b", "c"), domainForwarding = TRUE)
> i$records
uni_1 element_text
1 z
2 a
3 b
4 c
> j$records
i_1 element_text
1 z
2 a
3 b
4 c
> k$records
j_1 element_text
1 a
2 b
3 c

Describing Data

The methods describeSets, describeParameters, describeVariables, and describeEquations allow the user to get a summary view of key data statistics. The returned data frame aggregates the output for a number of other methods (depending on symbol type). A description of each Container method is provided in the following subsections:

describeSets

Argument Type Description Required Default
symbols list, string A list of sets in the Container to include in the output. describeSets will include aliases if they are explicitly passed by the user. No NULL (if NULL specified, will assume all sets)

Returns: data frame

The following table includes a short description of the column headings in the return.

Field / Statistic Description
name name of the symbol
isAlias logical if the symbol is an Alias
isSingleton logical if the set is a singleton set
domain domain labels for the symbol
domainType none, relaxed or regular depending on the symbol state
dim dimension
numberRecs number of records in the symbol
cardinality cartesian product of the domain information
sparsity 1 - numberRecs/cardinality
Example #1
m = Container$new("trnsport.gdx")
> m$describeSets()
name isAlias isSingleton domain domainType dim numberRecs cardinality
1 i FALSE FALSE * none 1 2 NA
2 j FALSE FALSE * none 1 3 NA
sparsity
1 NA
2 NA
Example #2 – with aliases
m = Container$new()
i = Set$new(m, "i", records=paste0("i", 1:10))
j = Set$new(m, "j", records=paste0("j", 1:10))
ip = Alias$new(m, "ip", i)
jp = Alias$new(m, "jp", j)
> m$describeSets()
name isSingleton domain domainType dim numberRecs cardinality sparsity
1 i FALSE * none 1 10 NA NA
2 j FALSE * none 1 10 NA NA
> m$describeSets(append(m$listSets(), m$listAliases()))
name isSingleton domain domainType dim numberRecs cardinality sparsity
1 i FALSE * none 1 10 NA NA
3 ip FALSE * none 1 10 NA NA
2 j FALSE * none 1 10 NA NA
4 jp FALSE * none 1 10 NA NA

describeParameters

Argument Type Description Required Default
symbols list, string A list of parameters in the Container to include in the output No NULL (if NULL specified, will assume all parameters)

Returns: data frame

The following table includes a short description of the column headings in the return.

Field / Statistic Description
name name of the symbol
isScalar logical if the symbol is a scalar (i.e., dimension = 0)
domain domain labels for the symbol
domainType none, relaxed or regular depending on the symbol state
dim dimension
numRecs number of records in the symbol
minValue min value in data
meanValue mean value in data
maxValue max value in data
whereMin row number min value (if multiple, returns only first occurrence)
whereMax row number of max value (if multiple, returns only first occurrence)
countEps number of SpecialValues$EPS in data
countNa number of SpecialValues[["NA"]] in data
countUndef number of SpecialValues$UNDEF in data
cardinality cartesian product of the domain information
sparsity 1 - numRecs/cardinality
m = Container$new("trnsport.gdx")
> m$describeParameters()
name isScalar domain domainType dim numRecs minValue meanValue maxValue
1 a FALSE i regular 1 2 350.000 475.0000 600.000
2 b FALSE j regular 1 3 275.000 300.0000 325.000
5 c FALSE i j regular 2 6 0.126 0.1755 0.225
3 d FALSE i j regular 2 6 1.400 1.9500 2.500
4 f TRUE NA none 0 1 90.000 90.0000 90.000
whereMin whereMax countEps countNa countUndef cardinality sparsity
1 1 2 0 0 0 2 0
2 3 1 0 0 0 3 0
5 6 1 0 0 0 6 0
3 6 1 0 0 0 6 0
4 1 1 0 0 0 NA NA

describeVariables

Argument Type Description Required Default
symbols list, string A list of variables in the Container to include in the output No NULL (if NULL specified, will assume all variables)

Returns: data frame

The following table includes a short description of the column headings in the return.

Field / Statistic Description
name name of the symbol
type type of variable (i.e., binary,integer,positive,negative,free,sos1,sos2,semicont,semiint)
domain domain labels for the symbol
domainType none, relaxed or regular depending on the symbol state
dim dimension
numRecs number of records in the symbol
cardinality cartesian product of the domain information
sparsity 1 - numRecs/cardinality
minLevel min value in the level
meanLevel mean value in the level
maxLevel max value in the level
whereMaxAbsLevel row number of max(abs(level)) in data
countEpsLevel number of SpecialValues$EPS in level
minMarginal min value in the marginal
meanMarginal mean value in the marginal
maxMarginal max value in the marginal
whereMaxAbsMarginal row number of max(abs(marginal)) in data
countEpsMarginal number of SpecialValues$EPS in marginal
m = Container$new("trnsport.gdx")
> m$describeVariables()
name type domain domainType dim numRecs cardinality sparsity minLevel
1 x positive i j regular 2 6 6 0 0.000
2 z free NA none 0 1 NA NA 153.675
meanLevel maxLevel whereMaxAbsLevel countEpsLevel minMarginal meanMarginal
1 150.000 300.000 2 0 0 0.0075
2 153.675 153.675 1 0 0 0.0000
maxMarginal whereMaxAbsMarginal countEpsMarginal
1 0.036 3 1
2 0.000 1 1

describeEquations

Argument Type Description Required Default
symbols list, string A list of equations in the Container to include in the output No NULL (if NULL specified, will assume all variables)

Returns: data frame

The following table includes a short description of the column headings in the return.

Field / Statistic Description
name name of the symbol
type type of variable (i.e., binary,integer,positive,negative,free,sos1,sos2,semicont,semiint)
domain domain labels for the symbol
domainType none, relaxed or regular depending on the symbol state
dim dimension
numRecs number of records in the symbol
cardinality cartesian product of the domain information
sparsity 1 - numRecs/cardinality
minLevel min value in the level
meanLevel mean value in the level
maxLevel max value in the level
whereMaxAbsLevel row number of max(abs(level)) in data
countEpsLevel number of SpecialValues$EPS in level
minMarginal min value in the marginal
meanMarginal mean value in the marginal
maxMarginal max value in the marginal
whereMaxAbsMarginal row number of max(abs(marginal)) in data
countEpsMarginal number of SpecialValues$EPS in marginal
m = Container$new("trnsport.gdx")
> m$describeEquations()
name type domain domainType dim numRecs cardinality sparsity minLevel
1 cost eq NA none 0 1 NA NA 0
3 demand geq j regular 1 3 3 0 275
2 supply leq i regular 1 2 2 0 350
meanLevel maxLevel whereMaxAbsLevel countEpsLevel minMarginal meanMarginal
1 0 0 1 1 1.000 1.000
3 300 325 1 0 0.126 0.168
2 450 550 2 0 0.000 0.000
maxMarginal whereMaxAbsMarginal countEpsMarginal
1 1.000 1 0
3 0.225 1 0
2 0.000 1 1

describeAliases

Argument Type Description Required Default
symbols list, string A list of aliases in the Container to include in the output. No NULL (if NULL specified, will assume all aliases)

Returns: data frame

The following table includes a short description of the column headings in the return.

Field / Statistic Description
name name of the symbol
aliasWith name of the parent set
isSingleton logical if an alias of a singleton set
domain domain labels for the symbol
domainType none, relaxed or regular depending on the symbol state
dim dimension
numberRecs number of records in the symbol
cardinality cartesian product of the domain information
sparsity 1 - numberRecs/cardinality
Example #1
m = Container$new()
i = Set$new(m, "i", records = paste0("i",1:5))
j = Set$new(m, "j", records = paste0("j",1:10))
ip = Alias$new(m, "ip", i)
ipp = Alias$new(m, "ipp", ip)
jp = Alias$new(m, "jp", j)
> m$describeAliases()
name aliasWith isSingleton domain domainType dim numberRecs cardinality
1 ip i FALSE * none 1 5 NA
2 ipp i FALSE * none 1 5 NA
3 jp j FALSE * none 1 10 NA
sparsity
1 NA
2 NA
3 NA

Matrix Generation

GAMS Transfer R stores data in a "flat" format, that is, one record entry per data frame row. However, it is often necessary to convert this data format into a matrix/array format. GAMS Transfer R enables users to do this with relative ease using the toDense symbol methods This method will return a dense N-dimensional array (matrix for 2-Dimensions) with each dimension corresponding to the GAMS symbol dimension; it is possible to output an array up to 20 dimensions (a GAMS limit).

Example (1D data w/o domain linking (i.e., a relaxed domain))
m = Container$new()
a = Parameter$new(m, "a", "i", records = data.frame(c("a","c"), c(1,3)))
> i$records
uni_1 element_text
1 a
2 b
3 c
4 d
> a$toDense()
[1] 1 3

Note that the parameter a is not linked to another symbol, so when converting to a matrix, the indexing is referenced to the data structure in a$records. Defining a sparse parameter a over a set i allows us to extract information from the i domain and construct a very different dense matrix, as the following example shows:

m = Container$new()
i = Set$new(m, "i", records = c("a", "b", "c", "d"))
a = Parameter$new(m, "a", i, records = data.frame(c("a","c"), c(1,3)))
> a$toDense()
[1] 1 0 3 0
Example (2D data w/ domain linking)
m = Container$new()
i = Set$new(m, "i", records = c("a", "b", "c", "d"))
a = Parameter$new(m, "a", c(i, i), records =
data.frame(c("a","c"), c("a","c"), c(1,3)))
> i$records
uni_1 element_text
1 a
2 b
3 c
4 d
> a$records
i_1 i_2 value
1 a a 1
2 c c 3
> a$toDense()
[,1] [,2] [,3] [,4]
[1,] 1 0 0 0
[2,] 0 0 0 0
[3,] 0 0 3 0
[4,] 0 0 0 0

The Universe Set

A Unique Element (UEL) is an (i,s) pair where i is an identification number for a string s. A list of all UELs is also known as 'the universe' or 'the universe set'. GAMS uses UELs to efficiently store domain entries of a record by storing the UEL ID i of a domain entry instead of the actual string s. This avoids storing the same string multiple times. The concept of UELs also exists in R and is called a "factor". GAMS Transfer R leverages these types in order to efficiently store strings and enable domain checking within the R environment.

Each domain column in a data frame can be a factor, the effect is that each symbol maintains its own list of UELs per dimension. R lets the user choose if levels in a factor should be regarded as ordered. GAMS Transfer R relies exclusively on ordered factors. By using ordered factors, GAMS Transfer R will order the UELs such that elements appear in the order in which they appeared in the data (which is how GAMS defines the UELs). GAMSTransfer allows the user to reorder the UELs with the uelPriority argument in the $write() method.

GAMS Transfer R does not keep track of the UELs separately from other symbols in the Container, it will be created internal to the $write() method and is based on the order in which data is added to the container. The user can access the current state of UELs with the $getUELs() container method. For example, we set a two dimensional set:

m = Container$new()
j = Set$new(m, "j", c("*", "*"), records = data.frame(c("i1","i2"), c("j1","j2")))
> j$records
uni_1 uni_2 element_text
1 i1 j1
2 i2 j2
> m$getUELs()
[1] "i1" "i2" "j1" "j2"

Customize The Universe Set

GAMS Transfer R allows the users to customize the Universe Set with the help of Symbol UELs methods shown in the following table.

Method Description Arguments/Defaults Returns
addUELs adds UELs to the symbol uels (character), dimension=NULL NULL
getUELs returns the UELs used in the Symbol dimension = NULL, codes = NULL, ignoreUnused=FALSE character vector
removeUELs removes UELs from the symbol uels=NULL (character), dimension=NULL NULL
renameUELs renames UELs in the symbol uels(character of same length as current UELs or named character vector), dimension=NULL, allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL
setUELs sets UELs for the Symbol uels(character), dimension, rename=FALSE NULL
Note
Symbols should be valid in order to use these methods.
when dimension argument is optional, the method is applied to all dimensions

Some of these methods are also available as Container methods as shown in the following table. Container methods internally call the corresponding Symbol methods.

Method Description Arguments/Defaults Returns
getUELs returns the UELs used in the Container symbols=NULL (character), ignoreUnused = FALSE character vector
removeUELs removes uels from all symbols the Container uels=NULL (character) NULL
renameUELs renames uels in the Container uels (named character), allowMerge=FALSE (logical). If allowMerge = TRUE, the underlying integer mapping of a factor is allowed to change to offer additional data flexibility NULL

getUELs Examples
m = Container$new()
i = Set$new(m, "i", records=c("i1","i2","i3"))
j = Set$new(m, "j", records=c("j1","j2","j3"))
a = Parameter$new(m, "a", c(i, j), records=data.frame(paste0("i",1:4), paste0("j",1:4), 1:4))
> i$getUELs()
[1] "i1" "i2" "i3"
> m$getUELs()
[1] "i1" "i2" "i3" "j1" "j2" "j3" "i4" "j4"
> m$getUELs("j")
[1] "j1" "j2" "j3"

addUELs Examples
m = Container$new()
i = Set$new(m, "i", records=c("i1","i2","i3"))
j = Set$new(m, "j", records=c("j1","j2","j3"))
a = Parameter$new(m, "a", c(i, j), records=data.frame(paste0("i",1:3), paste0("j",1:3), 1:3))
i$addUELs("ham")
a$addUELs("and", 1)
a$addUELs("cheese", 2)
> i$getUELs()
[1] "i1" "i2" "i3" "ham"
> a$getUELs()
[1] "i1" "i2" "i3" "and" "j1" "j2" "j3" "cheese"

In this example we have added three new (unused) UELs: ham, and, cheese. These three UELs will now appear in the GAMS universe set (accessible with m$getUELs()). The addition of unused UELs does not impact the validity of the symbols (i.e., unused UELs will not trigger domain violations).

removeUELs Examples
m = Container$new()
i = Set$new(m, "i", records=c("i1","i2","i3"))
j = Set$new(m, "j", records=c("j1","j2","j3"))
a = Parameter$new(m, "a", c(i, j), records=data.frame(paste0("i",1:3), paste0("j",1:3), 1:3))
i$addUELs("ham")
a$addUELs("and", 1)
a$addUELs("cheese", 2)
# remove symbol UELs explicitly by dimension
i$removeUELs("ham", 1)
a$removeUELs("and", 1)
a$removeUELs(c("and", "cheese"), 2)
# remove symbol UELs for the entire symbol
i$removeUELs("ham")
a$removeUELs(c("and", "cheese"))
# remove ONLY unused UELs from each symbol, independently
i$removeUELs()
a$removeUELs()
# remove ONLY unused UELs from the entire container (all symbols)
m$removeUELs()
> m$getUELs()
[1] "i1" "i2" "i3" "j1" "j2" "j3"

If a user removes a UEL that appears in data, that data will be lost perminately. The domain label will be transformed into an NA as seen in this example:

m = Container$new()
i = Set$new(m, "i", records=c("i1","i2","i3"))
j = Set$new(m, "j", records=c("j1","j2","j3"))
a = Parameter$new(m, "a", c(i, j), records=data.frame(paste0("i",1:3), paste0("j",1:3), 1:3))
m$removeUELs("i1")
> i$records
uni_1 element_text
1 <NA>
2 i2
3 i3
> a$records
i_1 j_2 value
1 <NA> j1 1
2 i2 j2 2
3 i3 j3 3
Attention
A container cannot be written if there are NA entries in any of the domain columns (in any symbol). An error is thrown if there are missing domain labels.

renameUELs Examples
renameUELs is a method of all GAMS Symbol classes as well as the Container class. This method allows the user to rename UELs in a symbol dimension(s), over several symbols, or over the entire container. This method is handy when attempting to harmonize labeling schemes between data structures that originated from different sources. For example:
m = Container$new()
a = Parameter$new(m, "a", c("*","*"),
records = data.frame(c("WI","IL","WI"),
c("IL", "IN", "IN"), c(10, 12.5, 8.7)),
description = "shipment quantities")
b = Parameter$new(m, "b", c("*"),
records = data.frame(c("wisconsin","illinois","indiana"),
c(1.2, 1.7, 1.2)), description = "multipliers")

results in the following records:

> a$records
uni_1 uni_2 value
1 WI IL 10.0
2 IL IN 12.5
3 WI IN 8.7
> b$records
uni_1 value
1 wisconsin 1.2
2 illinois 1.7
3 indiana 1.2

However, two different data sources were used to generate the parameters a and b – one data source used the uppercase postal abbreviation of the state name and the other source used a lowercase full state name as the uniqe identifier. With the following syntax the user can harmonize to a mixed case postal code labeling scheme (without losing any of the original UEL ordering).

m$renameUELs(c("WI"="Wi", "IL"="Il", "IN"="In",
"wisconsin"="Wi", "illinois"="Il","indiana"="In"))

This results in the following records and the universe set:

> a$records
uni_1 uni_2 value
1 Wi Il 10.0
2 Il In 12.5
3 Wi In 8.7
> b$records
uni_1 value
1 Wi 1.2
2 Il 1.7
> m$getUELs()
[1] "Wi" "Il" "In"

reorderUELs Examples

reorderUELs is a method of all GAMS symbol classes. This method allows the user to reorder UELs of a specific symbol dimension. reorderUELs will not add/remove any UELs. For example:

m = Container$new()
i = Set$new(m, "i", records=c("i1","i2","i3"))
> i$getUELs()
[1] "i1" "i2" "i3"
> m$getUELs()
[1] "i1" "i2" "i3" "j1" "j2" "j3"

But perhaps we want to reorder the UELs i1, i2, i3 to i3, i2, i1.

i$reorderUELs(c("i3","i2","i1"))
> i$getUELs()
[1] "i3" "i2" "i1"
> i$records
uni_1 element_text
1 i1
2 i2
3 i3
Note
This example does not change the indexing scheme of the data frame. It only changes the underlying integer numbering scheme for the factor levels.
We can see this by looking at the levels:
> as.integer(i$records$uni_1)
[1] 3 2 1

setUELs Examples
setUELs is a method of all GAMS symbol classes. This method allows the user to create new UELs, rename UELs, and reorder UELs all in one method. For example:
m = Container$new()
i = Set$new(m, "i", records=c("i1","i2","i3"))

A user could accomplish a UEL reorder operation with setUELs:

> i$setUELs(c("i3","i2","i1"))
> i$getUELs()
[1] "i3" "i2" "i1"
> i$records
uni_1 element_text
1 i1
2 i2
3 i3

A user could accomplish a UEL reorder + add UELs operation with setUELs:

> i$setUELs(c("i3", "i2", "i1", "j1", "j2"))
> i$getUELs()
[1] "i3" "i2" "i1" "j1" "j2"
> i$records
uni_1 element_text
1 i1
2 i2
3 i3
> as.integer(i$records$uni_1)
[1] 3 2 1

A user could accomplish a UEL reorder + add + rename with setUELs:

> i$setUELs(c("j3", "j2", "j1", "ham", "cheese"), rename=TRUE)
> i$setUELs(c("j3", "j2", "j1", "ham", "cheese"), rename=TRUE)
> i$getUELs()
[1] "j3" "j2" "j1" "ham" "cheese"
> i$records
uni_1 element_text
1 j1
2 j2
3 j3
> as.integer(i$records$uni_1)
[1] 3 2 1
Note
This example does not change the indexing scheme of the data frame, but the rename=TRUE flag means that the records will get updated as if a renameUELs call had been made.

If a user wanted to set new UELs on top of this data, without renaming, they would need to be careful to include the current UELs in the UELs being set. It is possible to lose these labels if they are not included (which will prevent the data from being written to GDX).

m = Container$new()
i = Set$new(m, "i", records=c("i1","i2","i3"))
i$setUELs(c("j1", "i2", "j3", "ham", "cheese"))
> i$getUELs()
[1] "j1" "i2" "j3" "ham" "cheese"
> i$records
uni_1 element_text
1 <NA>
2 i2
3 <NA>

Removing Symbols

Removing symbols from a Container is easy when using the removeSymbols container method; this method accepts either a string or a list of string.

Attention
Once a symbol has been removed, it is possible to have hanging references as domain links in other symbols. The user will need to repair these other symbols with the proper domain links in order to avoid validity errors.

Reordering Symbols

In order to write the contents of the Container, it is required that the symbols are sorted such that, for example, a Set used as a domain of another symbol appears before that symbol. The Container will try to establish a valid ordering when writing the data. This type of situation could be encountered if the user is adding and removing many symbols (and perhaps rewriting symbols with the same name). Users should attempt to only add symbols to a Container once, and care must be taken when naming. The method reorderSymbols attempts to fix symbol ordering problems. The following example shows how this can occur:

Example Symbol reordering
m = Container$new()
i = Set$new(m, "i", records = paste0("i", 1:5))
j = Set$new(m, "j", i, records = paste0("i", 1:3))
> names(m$data)
[1] "i" "j"
# now we remove the set i and recreate the data
m$removeSymbols("i")
i = Set$new(m, "i", records = paste0("i", 1:5))
> m$isValid()
[1] FALSE

Since the link to i is broken, Set j is now invalid. The user has to manually set the domain again.

# fix the domain reference in the set j
j$domain = i
> names(m$data)
[1] "j" "i"

Now even though j is valid, the symbols are out of order. The order can be fixed as follows.

# calling reorderSymbols() will order the list properly,
# but the domain reference in j is now broken
m$reorderSymbols()
> names(m$data)
[1] "i" "j"
> m$isValid()
[1] TRUE

Domain Violations

Domain violations occur when a symbol uses other Sets as domain(s) – and is thus of domain type regular, see Symbol Domain – and uses a domain entry in its records that is not present in the corresponding referenced domain set. Such a domain violation will lead to a GDX error when writing the data.

For example, the symbol j in the following example contains domain violations because j is defined over domain set i and the records entry "grayslake" is not present in i.

m = Container$new()
i = Set$new(m, "i", records = c("seattle", "san-diego", "washington_dc"))
j = Set$new(m, "j", i)
records = data.frame(cities=c("grayslake", "washington_dc"))
j$setRecords(records)

Trying to write this container to a GDX file will fail. To ask for domain violations, call the method Symbol$getDomainViolations(). It returns a list of DomainViolation objects with respect to each dimension of the symbol. This list can then be used to resolve the domain violations.

> dv = j$getDomainViolations()
> dv
[[1]]
GAMS Transfer: DomainViolation with properties:
Symbol: j
dimension:
domain: i
violations: grayslake

The GAMS Transfer R feature of domain forwarding offers an automatic expansion of the domain set with the violated entries in order to eliminate domain violations.

Note
Checking for domain violations is not a part of Symbol$isValid() for performance reasons.
The method for automatically resolving the domain violations can be convenient, but it effectively disables domain checking, which is a valuable tool for error detection. We encourage to use domain forwarding as rarely as possible. The same holds for using relaxed domain informaiton when regular domain information would be possible.

GAMS Special Values

The GAMS system contains five special values: UNDEF (undefined), NA (not available), EPS (epsilon), +INF (positive infinity), -INF (negative infinity). These special values must be mapped to their R equivalents. GAMS Transfer R follows the following convention to generate the 1:1 mapping:

  • +INF is mapped to Inf
  • -INF is mapped to -Inf
  • EPS is mapped to -0.0 (mathematically identical to zero)
  • NA is mapped to a NA
  • UNDEF is mapped to NaN

GAMS Transfer R syntax is designed to quickly get data into a form that is usable in further analyses or visualization. The user does not need to remember these constants as they are provided within the class SpecialValues as SpecialValues$POSINF, SpecialValues$NEGINF, SpecialValues$EPS, SpecialValues[["NA"]], and SpecialValues$UNDEF. Some examples are shown below.

m = Container$new()
x = Parameter$new(
m, "x", c("*"),
records = data.frame(paste0("i", 1:6), c(1, SpecialValues[["POSINF"]],
SpecialValues[["NEGINF"]], SpecialValues[["EPS"]], SpecialValues[["NA"]],
SpecialValues[["UNDEF"]])),
description = "special values"
)

The following data frame for x would look like:

> x$records
uni_1 value
1 i1 1
2 i2 Inf
3 i3 -Inf
4 i4 0
5 i5 NA
6 i6 NaN

The user can now easily test for specific special values in the value column of the DataFrame (returns a logical vector):

> SpecialValues$isNA(x$records$value)
[1] FALSE FALSE FALSE FALSE TRUE FALSE
> SpecialValues$isNA(x$records$value)
[1] FALSE FALSE FALSE FALSE TRUE FALSE
> SpecialValues$isEps(SpecialValues$EPS)
[1] TRUE
> SpecialValues$isPosInf(SpecialValues$POSINF)
[1] TRUE
> SpecialValues$isNegInf(SpecialValues$NEGINF)
[1] TRUE
> SpecialValues$isNA(SpecialValues$NEGINF)
SpecialValues$isNA
> SpecialValues$isNA(SpecialValues[["NA"]])
[1] TRUE
> SpecialValues$isUndef(SpecialValues$UNDEF)
[1] TRUE
> SpecialValues$isUndef(SpecialValues[["NA"]])
[1] FALSE
> SpecialValues$isNA(SpecialValues$UNDEF)
[1] FALSE
Note
The syntax SpecialValues$NA is not allowed in R. Therefore, to access NA, one has to use SpecialValues[["NA"]]. As shown in the example above, double bracket syntax works for other special values too.

Standard Data Formats

This section is meant to introduce the standard format that GAMS Transfer R expects for symbol records. It has already been mentioned that we store data as a data frame, but there is an assumed structure to the column headings and column types that will be important to understand. GAMS Transfer R includes convenience functions in order to ease the burden of converting data from a user-centric format to one that is understood by GAMS Transfer R. However, advanced users will want to convert their data first and add it directly to the Container.

Set Records Standard Format
All set records (including singleton sets) are stored as a data frame with n number of columns, where n is the dimensionality of the symbol + 1. The first n-1 columns include the domain elements while the last column includes the set element explanatory text. Records are organized such that there is one record per row.

The names of the domain columns follow a pattern of <domain_set_name>_<index_position>; a symbol dimension that is referenced to the universe is labeled uni_<index position>. The explanatory text column is called element_text and must take the last position in the data frame.

All domain columns must be factors and the element_text column must be a string type.

Some examples:

m = Container$new()
i = Set$new(m, "i", records=c("seattle", "san-diego"))
j = Set$new(m, "j", c(i, "*"), records=data.frame(c("seattle", "san-diego"), c("new-york", "st-louis")))
k = Set$new(m, "k", i, isSingleton=TRUE, records=c("seattle"))
> i$records
uni_1 element_text
1 seattle
2 san-diego
> j$records
i_1 uni_2 element_text
1 seattle new-york
2 san-diego st-louis
> k$records
i_1 element_text
1 seattle
Parameter Records Standard Format
All parameter records (including scalars) are stored as a data frame with n number of columns, where n is the dimensionality of the symbol + 1. The first n-1 columns include the domain elements while the last column includes the numerical value of the records. Records are organized such that there is one record per row. Scalar parameters have zero dimension, therefore they only have one column and one row.

The names of the domain columns follow a pattern of <domain_set_name>_<index_position>; a symbol dimension that is referenced to the universe is labeled uni_<index_position>. The value column is called value and must take the last position in the data frame.

All domain columns must be factors and the value column must be a numeric type. GAMS Transfer R requires that all the factor levels are of type string.

Some examples:

m = Container$new()
i = Set$new(m, "i", records=c("seattle", "san-diego"))
a = Parameter$new(
m, "a", "*", records=data.frame(c("seattle", "san-diego"), c(50, 100))
)
b = Parameter$new(
m,
"b",
c(i, "*"),
records= data.frame(c("seattle", "san-diego"),
c("new-york", "st-louis"), c(32.2, 123))
)
c = Parameter$new(m, "c", records=90)
> a$records
uni_1 value
1 seattle 50
2 san-diego 100
> b$records
i_1 uni_2 value
1 seattle new-york 32.2
2 san-diego st-louis 123.0
> c$records
value
1 90
Variable/Equation Records Standard Format

Variables and equations share the same standard data format. All records (including scalar variables/equations) are stored as a data frame with n number of columns, where n is the dimensionality of the symbol + 5. The first n-5 columns include the domain elements while the last five columns include the numerical values for different attributes of the records. Records are organized such that there is one record per row. Scalar variables/equations have zero dimension, therefore they have five columns and one row.

The names of the domain columns follow a pattern of <domain_set_name>_<index position>; a symbol dimension that is referenced to the universe is labeled uni_<index_position>. The attribute columns are called level, marginal, lower, upper, and scale. These attribute columns must appear in this order. Attributes that are not supplied by the user will be assigned the default GAMS values for that variable/equation type. It is possible to not pass any attributes, GAMS Transfer R would then simply assign default values to all attributes.

All domain columns must be factors and the attribute columns must be a numeric type. GAMS Transfer R requires that all the factor levels are of type string.

Some examples:

m = Container$new()
i = Set$new(m, "i", records=c("seattle", "san-diego"))
a = Variable$new(
m,
"a",
"free",
domain= i,
records=data.frame(
city = c("seattle", "san-diego"),
level = c(50, 100)
)
)
> a$records
i_1 level marginal lower upper scale
1 seattle 50 0 -Inf Inf 1
2 san-diego 100 0 -Inf Inf 1

Data Exchange with GDX

Up until now, we have been focused on using GAMS Transfer R to create symbols in an empty Container using the symbol constructors (or their corresponding container methods). These tools will enable users to ingest data from many different formats and add them to a Container. However, it is also possible to read in symbol data directly from GDX files using the read container method. In the following sections, we will discuss this method in detail as well as the write method, which allows users to write out to new GDX files.

Reading from GDX

There are two main ways to read in GDX based data.

  • Pass the file path directly to the Container or ConstContainer constructor (will read all symbols and records)
  • Pass the file path directly to the read method (default read all symbols, but can read partial files)

The first option here is provided for convenience and will, internally, call the read method. For a Container, this method will read in all symbols as well as their records. For a ConstContainer, this method will read only symbol metadata. This is the easiest and fastest way to get data out of a GDX file and into your R environment. For the following examples, we leverage the GDX output generated from the trnsport.gms model file.

Example (reading full data into a Container using the constructor)
m = Container$new("trnsport.gdx")
> names(m$data)
[1] "i" "j" "a" "b" "d" "f" "c" "x"
[9] "z" "cost" "supply" "demand"
> m$describeParameters()
name isScalar domain domainType dim numRecs minValue meanValue maxValue
1 a FALSE i regular 1 2 350.000 475.0000 600.000
2 b FALSE j regular 1 3 275.000 300.0000 325.000
5 c FALSE i j regular 2 6 0.126 0.1755 0.225
3 d FALSE i j regular 2 6 1.400 1.9500 2.500
4 f TRUE NA none 0 1 90.000 90.0000 90.000
whereMin whereMax countEps countNa countUndef cardinality sparsity
1 1 2 0 0 0 2 0
2 3 1 0 0 0 3 0
5 6 1 0 0 0 6 0
3 6 1 0 0 0 6 0
4 1 1 0 0 0 NA NA

A user could also read in data with the read method as shown in the following example.

Example (reading full data into a Container with read method)
m = Container$new()
m$read("trnsport.gdx")
> names(m$data)
[1] "i" "j" "a" "b" "d" "f" "c" "x"
[9] "z" "cost" "supply" "demand"

It is also possible to read in a partial GDX file with the read method, as shown in the following example:

m = Container$new()
m$read("trnsport.gdx", "x")
> names(m$data)
[1] "x"
> m$data$x$records
i_1 j_2 level marginal lower upper scale
1 seattle new-york 50 0.000 0 Inf 1
2 seattle chicago 300 0.000 0 Inf 1
3 seattle topeka 0 0.036 0 Inf 1
4 san-diego new-york 275 0.000 0 Inf 1
5 san-diego chicago 0 0.009 0 Inf 1
6 san-diego topeka 275 0.000 0 Inf 1

This syntax assumes that the user will always want to read in both the metadata as well as the actual data records, but it is possible to skip the reading of the records by passing the argument records=FALSE.

m = Container$new()
m$read("trnsport.gdx", "x", records = FALSE)
> names(m$data)
[1] "x"
> names(m$data)
[1] "x"
> m$data$x$summary
$name
[1] "x"
$type
[1] "positive"
$domain_objects
$domain_objects[[1]]
[1] "i"
$domain_objects[[2]]
[1] "j"
$domainNames
[1] "i" "j"
$dimension
[1] 2
$description
[1] "shipment quantities in cases"
$numberRecords
[1] 0
$domainType
[1] "relaxed"
> m$data$x$records
NULL
Attention
The read method attempts to link the domain objects together (in order to have a "regular" domainType) but if domain sets are not part of the read operation there is no choice but to default to a "relaxed" domainType. This can be seen in the last example where we only read in the variable x and not the domain sets (i and j) that the variable is defined over. All the data will be available to the user, but domain checking is no longer possible. The symbol x will remain with "relaxed" domain type even if the user were to read in sets i and j in a second read call.
Example (reading only metadata into a ConstContainer using the constructor)
h = ConstContainer$new("trnsport.gdx")
> names(h$data)
[1] "i" "j" "a" "b" "d" "f" "c" "x"
[9] "z" "cost" "supply" "demand"
> h$describeParameters()
name isScalar domain domainType dim numRecs minValue meanValue maxValue
1 a FALSE i regular 1 2 NA NA NA
2 b FALSE j regular 1 3 NA NA NA
5 c FALSE i j regular 2 6 NA NA NA
3 d FALSE i j regular 2 6 NA NA NA
4 f TRUE NA none 0 1 NA NA NA
whereMin whereMax countEps countNa countUndef cardinality sparsity
1 NA NA NA NA NA 2 0
2 NA NA NA NA NA 3 0
5 NA NA NA NA NA 6 0
3 NA NA NA NA NA 6 0
4 NA NA NA NA NA NA NA

Note that in this example we make use of the convenience notation contained in the constructor to read in only the metadata of the trnsport.gdx file. This allows users to quickly explore the symbols contained in a file and it also explains why there are many NA values in the columns of the $describeParameters() method.

Example (reading full data with ConstContainer$read() method)
h = ConstContainer$new()
h$read("trnsport.gdx")
> names(h$data)
[1] "i" "j" "a" "b" "d" "f" "c" "x"
[9] "z" "cost" "supply" "demand"
> h$describeParameters()
name isScalar domain domainType dim numRecs minValue meanValue maxValue
1 a FALSE i regular 1 2 350.000 475.0000 600.000
2 b FALSE j regular 1 3 275.000 300.0000 325.000
5 c FALSE i j regular 2 6 0.126 0.1755 0.225
3 d FALSE i j regular 2 6 1.400 1.9500 2.500
4 f TRUE NA none 0 1 90.000 90.0000 90.000
whereMin whereMax countEps countNa countUndef cardinality sparsity
1 1 2 0 0 0 2 0
2 3 1 0 0 0 3 0
5 6 1 0 0 0 6 0
3 6 1 0 0 0 6 0
4 1 1 0 0 0 NA NA

Writing to GDX

A user can write data to a GDX file by simply passing a file path (as a string). The write method will then create the GDX and write all data in the Container.

Note
It is not possible to write the Container when any of its symbols are invalid. If any symbols are invalid an error will be raised and the user will need to inspect the problematic symbols (perhaps using a combination of the listSymbols(isValid=FALSE) and isValid(verbose=TRUE) methods).
Example
m$write("path/to/file.gdx")
Example (write a compressed GDX file)
m$write("path/to/file.gdx", compress = TRUE)

Advanced users might want to specify an order to their UELs (i.e., the universe set); recall that the UEL ordering follows that dictated by the data. As a convenience, it is possible to prepend the list of UELs with a user specified order using the uelPriority argument.

Example (change the order of the UEL)
m = Container$new()
i = Set$new(m, "i", records=c("a", "b", "c"))
m$write("foo.gdx", uelPriority=c("a", "c"))

The original UEL order for this GDX file would have been c("a", "b", "c"), but since this example reorders the UELs with uelPriority, the positions of b and c have been swapped. This can be verified with the gdxdump utility (using the uelTable argument):

gdxdump foo.gdx ueltable=foo

Set foo /
  'a' ,
  'c' ,
  'b' /;
$onEmpty

Set i(*) /
'a',
'c',
'b' /;

$offEmpty
m