Introduction
Can GAMS mathematical models be coupled with Artificial intelligence (AI) to find solutions to complex problems? This blog post tackles this question and provides some starter code and application examples.
Mathematical programmes built in GAMS are good at finding optimal values for endogenous variables for problems with single scenarios. For an economy-wide CGE model, for example, those variables could be sectoral output, factor supply, trade, or prices. However it is more difficult for mathematical programmes to find a strategy based on exogenous parameters, such as tax/subsidy rates and government transfers, in an efficient way across many scenarios.
To address this problem, Mohammed Basheer, Victor Nechifor, Alvaro Calzadilla, Julien Harou and others have developed a framework that links GAMS models to multi-objective evolutionary search algorithms (MOEAs) and other AI algorithms. The approach is potentially of interest to other GAMS users.
The researchers used GAMS to reliably simulate large systems with many linear or non-linear decision variables and then used an external ‘black box’ solver to solve for the exogenous parameters. The black box solvers of choice were multi-objective evolutionary algorithms (MOEAs), many of which are available as open-source software and can be very effective.
In a final twist, the group linked their pipeline to machine learning to help make sense of the Pareto-front (the multi-dimensional efficient solutions) produced by the MOEA.
Technical Implementation
The MOEA was accessed from a Python open-source library called Platypus . Platypus is a framework that focuses on MOEA algorithms and provides access to them through a high-level Application Programming Interface (API). Several MOEA algorithms can be accessed through Platypus, including NSGA-II, NSGA-III, MOEA/D, IBEA, Epsilon-MOEA, SPEA2, GDE3, OMOPSO, SMPSO, and Epsilon-NSGA-II.
Figure 1 shows the technical implementation of the integration of GAMS-based models, other Python-based models, and the MOEA algorithm. GAMS enables accessing and modifying models through a Python API . The API enables modifying model parameters, accessing variable values, and solving GAMS models through a Python package.
GAMS model access in Python
Exposing a GAMS model to python requires creating a GAMS model instance through the API, which enables running and modifying the model. Creating a model instance in Python was performed through the following steps:
workspace = GamsWorkspace("directory containing the GAMS model")
model = workspace.add_job_from_file("path to the GAMS model file")
opt = workspace.add_options()
opt.all_model_types = "solver name"
checkpoint = GamsCheckpoint(workspace = workspace, checkpoint_name = "checkpoint name")
model.run(opt, checkpoint)
model_instance = GamsModelInstance(
checkpoint = checkpoint,
modelinstance_name = "add instance name")
The “model_instance” enables running and modifying the GAMS model. All variables that need to be accessed are declared as follows:
var1 = model_instance.sync_db.add_variable(
identifier = "variable name in GAMS",
dimension = "integer representing variable dimension")
Similarly, parameters that need to be modified during the simulation or by the MOEA were declared as follows:
par1 = model_instance.sync_db.add_parameter(
identifier = "parameter name in GAMS",
dimension = "integer representing parameter dimension")
The last step in creating the model instance is instantiating the model instance:
model_instance.instantiate(
model_definition = "add the model definition",
modifiers = [par1, para2,...],
options = opt)
Now that the model instance is created, the GAMS model can communicate with other Python-based models. This was performed by modifying values coming from other models to the GAMS model as follows, e.g.:
par1.add_record().value = "parameter value"
Once inputs to GAMS are set, the model can be solved as follows:
model_instance.solve()
Also, values of some variables from the GAMS model were extracted and provided to the other models as follows, e.g.:
value_to_provide_to_other_models = model_instance.sync_db["variable name in GAMS"].find_record()
MOEA Wrapper in Python
MOEA wrapper is a Python class designed to create the MOEA optimization problem and facilitate communication between the MOEA and the integrated simulation models. As Figure 1 shows, the wrapper collects the values of the objective values at the end of each run of the integrated models and provides the values of the decision variables to the integrated models following each MOEA evaluation. The wrapper is provided with an instance of the integrated models. The instance is of a class that includes the following methods:
- get_variables: a method that provides the names and upper and lower bounds of the MOEA decision variables.
- get_objectives: a method that provides the names of the MOEA objectives.
- get_constraints: a method that provides the names of the MOEA constraints.
- apply_variables: a method that applies values to the decision variables
- get_objective_values: a method that extracts the values of the MOEA objectives from the integrated models.
- run: a method for running the integrated models for a single simulation
The wrapper is structured as follows:
class wrapper(object):
def __init__(self, integrated_models, *args, **kwargs):
super(wrapper, self).__init__(*args, **kwargs)
variables = list(integrated_models.get_variables())
objectives = list(integrated_models.get_objectives())
constraints = list(integrated_models.get_constraints())
self.problem = platypus.Problem(
len (variables),
len(objectives),
len(constraints))
self.problem.function = self.evaluate
self.problem.wrapper = self
for i, var in enumerate(variables):
self.problem.types[i] = platypus.Real(var.lower_bounds, var.upper_bounds)
for i, constraint in enumerate(constraints):
self.problem.constraints[i] = platypus.Constraint(
constraint.operator,
value = constraint.limit)
def evaluate(self, solution):
self.integrated_models.apply_variables(solution)
self.integrated_models.run()
objectives = []
for objective in integrated_models.get_objective_values():
objectives.append(objective)
constraints = []
for constraint in integrated_models.get_constraint_values():
constraints.append(constraint)
return objectives, constraints
Finally, to access and run the MOEA optimization:
moea_wrapper = wrapper(integrated_models="instance of the integrated models")
An example run using the NSGAIII algorithm is performed as follows for a maximum number of iterations equals n:
moea_algorithm = platypus.NSGAIII(moea_wrapper.problem)
moea_algorithm.run(n)
Please note that all the code blocks above are for demonstration and would need to be largely adapted to the use case and overall code design.
Machine learning applied to the MOEA results
To help understand which decision variables have a high influence on the objective values, machine learning was applied to the outputs of the MOEA. The MOEA optimization process produces decision variable values and the corresponding objective values for each iteration between the MOEA algorithm and the integrated simulators. These values are collected and used to train a machine learning model to derive the level of influence of each of the decision variables on the optimization objectives. We used the Random Forest Regression Machine Learning algorithm through the scikit-learn library in Python . In this case, each objective value is used as a target to train a Machine Learning model, while the decision values were used as features. First, the data were split into testing and training data using the “train_test_split” method of scikit-learn as follows:
X_train, X_test, Y_train, Y_test = train_test_split(
features,
targets,
test_size = "splitting proportion of the data 0-1")
The next step is to train a random forest model based on the training data, as follows:
model = RandomForestRegressor(
max_depth="max tree depth",
n_estimators = "number of trees")
model.fit(features,target)
importances = model.feature_importances
It is important to also examine how the model performs with the testing data compared to training data to avoid overfitting or underfitting the data. For example, the r2 can be calculated through the “r2_score” method of scikit-learn as follows:
Y_predicted_train = model.predict(X_train)
Y_predicted_test = model.predict(X_test)
R2_train = r2_score(y_true=Y_train,y_pred=Y_predicted_train)
R2_test = r2_score(y_true=Y_test,y_pred=Y_predicted_test)
Example Uses
It is best to explain the benefits of the approach by example. Below we describe briefly three different application areas where GAMS-AI linkages have been successfully applied.
Research Area 1: Water Resource Systems
Estimating the economic value of water can help planners manage it better. This application considered California’s largest inter-tied water resource system - the California Central Valley and surrounding mountain ranges, comprised of 30 reservoirs, 22 aquifers, and 51 urban and agricultural water demand sites. Water there has multiple economic uses: farmers can grow plants, industry can produce, hydropower plants can generate electricity, etc. At the same time, there are costs associated with operating the system: groundwater and surface water transfers require using energy-intensive pumps. Costs and benefits tend to be non-linearly dependent on the water levels in the reservoirs of the system. In addition to allocating and distributing water, planners have to manage for uncertainty. Future precipitation and temperature dependent evaporation are unknown yet they determine how much stored water will be needed in later years. Given these uncertainties, water managers have to decide on the release for this year’s uses vs. how much water to save.
This study estimates “carry over storage value functions”, i.e curves quantifying stored water’s value. In the work published open-access in Water Resources Research , the authors link multiple scenario simulations done with GAMS models to a multi-objective evolutionary algorithm (MOEA) to find the valuation curves of each surface reservoir’s storage that jointly maximize long-term economic benefits of water use in California. Knowing the regional economic value of leaving water in a large dam for subsequent potentially dry years helps water managers and planners make better infrastructure operation decisions.
Research Area 2: Economic Modeling
Countries around the world plan policies aiming to achieve sustainable development goals (SDGs) and improve national-level performance on social, economic, and environmental dimensions. However, in reality, tradeoffs might exist between different goals. For instance, reducing CO2 emissions can often have tradeoffs with poverty reduction and economic growth. Designing national economic policies aimed at achieving multiple SDG targets at once is a complex problem due to the tradeoffs involved. In a paper published in Nature Communications , the authors show that artificial intelligence (AI)-driven search and machine learning can be useful in navigating these tradeoffs. This application of GAMS and AI combined an economy-wide model (developed in GAMS) with a multi-objective evolutionary algorithm to search for exogenous national-level economic interventions that balance the performance across multiple SGD targets at once and identify the lowest possible tradeoffs. This is achieved through thousands of iterations between the economy-wide model and the search algorithm, as explained earlier in the technical implementation. The outcome of each iteration is a set of 47 exogenous economic interventions (changes to direct, production, and sales taxes/subsidies and household transfers), and the associated performance is measured by SGD indicators (i.e., the objectives). The SDG objectives included in the study are maximizing GDP growth and household income, minimizing income inequalities, and minimizing CO2 emissions. This framing is applied to the case of the Egyptian economy. Results of the study show that achieving sustainable development across multiple performance objectives requires a combination of exogenous interventions that are not necessarily intuitive due to the non-linear nature of economies. It was found that a compromise solution for the Egyptian case that improves performance across multiple objectives is achievable through tweaks to 47 individual exogenous parameters. Machine learning was used to understand how different economic interventions considered (i.e., the 47 exogenous decision variables) in the search process can influence sustainability performance. The Machine Learning analysis showed that among the 47 decision variables, changing the tax/subsidy on petroleum sales is the most effective policy instrument for achieving multiple SDGs, followed by producer tax/subsidy on private services, and then direct government transfers to households. The open-access published paper provides more details on the study and the results.
Research Area 3: Integrated economic and water resources modeling
The Nile is one of the longest rivers in the world and is geographically shared between 11 countries. Although the Nile has a large basin area that covers around 10% of the African continent, its streamflow is lower than other rivers that have a similar or smaller basin area, such as the Congo, the Niger, and the Mississippi. The limited water resources of the Nile River and the growing demand for water resources in the Nile riparian countries created political tensions. An example of such tensions is the ongoing disagreements between Ethiopia, Sudan, and Egypt on the construction and operation of the Grand Ethiopian Renaissance Dam. The dam is currently under construction in Ethiopia and, when completed, is expected to double Ethiopia’s electricity generation and improve electricity access in one of the poorest regions globally. However, neighboring Sudan and Egypt are worried that the dam would change the quantity and quality of river flow to them and negatively affect their use of the Nile. This represents a problem with tradeoffs between some objectives of the countries. A recent study published in Nature Climate Change used the combined simulation and AI framework described in Figure 1 to navigate the tradeoffs and discover solutions that can balance the performance between the three countries.
The study integrates economy-wide models of Ethiopia, Sudan, and Egypt (developed in GAMS), a water resources system model of the Nile (developed using Pywr in Python), and AI search and Machine Learning Algorithms. This combination enables searching for ways to manage the Grand Ethiopian Renaissance Dam (i.e., decision variables) considering economy-wide objectives (e.g., GDP of each of the three countries) and engineering objectives (e.g., hydropower generation and irrigation water supply) simultaneously. Results reveal that, especially with extreme wet or dry projections, implementing cooperative adaptive strategies for managing the Grand Ethiopian Renaissance Dam brings about advantages for Ethiopia, Sudan, and Egypt in terms of both economic and water management aspects. Nonetheless, if the adaptive management plans focus solely on maximizing economic benefits for one country, it leads to negative consequences for at least one of the remaining two countries. The outputs of the search process were then used to train Machine Learning models to understand how different decisions in managing the Grand Ethiopian Renaissance Dam affect the performance objectives of each of the three riparian countries. The Machine Learning analysis shows that the Egyptian GDP is most influenced by water releases during droughts, and the Sudanese and Ethiopian GDPs are more influenced by the hydropower generation targets of the dam. More details about the study can be found through the following link to the open-access paper: https://www.nature.com/articles/s41558-022-01556-6 .