GAMS Engine provides a REST API which follows a specification called OpenAPI version 2.0. This specification defines a
standard, language-agnostic interface to RESTful APIs which allows both humans
and computers to easily discover and understand the abilities of the software.
One main advantage of this technology is that client libraries can easily be
autogenerated for several programming languages using the OpenAPI
Generator. This includes languages like Python, Java, C, C#, C++, R,
Node.js/JavaScript, Ruby, and many more. A complete list of supported API
clients can be found here. GAMS Engine can be accessed via any of these
clients. A typical workflow could be to take an autogenerated client and use it
as a starting point for a custom solution.
Tip:
For the communication between the client and Engine, only the REST API is
relevant. You don't have to care about anything that happens inside
Engine. For example, to submit a GAMS job, only the corresponding
endpoint of the API must be addressed. All subsequent steps (e.g.
sending the job to a free worker or queue it, calling GAMS, etc.) are
taken care of by Engine.
Tip:
The (technical) API documentation can be found here. You can download the definition
file for the latest version of GAMS Engine here.
Let's say we want to submit GAMS jobs via a python client using an autogenerated
client as a starting point. On https://openapi-generator.tech/ you can find several
ways on how to get such a client. One approach is via npm. First, install the OpenAPI generator tool:
> npm install
@openapitools/openapi-generator-cli -g
and set the version to version 5.4.0:
> npx @openapitools/openapi-generator-cli
version-manager set 5.4.0
Next, we can generate the python client. For this we
need a valid definition file for the Engine API. You can download this file for
the latest version of GAMS Engine here:
> npx
@openapitools/openapi-generator-cli generate -i
path/to/gams_engine.json -g python -o path/to/output/files
--package-name=gams_engine
In the directory specified in the above command we will
then find all the client and (also auto-generated) documentation files
containing a lot of examples. In ./docs/JobsApi.md we can find an
example for the job submission:
import time
import gams_engine
from gams_engine.api import jobs_api
from gams_engine.model.message import Message
from gams_engine.model.quota_exceeded import QuotaExceeded
from gams_engine.model.message_and_token import MessageAndToken
from pprint import pprint
# Defining the host is optional and defaults to http://localhost
# See configuration.py for a list of all supported configuration parameters.
configuration = gams_engine.Configuration(
host = "http://localhost/api"
)
# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.
# Configure HTTP basic authorization: BasicAuth
configuration = gams_engine.Configuration(
username = 'YOUR_USERNAME',
password = 'YOUR_PASSWORD'
)
# Enter a context with an instance of the API client
with gams_engine.ApiClient(configuration) as api_client:
# Create an instance of the API class
api_instance = jobs_api.JobsApi(api_client)
model = "model_example" # str | Name of the model
namespace = "namespace_example" # str | Namespace containing(or will contain) the model
run = "run_example" # str | Name of the main gms file with its extension. Will use model + '.gms' if not provided. (optional)
inex_string = "inex_string_example" # str | Optional JSON string to filter the contents of the result zip file (inex_file takes precedence if specified) (optional)
text_entries = [
"text_entries_example",
] # [str] | (optional)
stream_entries = [
"stream_entries_example",
] # [str] | (optional)
stdout_filename = "log_stdout.txt" # str | Name of the file that captures stdout (optional) if omitted the server will use the default value of "log_stdout.txt"
arguments = [
"arguments_example",
] # [str] | Arguments that will be passed to GAMS call (optional)
dep_tokens = [
"dep_tokens_example",
] # [str] | Tokens of jobs on which this job depends. The order defines the order in which the results of dependent jobs are extracted. (optional)
labels = [
"labels_example",
] # [str] | Labels that will be attached to the job in key=value. Currently supported labels are: cpu_request, memory_request, workspace_request, node_selectors, tolerations, instance (optional)
model_data = open('/path/to/file', 'rb') # file_type | Zip file containing model files, if model is not registered (optional)
data = open('/path/to/file', 'rb') # file_type | File containing data in zip (optional)
inex_file = open('/path/to/file', 'rb') # file_type | Optional JSON file to filter the contents of the result zip file (optional)
# example passing only required values which don't have defaults set
# and optional values
try:
# Submits a new job to be solved
api_response = api_instance.create_job(model, namespace, run=run, inex_string=inex_string, text_entries=text_entries, stream_entries=stream_entries, stdout_filename=stdout_filename, arguments=arguments, dep_tokens=dep_tokens, labels=labels, model_data=model_data, data=data, inex_file=inex_file)
pprint(api_response)
except gams_engine.ApiException as e:
print("Exception when calling JobsApi->create_job: %s\n" % e)
Using this as a basis, we can now write our own python client. In the following
code some of the above elements are used to submit a job for the model
trnsport. The GAMS log is shown, the ZIP file containing the results is
unpacked and a few results are displayed. Note that the GAMS Python API is also used in this example.
import sys
import time
import os
import zipfile
from gams import *
import gams_engine
from gams_engine.api import jobs_api
if __name__ == "__main__":
if len(sys.argv) > 1:
ws = GamsWorkspace(system_directory = sys.argv[1])
else:
ws = GamsWorkspace()
model = 'trnsport' # str | Name of the main .gms file
ws.gamslib(model)
model_data_path = os.path.join(ws.working_directory, model + '.zip')
with zipfile.ZipFile(model_data_path, 'w', zipfile.ZIP_DEFLATED) as model_data:
model_data.write(os.path.join(ws.working_directory, model+'.gms'), arcname=model+'.gms')
stdout_filename = 'log_stdout.txt' # str | Name of the file that captures stdout (default to 'log_stdout.txt')
arguments = ['gdx=default'] # list[str] | Arguments that will be passed to GAMS call (optional)
data = None # file | File containing data in zip (optional)
pf_file_name = None # str | Name of the pf file in the zip, if there is (optional)
text_entries = [] # list[str] | (optional)
stream_entries = [] # list[str] | (optional)
inex_file = None # file | Optional file to filter what will be inside the result zip file (optional)
configuration = gams_engine.Configuration(
host = os.environ['ENGINE_URL'],
username = os.environ['ENGINE_USER'],
password = os.environ['ENGINE_PASSWORD']
)
namespace = os.environ['ENGINE_NAMESPACE'] # str | Namespace containing(or will contain) the model
# Enter a context with an instance of the API client
with gams_engine.ApiClient(configuration) as api_client:
# Create an instance of the API class
job_api_instance = jobs_api.JobsApi(api_client)
try:
print('Posting ' + model)
with open(model_data_path, 'rb') as model_data:
create_job_response = job_api_instance.create_job(model, namespace,
stdout_filename=stdout_filename, model_data=model_data, arguments=arguments)
token = create_job_response.token
print('Job token: %s' % token)
except gams_engine.ApiException as e:
print("Exception when calling JobsApi.create_job(): %s\n" % e)
sys.exit(1)
finished = False
time_spent = 0
while not finished:
try:
resp = job_api_instance.pop_job_logs(token)
print(resp.message, end='')
if resp.queue_finished:
finished = True
time.sleep(0.5)
except gams_engine.ApiException as e:
if e.status == 403:
print('Job still in queue. Wait 0.5 seconds.')
time.sleep(0.5)
time_spent += 0.5
if time_spent > 120:
print("The Engine instance seems to be busy. Please try again later.")
sys.exit(1)
if job_api_instance.get_job(token).process_status != 0:
print("Job did not terminate successfully.")
try:
print('Fetching results of model: ' + model)
with zipfile.ZipFile(job_api_instance.get_job_zip(token)) as zf:
gdxfile = zf.extract(model+".gdx", path=ws.working_directory)
except gams_engine.ApiException as e:
print("Exception when calling JobsApi.get_job_zip(): %s\n" % e)
sys.exit(1)
try:
# remove results from server
job_api_instance.delete_job_zip(token)
except gams_engine.ApiException as e:
print("Exception when calling JobsApi.delete_job_zip(): %s\n" % e)
sys.exit(1)
result_db = ws.add_database_from_gdx(os.path.join(ws.working_directory, gdxfile))
for rec in result_db["x"]:
print("x(" + rec.key(0) + "," + rec.key(1) + "): level=" + str(rec.level) + " marginal=" + str(rec.marginal))
The job submission is done here:
create_job_response = job_api_instance.create_job(model,
namespace,
model_data=model_data,
arguments=arguments)
The log is streamed here:
token = create_job_response.token
[...]
resp = job_api_instance.pop_job_logs(token)
The zip file with the results is downloaded here:
result_zip = job_api_instance.get_job_zip(token)
Did you know...
...that the interface from GAMS Studio to Engine was developed in the same
way? It's based on an autogenerated C++ client which was used as
a starting point for an integrated solution in Studio.