Description

This test ensures that the substitution mechanism for Connect works properly
for the three uses of Connect: GAMS CLP connectIn/Out, standalone gamsconnect,
and embedded code. In addition it tests that the interaction of connectIn/IDCGDXInput
and connectOut/IDCGDXOutput works as expected.

Contributor: Michael Bussieck, February 2022

Small Model of Type : GAMS

Category : GAMS Test library

Main file : connectsub.gms

$title 'Test substitution for Connect' (CONNECTSUB,SEQ=900)

$onText
This test ensures that the substitution mechanism for Connect works properly
for the three uses of Connect: GAMS CLP connectIn/Out, standalone gamsconnect,
and embedded code. In addition it tests that the interaction of connectIn/IDCGDXInput
and connectOut/IDCGDXOutput works as expected.

Contributor: Michael Bussieck, February 2022
$offText


$log --- Using Python library %sysEnv.GMSPYTHONLIB%

$onEcho > in.csv
date,symbol,price
2016/01/04,AAPL,105.35
2016/01/04,AXP,67.59
2016/01/04,BA,140.50
$offEcho
$onEcho > sp.gms
Set dates, stocks;
$onExternalInput
Parameter stockprice(dates<,stocks<) / /;
$offExternalInput
abort$(card(stockprice)<>3) 'wrong parameter stockprice', stockprice;
$onExternalOutput
Parameter numData; numData = card(stockprice);
$offExternalOutput
$offEcho
$onEchoV > cin.yaml
- CSVReader:
    file: %CSVInFile%
    name: stockprice
    indexColumns: [1, 2]
    valueColumns: [3]
- GDXWriter:
    file: %gams.input%.gdx
    symbols:
      - name: stockprice
$offEcho
$onEchoV > cout.yaml
- GDXReader:
    file: %GDXOUT%
    symbols:
      - name: numData
- CSVWriter:
    file: %gams.input%.csv
    name: numData
    header: False
$offEcho

$log Test connectIn/Out connection with IDCGDXInput/Output and substitution at connectIn/Out level
$call.checkErrorLevel gams sp.gms lo=2 connectIn=cin.yaml --GDXOUT=cout.gdx --CSVINFILE=in.csv IDCGDXInput=sp.gms.gdx connectOut=cout.yaml IDCGDXOutput=cout.gdx
$echo 3.0 > out_expected.csv
$call.checkErrorLevel diff -bwi sp.gms.csv out_expected.csv

$log Test substitution at gamsconnect level
$call rm -f sp.gms.csv
$call.checkErrorLevel gamsconnect cin.yaml CSVINFILE=in.csv GAMS.INPUT=sp.gms
$call.checkErrorLevel gams sp.gms lo=2 IDCGDXInput=sp.gms.gdx IDCGDXOutput=cout.gdx
$call.checkErrorLevel gamsconnect cout.yaml GDXOUT=cout.gdx GAMS.INPUT=sp.gms
$call.checkErrorLevel diff -bwi sp.gms.csv out_expected.csv

Set dates, stocks;
Parameter stockprice(dates<,stocks<);
$if not set CSVINPUT $set CSVINPUT in
$onEmbeddedCodeV Connect: --CSVINPUT=%CSVINPUT%.csv
- CSVReader:
    file: %CSVINPUT%
    name: stockprice
    indexColumns: [1, 2]
    valueColumns: [3]
- GAMSWriter:
    symbols:
      - name: stockprice
$offEmbeddedCode
abort$(card(stockprice)<>3) 'wrong parameter stockprice', stockprice;
Parameter numData; numData = card(stockprice);
put_utility 'ECArguments' / ' --NUMDATA=numData';
embeddedCodeV Connect: --CSVOUTPUT=sp.gms
- GAMSReader:
    symbols:
      - name: %NUMDATA%
- CSVWriter:
    file: %CSVOUTPUT%.csv
    name: numData
    header: False
endEmbeddedCode  
execute.checkErrorLevel 'diff -bwi sp.gms.csv out_expected.csv';

$log Test EC arguments containing quotes (compile time)
$onEmbeddedCode Connect: argument containing 'single' and "double" quotes
- PythonCode:
    code: |
      args = gams.arguments
      if args != '''argument containing 'single' and "double" quotes''':
        raise Exception("gams.arguments not as expected")
$offEmbeddedCode

$log Test EC arguments containing quotes (run time)
put_utility "ecArguments" / "argument containing 'single' and " '"double" quotes';
embeddedCode Connect:
- PythonCode:
    code: |
      args = gams.arguments
      if args != '''argument containing 'single' and "double" quotes''':
        raise Exception("gams.arguments not as expected")
endEmbeddedCode

$log Test error message preserving "%"
$onEchoV > print_log_percent.gms
$onEmbeddedCode Connect:
- GDXReader:
    file: %file%
    readAll: True
$offEmbeddedCode
$offEcho
$call gams print_log_percent.gms > print_log_percent.log
$onEmbeddedCode Python:
with open("print_log_percent.log", "r")as f:
  s = f.read()
  if s.count("%file%") != 1 or s.count("%") != 3:
    raise Exception('Expected one occurrence of "%file%" and three occurrences of "%" in log output')
$offEmbeddedCode

$log Test parsing Connect code with different Python versions
$onEcho > connect_ast.py
import ast
import os
import gams

connect_dir = os.path.join(os.path.dirname(gams.__file__), "connect")
for dirpath, dirnames, filenames in os.walk(connect_dir):
    for file in filenames:
        if file.endswith(".py"):
            with open(os.path.join(dirpath, file), "r") as f:
                ast.parse(f.read())
$offEcho

$ifThen setenv PYTHON38
$  call.checkErrorLevel cd . && "%sysenv.PYTHON38%" "connect_ast.py"
$endIf

$ifThen setenv PYTHON39
$  call.checkErrorLevel cd . && "%sysenv.PYTHON39%" "connect_ast.py"
$endIf

$ifThen setenv PYTHON310
$  call.checkErrorLevel cd . && "%sysenv.PYTHON310%" "connect_ast.py"
$endIf

$ifThen setenv PYTHON311
$  call.checkErrorLevel cd . && "%sysenv.PYTHON311%" "connect_ast.py"
$endIf

$ifThen setenv PYTHON312
$  call.checkErrorLevel cd . && "%sysenv.PYTHON312%" "connect_ast.py"
$endIf