csv2gdx5.gms : CSV2GDX - Testing the autoRow and autoCol Option

Description

At first, the autoRow and autoCol option are tested particular. Afterwards,
autoCol and autoRow are tested in combination with the useHeader resp. index
option.

Contributor Jan-Erik Justkowiak, March 2018


Small Model of Type : GAMS


Category : GAMS Test library


Main file : csv2gdx5.gms

$title CSV2GDX - Testing the autoRow and autoCol Option (CSV2GDX5,SEQ=750)

$onText
At first, the autoRow and autoCol option are tested particular. Afterwards,
autoCol and autoRow are tested in combination with the useHeader resp. index
option.

Contributor Jan-Erik Justkowiak, March 2018
$offText


* Testing the autoRow option
$onEcho > data.csv
x1,x2
565.0,575.0
25.0,185.0
345.0,750.0
945.0,685.0
845.0,655.0
$offEcho

$onEcho > testAutoRow.gms
* 1. This should fail, since a second dimension must be specified
$log
$log 1. Without autoRow
$call csv2gdx data.csv id=d useHeader=Y values=1,2 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!


* 2. Using the autoRow option, the missing dimension with unique UELs should
*    be generated automatically
$log
$log 2. With autoRow
$call csv2gdx data.csv id=d useheader=Y values=1,2 autoRow=rowPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log

* Check the results
Set dim1 / rowPrefix1*rowPrefix5 /, dim2 / x1*x2 /;
Parameter d
/
rowPrefix1.x1  565.0
rowPrefix1.x2  575.0
rowPrefix2.x1   25.0
rowPrefix2.x2  185.0
rowPrefix3.x1  345.0
rowPrefix3.x2  750.0
rowPrefix4.x1  945.0
rowPrefix4.x2  685.0
rowPrefix5.x1  845.0
rowPrefix5.x2  655.0
/;

$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho

$call gams testAutoRow.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoRow!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoRow!
*-------------------------------------------------------------------------------



* Testing the autoCol option
$onEcho > data.csv
1,2,3,1,2,3,1,2,3,1,2,3
$offEcho

$onEcho > testAutoCol.gms
* 3. This should fail, since a second dimension must be specified
$log
$log 3. Without autoCol
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=12 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Multiple value columns require "autoCol=xyz" or "useHeader=Y"!
$log


* 4. Using the autoCol option, the missing dimension with unique UELs should
*    be generated automatically
$log
$log 4. With autoCol
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=12 autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log

* Check the results
Set dim1 / colPrefix1*colPrefix12 /;
Parameter d
/
colPrefix1  1
colPrefix2  2
colPrefix3  3
colPrefix4  1
colPrefix5  2
colPrefix6  3
colPrefix7  1
colPrefix8  2
colPrefix9  3
colPrefix10 1
colPrefix11 2
colPrefix12 3
/;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho

$call gams testAutoCol.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoCol!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoCol!
*-------------------------------------------------------------------------------



* Testing the autoRow and autoCol option in combination
$onEcho > data.csv
1,2,3,4
5,6,7,8
9,10,11,12
13,14,15,16
$offEcho

$onEcho > testAutoRowAutoCol.gms
* 5. This should fail, since there are no dimensions specified
$log
$log 5. Without autoCol and without autoRow
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=4 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Multiple value columns require "autoCol=xyz" or "useHeader=Y"!
$log


* 6. Using the autoCol and autoRow option, the missing dimensions with unique
*    UELs should be generated automatically
$log
$log 6. With autoCol and autoRow
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=4 autoCol=colPrefix autoRow=rowPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log

* Check the results
Set dim1 / rowPrefix1*rowPrefix4 /, dim2 / colPrefix1*colPrefix4 /;
Parameter d
/
rowPrefix1.colPrefix1  1
rowPrefix1.colPrefix2  2
rowPrefix1.colPrefix3  3
rowPrefix1.colPrefix4  4
rowPrefix2.colPrefix1  5
rowPrefix2.colPrefix2  6
rowPrefix2.colPrefix3  7
rowPrefix2.colPrefix4  8
rowPrefix3.colPrefix1  9
rowPrefix3.colPrefix2 10
rowPrefix3.colPrefix3 11
rowPrefix3.colPrefix4 12
rowPrefix4.colPrefix1 13
rowPrefix4.colPrefix2 14
rowPrefix4.colPrefix3 15
rowPrefix4.colPrefix4 16
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho

$call gams testAutoRowAutoCol.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoRowAutoCol!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoRowAutoCol!
*-------------------------------------------------------------------------------



* Testing the autoRow and index option
$onEcho > data.csv
1,3
1,4
2,5
2,6
$offEcho

$onEcho > testAutoRowIndex.gms
* 7. This should fail, since there are duplicates
$log
$log 7. Without autoRow and with index
$call csv2gdx data.csv id=d useHeader=N index=1 value=lastCol colCount=2 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log


* 8. Using the autoRow option, unique UELs should be generated automatically
*    to prevent duplicate records/keys
$log
$log 8. With autoRow and index
$call csv2gdx data.csv id=d useHeader=N index=1 value=lastCol colCount=2 autoRow=rowPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log

* Check the results
Set dim1 / rowPrefix1*rowPrefix4 /, dim2 / 1*2 /;
Parameter d
/
rowPrefix1.1 3
rowPrefix2.1 4
rowPrefix3.2 5
rowPrefix4.2 6
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho

$call gams testAutoRowIndex.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoRowIndex!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoRowIndex!
*-------------------------------------------------------------------------------



* Testing the autoCol and useHeader=Y option
$onEcho > data.csv
1,1,2,2
3,4,5,6
$offEcho

$onEcho > testAutoColuseHeaderY.gms
* 9. This should fail, since there are duplicates
$log
$log 9. Without autoCol and with useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y values=1..lastCol colCount=4 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log


* 10. Using the autoCol option, unique UELs should be generated automatically
*     to prevent duplicate records/keys. The Header row is skipped.
*     Note that lastCol cannot be used in the values option. CSV2GDX maight fail!
$log
$log 10. With autoCol and useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y values=1..4 colCount=4 autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log


* Check the results
Set dim1 / colPrefix1*colPrefix4 /;
Parameter d
/
colPrefix1 3
colPrefix2 4
colPrefix3 5
colPrefix4 6
/;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho

$call gams testAutoColuseHeaderY.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoColuseHeaderY!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoColuseHeaderY!
*-------------------------------------------------------------------------------



* Testing the autoCol and useHeader=N option
$onEcho > data.csv
1,1,2,2
$offEcho

$onEcho > testAutoColuseHeaderN.gms
* 11. Using the autoCol option, unique UELs should be generated automatically
*     to prevent duplicate records/keys. If there is a header row, useHeader=N
*     might fail, if the data cannot be interpreted correctly (for the value columns).
$log
$log 11. With autoCol and useHeader=N
$call csv2gdx data.csv id=d useHeader=N values=1..4 colCount=4 autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log


* Check the results
Set dim1 / colPrefix1*colPrefix4 /;
Parameter d
/
colPrefix1 1
colPrefix2 1
colPrefix3 2
colPrefix4 2
/;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho

$call gams testAutoColuseHeaderN.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoColuseHeaderN!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoColuseHeaderN!
*-------------------------------------------------------------------------------



* Testing the autoRow, autoCol, index and useHeader=Y option
$onEcho > data.csv
a,b,b
I,1,5
I,2,6
II,3,7
II,4,8
$offEcho

$onEcho > testAlluseHeaderY.gms
* 12. This should fail, since there are duplicates
$log
$log 12. Without autoCol, autoRow, with index and useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y index=1 values=2,3 colCount=3 trace=0
$ifE errorLevel=0 $abort Error: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log


* 13. Using the autoCol and autoRow option, unique UELs should be generated automatically
*     to prevent duplicate records/keys. If there is a Header row, useHeader=N
*     will fail.
$log
$log 13. With autoCol, autoRow, index and useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y index=1 values=2,3 colCount=3 autoRow=rowPrefix autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log


* Check the results
Set dim3 / colPrefix1*colPrefix2 /, dim2 / I, II /, dim1 / rowPrefix1*rowPrefix4 /;
Parameter d
/
rowPrefix1.I.colPrefix1  1
rowPrefix1.I.colPrefix2  5
rowPrefix2.I.colPrefix1  2
rowPrefix2.I.colPrefix2  6
rowPrefix3.II.colPrefix1 3
rowPrefix3.II.colPrefix2 7
rowPrefix4.II.colPrefix1 4
rowPrefix4.II.colPrefix2 8
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 d
$gdxOut
$offEcho

$call gams testAlluseHeaderY.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAlluseHeaderY!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAlluseHeaderY!
*-------------------------------------------------------------------------------



* Testing the autoRow, autoCol, index and useHeader=N option
$onEcho > data.csv
I,1,5
I,2,6
II,3,7
II,4,8
$offEcho

$onEcho > testAlluseHeaderN.gms
* 14. This should fail, since there are duplicates
$log
$log 14. Without autoCol, autoRow, with index and useHeader=N
$call csv2gdx data.csv id=d useHeader=N index=1 values=2,3 colCount=3 trace=0
$ifE errorLevel=0 $abort Error: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log


* 15. Using the autoCol option, unique UELs should be generated automatically
*     to prevent duplicate records/keys.
$log
$log 15. With autoCol, autoRow, index and useHeader=N
$call csv2gdx data.csv id=d useHeader=N index=1 values=2,3 colCount=3 autoRow=rowPrefix autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log


* Check the results
Set dim3 / colPrefix1*colPrefix2 /, dim2 / I, II /, dim1 / rowPrefix1*rowPrefix4 /;
Parameter d
/
rowPrefix1.I.colPrefix1  1
rowPrefix1.I.colPrefix2  5
rowPrefix2.I.colPrefix1  2
rowPrefix2.I.colPrefix2  6
rowPrefix3.II.colPrefix1 3
rowPrefix3.II.colPrefix2 7
rowPrefix4.II.colPrefix1 4
rowPrefix4.II.colPrefix2 8
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 d
$gdxOut
$offEcho

$call gams testAlluseHeaderN.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAlluseHeaderN!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAlluseHeaderN!
*-------------------------------------------------------------------------------