Description
At first, the autoRow and autoCol option are tested particular. Afterwards, autoCol and autoRow are tested in combination with the useHeader resp. index option. Contributor Jan-Erik Justkowiak, March 2018
Small Model of Type : GAMS
Category : GAMS Test library
Main file : csv2gdx5.gms
$title CSV2GDX - Testing the autoRow and autoCol Option (CSV2GDX5,SEQ=750)
$onText
At first, the autoRow and autoCol option are tested particular. Afterwards,
autoCol and autoRow are tested in combination with the useHeader resp. index
option.
Contributor Jan-Erik Justkowiak, March 2018
$offText
* Testing the autoRow option
$onEcho > data.csv
x1,x2
565.0,575.0
25.0,185.0
345.0,750.0
945.0,685.0
845.0,655.0
$offEcho
$onEcho > testAutoRow.gms
* 1. This should fail, since a second dimension must be specified
$log
$log 1. Without autoRow
$call csv2gdx data.csv id=d useHeader=Y values=1,2 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
* 2. Using the autoRow option, the missing dimension with unique UELs should
* be generated automatically
$log
$log 2. With autoRow
$call csv2gdx data.csv id=d useheader=Y values=1,2 autoRow=rowPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim1 / rowPrefix1*rowPrefix5 /, dim2 / x1*x2 /;
Parameter d
/
rowPrefix1.x1 565.0
rowPrefix1.x2 575.0
rowPrefix2.x1 25.0
rowPrefix2.x2 185.0
rowPrefix3.x1 345.0
rowPrefix3.x2 750.0
rowPrefix4.x1 945.0
rowPrefix4.x2 685.0
rowPrefix5.x1 845.0
rowPrefix5.x2 655.0
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho
$call gams testAutoRow.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoRow!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoRow!
*-------------------------------------------------------------------------------
* Testing the autoCol option
$onEcho > data.csv
1,2,3,1,2,3,1,2,3,1,2,3
$offEcho
$onEcho > testAutoCol.gms
* 3. This should fail, since a second dimension must be specified
$log
$log 3. Without autoCol
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=12 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Multiple value columns require "autoCol=xyz" or "useHeader=Y"!
$log
* 4. Using the autoCol option, the missing dimension with unique UELs should
* be generated automatically
$log
$log 4. With autoCol
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=12 autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim1 / colPrefix1*colPrefix12 /;
Parameter d
/
colPrefix1 1
colPrefix2 2
colPrefix3 3
colPrefix4 1
colPrefix5 2
colPrefix6 3
colPrefix7 1
colPrefix8 2
colPrefix9 3
colPrefix10 1
colPrefix11 2
colPrefix12 3
/;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho
$call gams testAutoCol.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoCol!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoCol!
*-------------------------------------------------------------------------------
* Testing the autoRow and autoCol option in combination
$onEcho > data.csv
1,2,3,4
5,6,7,8
9,10,11,12
13,14,15,16
$offEcho
$onEcho > testAutoRowAutoCol.gms
* 5. This should fail, since there are no dimensions specified
$log
$log 5. Without autoCol and without autoRow
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=4 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Multiple value columns require "autoCol=xyz" or "useHeader=Y"!
$log
* 6. Using the autoCol and autoRow option, the missing dimensions with unique
* UELs should be generated automatically
$log
$log 6. With autoCol and autoRow
$call csv2gdx data.csv id=d useHeader=N values=1..lastCol colCount=4 autoCol=colPrefix autoRow=rowPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim1 / rowPrefix1*rowPrefix4 /, dim2 / colPrefix1*colPrefix4 /;
Parameter d
/
rowPrefix1.colPrefix1 1
rowPrefix1.colPrefix2 2
rowPrefix1.colPrefix3 3
rowPrefix1.colPrefix4 4
rowPrefix2.colPrefix1 5
rowPrefix2.colPrefix2 6
rowPrefix2.colPrefix3 7
rowPrefix2.colPrefix4 8
rowPrefix3.colPrefix1 9
rowPrefix3.colPrefix2 10
rowPrefix3.colPrefix3 11
rowPrefix3.colPrefix4 12
rowPrefix4.colPrefix1 13
rowPrefix4.colPrefix2 14
rowPrefix4.colPrefix3 15
rowPrefix4.colPrefix4 16
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho
$call gams testAutoRowAutoCol.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoRowAutoCol!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoRowAutoCol!
*-------------------------------------------------------------------------------
* Testing the autoRow and index option
$onEcho > data.csv
1,3
1,4
2,5
2,6
$offEcho
$onEcho > testAutoRowIndex.gms
* 7. This should fail, since there are duplicates
$log
$log 7. Without autoRow and with index
$call csv2gdx data.csv id=d useHeader=N index=1 value=lastCol colCount=2 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log
* 8. Using the autoRow option, unique UELs should be generated automatically
* to prevent duplicate records/keys
$log
$log 8. With autoRow and index
$call csv2gdx data.csv id=d useHeader=N index=1 value=lastCol colCount=2 autoRow=rowPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim1 / rowPrefix1*rowPrefix4 /, dim2 / 1*2 /;
Parameter d
/
rowPrefix1.1 3
rowPrefix2.1 4
rowPrefix3.2 5
rowPrefix4.2 6
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho
$call gams testAutoRowIndex.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoRowIndex!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoRowIndex!
*-------------------------------------------------------------------------------
* Testing the autoCol and useHeader=Y option
$onEcho > data.csv
1,1,2,2
3,4,5,6
$offEcho
$onEcho > testAutoColuseHeaderY.gms
* 9. This should fail, since there are duplicates
$log
$log 9. Without autoCol and with useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y values=1..lastCol colCount=4 trace=0
$ifE errorLevel=0 $abort Error expected: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log
* 10. Using the autoCol option, unique UELs should be generated automatically
* to prevent duplicate records/keys. The Header row is skipped.
* Note that lastCol cannot be used in the values option. CSV2GDX maight fail!
$log
$log 10. With autoCol and useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y values=1..4 colCount=4 autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim1 / colPrefix1*colPrefix4 /;
Parameter d
/
colPrefix1 3
colPrefix2 4
colPrefix3 5
colPrefix4 6
/;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho
$call gams testAutoColuseHeaderY.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoColuseHeaderY!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoColuseHeaderY!
*-------------------------------------------------------------------------------
* Testing the autoCol and useHeader=N option
$onEcho > data.csv
1,1,2,2
$offEcho
$onEcho > testAutoColuseHeaderN.gms
* 11. Using the autoCol option, unique UELs should be generated automatically
* to prevent duplicate records/keys. If there is a header row, useHeader=N
* might fail, if the data cannot be interpreted correctly (for the value columns).
$log
$log 11. With autoCol and useHeader=N
$call csv2gdx data.csv id=d useHeader=N values=1..4 colCount=4 autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim1 / colPrefix1*colPrefix4 /;
Parameter d
/
colPrefix1 1
colPrefix2 1
colPrefix3 2
colPrefix4 2
/;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho
$call gams testAutoColuseHeaderN.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAutoColuseHeaderN!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAutoColuseHeaderN!
*-------------------------------------------------------------------------------
* Testing the autoRow, autoCol, index and useHeader=Y option
$onEcho > data.csv
a,b,b
I,1,5
I,2,6
II,3,7
II,4,8
$offEcho
$onEcho > testAlluseHeaderY.gms
* 12. This should fail, since there are duplicates
$log
$log 12. Without autoCol, autoRow, with index and useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y index=1 values=2,3 colCount=3 trace=0
$ifE errorLevel=0 $abort Error: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log
* 13. Using the autoCol and autoRow option, unique UELs should be generated automatically
* to prevent duplicate records/keys. If there is a Header row, useHeader=N
* will fail.
$log
$log 13. With autoCol, autoRow, index and useHeader=Y
$call csv2gdx data.csv id=d useHeader=Y index=1 values=2,3 colCount=3 autoRow=rowPrefix autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim3 / colPrefix1*colPrefix2 /, dim2 / I, II /, dim1 / rowPrefix1*rowPrefix4 /;
Parameter d
/
rowPrefix1.I.colPrefix1 1
rowPrefix1.I.colPrefix2 5
rowPrefix2.I.colPrefix1 2
rowPrefix2.I.colPrefix2 6
rowPrefix3.II.colPrefix1 3
rowPrefix3.II.colPrefix2 7
rowPrefix4.II.colPrefix1 4
rowPrefix4.II.colPrefix2 8
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 d
$gdxOut
$offEcho
$call gams testAlluseHeaderY.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAlluseHeaderY!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAlluseHeaderY!
*-------------------------------------------------------------------------------
* Testing the autoRow, autoCol, index and useHeader=N option
$onEcho > data.csv
I,1,5
I,2,6
II,3,7
II,4,8
$offEcho
$onEcho > testAlluseHeaderN.gms
* 14. This should fail, since there are duplicates
$log
$log 14. Without autoCol, autoRow, with index and useHeader=N
$call csv2gdx data.csv id=d useHeader=N index=1 values=2,3 colCount=3 trace=0
$ifE errorLevel=0 $abort Error: CSV2GDX should have failed!
$log Expected Error: Duplicate records/keys!
$log
* 15. Using the autoCol option, unique UELs should be generated automatically
* to prevent duplicate records/keys.
$log
$log 15. With autoCol, autoRow, index and useHeader=N
$call csv2gdx data.csv id=d useHeader=N index=1 values=2,3 colCount=3 autoRow=rowPrefix autoCol=colPrefix trace=0
$ifE errorLevel<>0 $abort Error: CSV2GDX should have succeeded!
$log
* Check the results
Set dim3 / colPrefix1*colPrefix2 /, dim2 / I, II /, dim1 / rowPrefix1*rowPrefix4 /;
Parameter d
/
rowPrefix1.I.colPrefix1 1
rowPrefix1.I.colPrefix2 5
rowPrefix2.I.colPrefix1 2
rowPrefix2.I.colPrefix2 6
rowPrefix3.II.colPrefix1 3
rowPrefix3.II.colPrefix2 7
rowPrefix4.II.colPrefix1 4
rowPrefix4.II.colPrefix2 8
/;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 d
$gdxOut
$offEcho
$call gams testAlluseHeaderN.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testAlluseHeaderN!
$call gdxdiff data.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testAlluseHeaderN!
*-------------------------------------------------------------------------------