Here is an initial outline of the things we should validate:
Legal RPS2 files should be read correctly.
All fields in the proposal should be recognized and mapped into the appropriate fields in the APT data model.
All SR's and OP's should be validated.
Various nestings of exposure groups should be validated.
A few modes from each instrument should be validated.
Use of restricted mode should be validated.
Variations in white space should be accepted without complaint. White space presented the single biggest hurdle in development, so I would expect some lingering issue there.
In particular, I had manually specify each place in the grammar where white space is allowed, where it is required, and where it is not allowed. So checking that a new line is OK in one field, for example, doesn't necessarily check that it's OK in another.
Optional fields (including the new fields like exposure_label: that support this import) should really be optional.
Proposals with non-consecutive target or exposure numbers should be read in correctly. APT's numbering will be different after the import, but references to those targets and exposures should be resolved as specified.
Complex fields in target specifications should be exercised.
Illegal RPS2 files should be caught, and the user should be notified of the illegality via one of three types of APT diagnostics.
Parse Errors - These diagnostics will be generated by the parser when the input doesn't match the grammar the parser is using. Examples include misspelled keywords, missing required keywords and alphabetic characters where a number is required. This type of diagnostic will appear on the proposal object and will have a source of Rps2Importer.
Import Errors - The input matched the syntax specified by the grammar, but the importer couldn't figure out how to put it consistently in APT. Some examples include multiple "Contact: Y"'s on CoI's, invalid target descriptions and multiple exposures with the same number. This type of diagnostic will appear on the proposal object and will have a source of Rps2Importer.
APT Errors - The input was successfully mapped into the APT data model, but the data model notices an illegality. Examples include invalid config/mode combinations and all the reactive checks.
For some of the above cases, we can leverage the fact that APT writes out consistent RPS2 and APT formats. Separate proposals which are equivalent in APT have identical RPS2 and APT formatted output.
So, for example, we could create a proposal in APT and export it to RPS2, then import that RPS2 file. We would then expect an exported RPS2 file from there to match the original, and we would expect a saved APT file from there to match the original.
In addition, if we made a bunch of copies of the original RPS2 file, changing each copy only in white space, we could import those copies and expect their exported RPS2's to match the original.
We may also be able to use the existing preprocessor, and it's ability to write APT files. For example, we could PP an RPS2 file to create an xml file which we could read into APT. If we then exported that to RPS2, we would expect that RPS2 file to match an RPS2 file exported from an APT session that read in the original RPS2 file.
The PP can also help us verify which proposals are legal, and which are not. It is possible to contrive cases that will confuse the importer. It is also possible to contrive such cases for the PP. While the results in either case are not desirable, my goal for the importer is to behave at least as well as the PP in those contrived cases. When we find cases where that goal is not met, we can decide whether the case is worth fixing.
We can do some testing that simulates how a user will actually use this importer. Three major cases may be:
The user creates a large RPS2 file using a script and/or text editor, imports into APT once, and completes the proposal using APT.
The user creates an RPS2 file using a script and/or text editor, imports into APT, then makes changes to the RPS2 file in an editor and imports again, etc.
The user creates an RPS2 file using a script and/or text editor, imports into APT, makes changes to the proposal in APT, exports a new RPS2 file, then makes changes to the RPS2 file in an editor and imports again, etc.
Write proposal in APT, export to RPS2, do manual cutting/pasting in the RPS2 to extend it, then import the RPS2.
Hand-edit an existing RPS2 to be compatible with the new syntax, then import it.
Obviously the broader the testing the better, but as we have limited resources, I think we should definitely focus on tests that most closely match the cases we expect our users to create or encounter.
PR 49784 describes the syntax changes and extensions that have been implemented to address APT/RPS2 inconsistencies.
PP Grammar Definition File (as of 12/29/03). This file describes the top level of the grammar that the PP uses for parsing. It lists each major keyword, followed by a number (or range) of times that keyword is expected. If that range is "0 - 1", then the field is not required. If the number is 1, then the field is required.
We will track Import Issues here.
I should implement a command line interface that would import and RPS2 file, then write out the diagnostic report, save the APT version of the proposal, and export the RPS2 version. The file naming is the tricky part with this. The command line will have to include the APT filename and directory. If the imported file did not contain the prop id, I'm not sure what we should do for the exported prop file name.
Implement a pop-up indicating that the importer created some important diagnostics during the import.
For now, do not implement any feature which would allow the interactive clearing/acknowledging of import diagnostics. Also do not implement a feature which would cause the import diagnostics to be saved with the proposal.