Input / Output
A input / output problem takes input from stdin or arguments and compares the submission output (stdout) with an expected answer.
The expected answers are specified within test cases. For each test case, you can specify:
Standard Input (Optional)
Arguments (Optional)
Expected Answer (Required)
Comment (Optional)
Getting Started
In order for Kodethon to know how to run the submission, you must provide a run command. For example:
Because the example command expects main.py to exist, this means that the submission file should be called main.py
In some cases where the file needs to be compiled before it is run, an init command should be used to compile the program. For example:
Settings
Init Command
The init command should be a UNIX shell command used to prepare the submission for testing. This command will be only run once.
For example, specifying gcc main.c as the init command will create an executable file a.out which can then be used by the run command.
Run Command
The run command should be a UNIX shell command used to execute the submission file. This command will be run once per test case.
For example, if the submission file is expected to be called main.py, then the corresponding run command should be python main.py.
Ignore Whitespace
This setting has three options:
None
Trailing
All
The setting defaults to None which is a strict comparison of the submission output and the expected answer; every character is expected to match.
When the setting is set to Trailing, all whitespace characters after the last non-whitespace character is ignored.
When the setting is set to All, all whitespace is ignored. For example, "HelloWorld" will match the expected answer "Hello World".
Problem File Structure
When a problem is created, the below files are created:
autograder/
The folder that contains all files needed for scoring a submission.
autograder/.answers
The folder where answer files are stored after being added in the test cases.
autograder/.inputs
The folder where stdin files are stored after being added in the test cases.
autograder/.arguments
The folder where argument files are stored after being added in the test cases.
autograder/.utils
Where utility files are stored. This folder should not be modified.
handout/
The handout folder should contain the files you expect your users to have.
.ref/
If you choose to use a reference program, the uploaded file(s) will be stored here.
.snapshots/
The snapshots folder is where all submissions are stored.
submission/
The submission folder can optionally contain a mock submission that can be used to test scoring.
.submissions/
The hidden submissions folder is where the latest submissions will be stored. Inside each submission folder will be the submitted files along with an output file with your test script’s output.
Submission File Structure
When a submission is made, we will generate the following folders under a new path in .submissions. Please see section 4.1.4 for more details.
autograder/
This folder contains symbolic links to all files in the problem’s autograder folder
submission/
This folder contains the submission files
output/
It is recommended to place log files here.
A sample submission path will look like:
How grading works
When a student submits their program for grading, Kodethon will generate a test script based on the problem settings and test cases. For example, if you set standard input to be ‘abc’ and arguments ‘1 2 3’ for Test Case 1, then the generated test command will look like:
The specified standard input ‘abc’ will be written into a file called ‘1’ and redirected into the student’s program.
Last updated
