Rust Warning: Cargo Test Passes Its Command Line Arguments to Your (Test) Programs

This blog post describes an issue that you may face using cargo to test rust programs that parse command line arguments.

I used something like the following command line to invoke rusts tests for some logic that parses command line arguments.

cargo test -v --workspace --all-features --target-dir $linbld -- --nocapture -- -epdv word a b c

I would break this into some parts:

cargo test -v --workspace --all-features --target-dir $linbld

I think that this tells cargo to build the test binary in the $linbld directory and run all tests in the project (possibly excluding code samples in documentation comments) in verbose mode.

--

After this, the remaining arguments should go to the cargo test infrastructure.

--nocapture

Tell the cargo test infrastructure not to capture and hence prevent stdout and stderr from appearing in the output of running the tests, so as to render println! macros that running tests would normally omit.

--

Tell the cargo test infrastructure to pass the remaining arguments to my program.

-epdv word a b c

These should be command line arguments to my program.

My test program failed testing. The reason is that, to run tests, cargo builds a binary that uses a main() function that is *not* the main() function in your /main.rs file, but something that invokes the tests. This executable test binary has a unique name, such as /tmp/wink.build/debug/deps/wink-442d9986badfd864 in one case, and must contain some logic that parses command line arguments such as –show-output (two hyphens) and sets some global flags (or something) to enable the tests to write to the console (tests generally assert and panic rather than generating output). The test binary apparently ignores command line arguments such as — (two hyphens), which my program also ignores. When the tests run, the program name is not the name in my package, as the tests expected, and the arguments include those passed to the binary that runs the tests, which the tests did not expect.

My program/tests actually include some infrastructure to show the command line used to run the tests and the the parameters passed to them. With this command line:

cargo test -v --workspace --all-features --target-dir /tmp/wink.build -- --nocapture -- -epdv word a b c

The build script that runs this command to invoke the tests generate the following output, showing the binary name, –nocapture, and — arguments that I had not expected.

cargo test -v --workspace --all-features --target-dir /tmp/wink.build -- --nocapture -- -epdv word a b c
+ tee /dev/fd/2
       Fresh unicode-xid v0.2.2
       ...
       Fresh wink v0.1.0 (/mnt/c/temp/wink)
    Finished test [unoptimized + debuginfo] target(s) in 0.06s
     Running `/tmp/wink.build/debug/deps/wink-2c659bbd243ff8ff --nocapture -- -epdv word a b c`

running 2 tests
thread 'tests::it_has_a_path' panicked at 'it_has_a_path intentional panic to render path to parent of this tests module.', src/lib.rs:116:9
stack backtrace:
it_gets_from_command_line_args: {
  "cmd_name": "wink-2c659bbd243ff8ff",
  "verbose": true,
  "dry_run": true,
  "command_code": "word",
  "export": true,
  "pretty_print": true,
  "all_args": [
    "/tmp/wink.build/debug/deps/wink-2c659bbd243ff8ff",
    "--nocapture",
    "--",
    "-epdv",
    "word",
    "a",
    "b",
    "c"
  ],
  "cmd_args": [
    "a",
    "b",
    "c"
  ],
  "help_msg": ""
}
test tests::it_gets_from_command_line_args ... ok
   0: std::panicking::begin_panic
             at /rustc/53cb7b09b00cbea8754ffb78e7e3cb521cb8af4b/library/std/src/panicking.rs:519:12
   1: wink::tests::it_has_a_path
             at ./src/lib.rs:116:9
   2: wink::tests::it_has_a_path::{{closure}}
             at ./src/lib.rs:115:5
...
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

This issue could cause challenges or issues for testing and logic that depends on the command name or uses the same command line parameter tokens that the cargo test infrastructure uses, but there do not seem to many arguments used by the test executables, so this seems unlikely. It also require logic in the tests such as that bolded in the example below, which may require additions over time. There are some ways that this could complicate or invalidate testing, such if you *want* see how your code *would* respond to the test infrastructure arguments, or if your function needs to parse command line arguments directly from std::env::args() rather than accepting them as a parameter.

/// The WinkConfig struct represents command line options passed
/// to the wink command.
#[derive(serde::Serialize)]
pub struct WinkConfig {
    /// The name of the command without the path, such as wink or wink.exe.
    pub cmd_name: String,

    /// Verbose: true if the -v command line option is present. Generates more output.
    pub verbose: bool,

    /// DryRun: true if the -d command line option is present. Do not run the command.
    pub dry_run: bool,

    /// the command code entered by the user, such as EXP or CMD.
    pub command_code: String,

    /// Export: true if the -e command line option is present. Export JSON configuration.
    pub export: bool,

    /// PrettyPrint: true if the -p command line option is present. Pretty-print JSON exports.
    pub pretty_print: bool,

    /// all of the arguments on the command line, including cmd_name
    pub all_args: Vec<String>,

    /// unprocessed command line arguments to pass to the command that wink will invoke.
    pub cmd_args: Vec<String>,

    /// empty unless there is a problem parsing command line arguments
    pub help_msg: String,
}

/// Implement the Display trait for WinkConfig to render the struct as JSON.
impl std::fmt::Display for WinkConfig {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        if self.pretty_print {
            write!(f, "{}", serde_json::to_string_pretty(&self).unwrap())
        } else {
            write!(f, "{}", serde_json::to_string(&self).unwrap())
        }
    }
}

impl WinkConfig {
    /// The get_from_cmd_line_args function return a WinkConfig
    /// created from parsing the command line.
    pub fn new(args: Vec<String>) -> WinkConfig {
        let mut dry_run: bool = false; // -d command line option
        let mut verbose: bool = false; // -v command line option
        let mut export: bool = false; // -e command line option
        let mut pretty_print: bool = false; // -p command line option
        let mut first_arg_index = 1; // number of processed command line arguments (first is command name, such as wink)
        let mut help_msg = String::new();

        for arg in args.iter().skip(first_arg_index) {
            if arg.to_lowercase() == "help" {
                help_msg = format!("Help requested by {0}", arg);
                break;
            }

            let prefix: char = arg.chars().next().unwrap();

            // if the argument is not help and does not start with a slash or a dash, then it should be a command code
            if prefix != '/' && prefix != '-' {
                break;
            }

            for char in arg.chars() {
                match char {
                    '/' | '-' => continue,
                    'v' => verbose = true,
                    'd' => dry_run = true,
                    'p' => pretty_print = true,
                    'e' => export = true,
                    'h' | '?' => {
                        help_msg = format!("Help requested by {0}", arg);
                        break;
                    }
                    _ => {
                        help_msg = format!("Unrecognized command line option: {0}", arg);
                        break;
                    }
                }
            }

            first_arg_index += 1; // just to offend C++ programmers
        }

        let mut command_code = String::new();

        if first_arg_index < args.len() {
            command_code = args[first_arg_index].to_owned();
            first_arg_index += 1;
        }

        WinkConfig {
            cmd_name: regex::Regex::new(r".*[\\/](?P<name>[^\\/]+$)")
                .unwrap()
                .replace_all(args[0].as_str(), "$name")
                .to_string(),
            verbose,
            dry_run,
            command_code,
            export,
            pretty_print,
            cmd_args: (args[first_arg_index..]).to_vec(),
            all_args: args,
            help_msg,
        }
    }
}

#[cfg(test)]
mod tests {
    #[test]
    /// to pass, run tests like this:
    /// cargo test -v --all-features --all=targets --target-dir $linbld -- --show-output -epdv word a b c
    fn it_gets_from_command_line_args() {
        let mut args: Vec<String> = std::env::args().collect();

        if let Some(pos) = args
            .iter()
            .position(|x| *x == "--show-output" || *x == "--nocapture")
        { 
            args.remove(pos);
        }

        let wink_config = crate::WinkConfig::new(args);
        println!("it_gets_from_command_line_args: {0}", wink_config);
        assert!(
            wink_config.cmd_name.starts_with("wink"), // cargo test adds a suffix
            "{0} !starts_with({1})",
            wink_config,
            "wink"
        );
        assert!(wink_config.verbose);
        assert!(wink_config.dry_run);
        assert!(wink_config.export);
        assert!(wink_config.pretty_print);
        assert_eq!(wink_config.command_code, "word");
        assert!(wink_config.help_msg.is_empty(), "{}", wink_config.help_msg);
    }

    #[test]
    #[should_panic]
    /// this test renders the path to the parent of this tests module
    fn it_has_a_path() {
        panic!("it_has_a_path intentional panic to render path to parent of this tests module.")
    }
}

Coincidentally, tests in my rust program revealed what I consider to be a defect, or at least an issue, with the wslpath command that converts between Windows and Unix file system paths under Windows Subsystem for Linux. When converting C:\ from a Windows path to a Unix path, the wslpath command outputs /mnt/c/ (I guess the trailing slash is logical). When converting C: to a Unix path (without the trailing backslash), wslpath returns C: without any slashes or backslashes where I would expect /mnt/c.

thread 'wsl::tests::it_converts_c_drive' panicked at 'assertion failed: (left == right)
left: "C:",
right: "/mnt/c"', src/wsl.rs:110:9

So I had to comment that test elsewhere in my code and will have to remember not to use C: with wslpath.

#[test]
fn it_converts_c_drive() {
    assert_eq!(
        &crate::wsl::wsl_path_or_self("C:\\", true /*unix*/),
        "/mnt/c/"
    );

    // wslpath C: unexpectedly returns C:
    // assert_eq!(&crate::wsl::wsl_path_or_self("C:", true /*unix*/), "/mnt/c"); 
}

I filed an issue with WSL on github.