Package Developer Notes¶

Package Installation¶

Create a Virtual Environment¶

$ python3.7 -m venv --prompt mcs venv
$ source venv/bin/activate
(mcs) $ python -m pip install --upgrade pip setuptools wheel

Install the MCS Package and dependencies¶

Install the packages included in the requirements file so that linting and documentation work. The requirements.txt file includes developer and package dependencies.

(mcs) $ python -m pip install -r requirements.txt

From the MCS root folder, install the package in your virtual environment in editable mode (-e) so that local changes will automatically reflect in the virtual environment.

(mcs) $ python -m pip install -e .

Run pre-commit¶

Run pre-commit install to set up the git hooks for linting and auto-documentation.

(mcs) $ pre-commit install

Linting¶

We are currently using flake8 and autopep8 for linting and formatting our Python code. This is enforced within the python_api and scene_generator projects. Both are PEP 8 compliant (besides some inline exceptions), although we are ignoring the following rules: - E402: Module level import not at top of file - W504: Line break occurred after a binary operator

A full list of error codes and warnings enforced can be found here

Both have settings so that they should run on save within Visual Studio Code settings.json as well as on commit after running pre-commit install (see .pre-commit-config.yaml and .flake8), but can also be run on the command line:

(mcs) $ flake8

and

(mcs) $ autopep8 --in-place --aggressive --recursive <directory>

or

Testing¶

See our tests README

Sphinx Documentation¶

Good Sphinx Tutorial: https://medium.com/@richdayandnight/a-simple-tutorial-on-how-to-document-your-python-project-using-sphinx-and-rinohtype-177c22a15b5b
Markdown Builder: https://pypi.org/project/sphinx-markdown-builder/
Sphinx: https://www.sphinx-doc.org/en/master/
Sphinx’s own Tutorial: https://www.sphinx-doc.org/en/master/usage/quickstart.html

To generate the Sphinx documentation locally, from the docs folder:

(mcs) $ make html

Python Style Guide¶

See https://numpydoc.readthedocs.io/en/latest/format.html

Running¶

We have made multiple run scripts:

To run a script (like run_human_input.py) from the terminal with visual output:

(mcs) $ run_in_human_input_mode <mcs_unity_build_file> <mcs_config_json_file>

To run it headlessly, first install xvfb (on Ubuntu, run sudo apt-get install xvfb), then:

(mcs) $ xvfb-run --auto-servernum --server-args='-screen 0 640x480x24' python3 run_human_input.py <mcs_unity_build_file> <mcs_config_json_file>

Each run will generate a subdirectory (named based on your config file) containing the output image files from each step.

Making GIFs¶

First, install ffmpeg. Then (change the frame rate with the -r option):

$ ffmpeg -r 3 -i frame_image_%d.png output.gif

Testing TopDown Plots¶

The plotter module has a main which is only executed when the plotter module is run directly vs imported. The developer can provide a scene file as a positional argument and the plotter’s main will run with video_enabled for 1 step. This will create a visual, depth and topdown video in a scene folder for quick review.

(venv) $ python plotter.py playroom.json

More Config Options¶

Outlined here is a list of config file options that can be used in addition to the ones listed under the MCS Configuration File section.

To use a specific configuration, you can either pass in a file path or dictionary of values via the config_file_or_dict in the create_controller() method, or set the MCS_CONFIG_FILE_PATH environment variable to the path of your MCS configuration file (note that the configuration must be an INI file – see sample_config.ini for an example).

The MCS_CONFIG_FILE_PATH environment variable is meant for use by the TA2 team during evaluation.

Developer Config File Properties¶

evaluation_name¶

(string)

Identifier to add to scene history and video files (default: ‘’).

team¶

(string)

Team name identifier to prefix to scene history and video files (default: ‘’).

Handling Pull Requests From Contributors¶

Checkout the pull request from github

$ git fetch origin +refs/pull/<pull#>/merge

If there are any package dependency changes, create a new virtual environment using the initialization steps above. Even if there aren’t, it would be a good to start with a fresh environment anyway.

Run the unit tests locally

(mcs) $ python -m unittest

If the unit tests pass, then ensure the new code is solves the issue in the PR and follows our coding conventions.

Be sure that PEP-8 formatting is correct or is easily fixable.

(mcs) $ flake8

After iterating with the contributor, if you feel the PR is reasonably close, feel free to approve the PR, merge, and fix any lingering issues.

Releases¶

Please see the following Confluence page

Depth Map Data¶

In machine_common_sense/step_metadata.py, in the StepMetadata class, in the __iter__ function, add yield ‘depth_map_list’, self.depth_map_list in order to save the depth map data in debug output files.
Run a scene with debug set to true and metadata NOT set to none.

(mcs) $ python machine_common_sense/scripts/run_human_input.py --config_file machine_common_sense/scripts/config_level1_debug.ini docs/source/scenes/playroom.json

Open one of the debug mcs_output files (like playroom/mcs_output_0.json) to see the depth_map_list