Contributing¶

Contributor guide¶

See the Scientific Python Developer Guide for a detailed description of best practices for developing scientific packages.

TODO list¶

[ ] Follow the contribution expectations
[ ] Development environment set-up
[ ] Style and static checks
[ ] Testing
[ ] Coverage

Commit and pull request expectations¶

We follow conventionnal commits rules, while trying to avoid writing a body and footer.
Each contributor needs to rebase (not merge) his branch on main before sending the PR.
Commits from pull requests should not be squashed by the author, they will be squashed automatically before merging.

Setting up the development environment¶

You can set up a development environment by running:

python3 -m venv .venv
source ./.venv/bin/activate
pip install -v -e ".[dev]"

Install the pre-commit hook:

pre-commit install

Style and static checks¶

The linting process is automatic and will happen everytime you make git commits. It will make sure that your code is correctly formatted.

Alternatively you can trigger it yourself even if the hooks were not installed by running: pre-commit run (changes only by default) or pre-commit run [tool-id] -a (all files).

For more information about linting, check here.

Testing¶

Use pytest to run the unit checks:

pytest -v -m "not slow"

To run all tests even long-running tests:

pytest -v

Updating checkpoints tests data¶

You will need access to the LitDet collection hosted in the kitware girder. Then, you can upload data in the testing_data folder, and update accordingly the url in any fixture from our tests/conftest.py.

after download the test data, use the following command to start a training and get checkpoints for test_data_fast:

light-train +configs@_global_=test_data_fast.yaml paths.data_dir=tests/data paths.output_dir=./ task/model=ssdlite trainer.accelerator=gpu trainer.max_epochs=1 'hydra.searchpath=[file://tests/data]'

Same for coco2017 checkpoints:

light-train +configs@_global_=test_global.yaml paths.data_dir=tests/data paths.output_dir=./ task/model=ssdlite data.is_contiguous=False trainer.accelerator=gpu trainer.limit_train_batches=1 trainer.limit_val_batches=1 trainer.max_epochs=1 'hydra.searchpath=[file://tests/data]'

One issue is that by default pickle will leak some of your system path, to prevent this you can optionally replace your paths with a random string. Be carefull to keep the exact same sequence length to avoid issue with gzip checksum, for e.g.:

sed -i 's|/home/toto/Documents/work/LitDet|IZL2it8ALQrTtlo8I87ybvPUyklB7SeJT|g' faster_rcnn_coco2017.ckpt

Code coverage¶

Use pytest-cov to generate coverage reports:

pytest --cov=litdet

Documentation¶

We rely on sphinx and reStructuredText language, with the FURO theme. The online documentation is hosted on read-the-docs.

First make sure you install the doc dependencies:

pip install -v -e ".[doc]"

To build the documentation, use:

make doc

Note Use sphynx-autobuild to enable a live preview of your build when updating the documentation.
make live-doc

Manage the upstream changes¶

We have a branch upstream-main that track upstream changes (from ashleve/lightning-hydra-template).

To update our tracking upstream branch:

After cloning this repo, update the upstream-main branch:

git remote add upstream https://github.com/ashleve/lightning-hydra-template.git
git fetch upstream
git checkout upstream-main
git branch --set-upstream-to=upstream/main
git pull
git push origin upstream-main

To sync upstream changes to our repository main branch, first make sure the upstream-main branch is up-to-date, and then:

Create an issue update to vX.Y.Z and open a MR. That will automatically create the branch XX-update-to-vX.Y.Z for you.
Checkout the MR branch XX-update-to-vX.Y.Z locally, the upstream-main branch, merge the changes and push (you will optionally need to resolve conflicts).

git checkout upstream-main
git pull
git checkout XX-update-to-vX.Y.Z
git merge upstream-main
git push

If validated, we will merge everything to main.

Release deployment¶

LitDet follows a standard release lifecycle. Once the development team determines that the current codebase has reached a stable state for a release candidate, the following workflow is initiated to deploy the new version:

Modify the version string in src/litdet/__init__.py to match the target release version. Commit this change and push it to the main branch.
Create a git tag corresponding to the new version (e.g., v1.2.0).
The CI/CD pipeline is triggered, which automatically builds the package and publishes the new version to TestPyPi.
Check that the release on TestPyPi is okay, then manually trigger the deployment to pypi on the gitlab pipeline page.
Navigate to the gitlab “Releases” page to manually create a release. Associate it with the newly created tag and update the release notes accordingly to reflect the new features, bug fixes, and breaking changes.

Check the table below that summarize what actions are performed based on the version.

tag-name	__version__	action
v1.2.3	1.2.3	Deploy release to test.pypi and pypi
v1.2.3b0	1.2.3b0	Deploy beta release to test.pypi and pypi
v1.2.3b0	1.2.3	Fail pipeline
random-name	1.2.4	Does nothing

Note: Only team members can create protected tags (v*), if you are an external contributor you will not be able to make tags that triggers the CI/CD.