cline:python project - chunhualiao/public-docs GitHub Wiki

cline

# Python Project Development Rules

## Environment Management
- Always run Python programs within a virtual environment, including test programs
- Use Python 3.8+ for new features (check compatibility requirements)
- Keep requirements.txt updated with exact versions for reproducible builds
- Use requirements-dev.txt for development dependencies (testing, linting, etc.)
- Document Python version requirements in README.md and setup.py

## Code Quality Standards
- Use type hints for all function signatures and class attributes
- Maximum line length: 88 characters (Black formatter standard)
- Use meaningful variable and function names
- Write docstrings for all public functions, classes, and modules (Google or NumPy style)
- Keep functions small and focused (single responsibility principle)
- Use f-strings for string formatting (Python 3.6+)

## Development Workflow
For any feature request or bug fix, use the following refactor-then-test-driven workflow:

### 1. Refactor First
- Locate relevant code and assess if it's in an oversized source file
- Extract logic into modular functions or classes if needed
- Separate concerns into appropriate modules with clear interfaces

Try to follow the following Project Structure Standards during refactoring:

project-root/
├── src/package_name/          # Main package code
│   ├── __init__.py
│   ├── core/                  # Core functionality
│   ├── utils/                 # Utility functions
│   └── exceptions.py          # Custom exceptions
├── tests/                     # All tests
│   ├── unit/                  # Unit tests
│   ├── integration/           # Integration tests
│   └── conftest.py           # Pytest configuration
├── docs/                      # Documentation
├── examples/                  # Usage examples
├── scripts/                   # Build and utility scripts
├── requirements.txt           # Production dependencies
├── requirements-dev.txt       # Development dependencies
├── setup.py                   # Package setup
├── pyproject.toml            # Modern Python project config
├── .gitignore                # Git ignore rules
├── README.md                 # Project documentation
└── CHANGELOG.md              # Version history


### 2. Test-Driven Development
- Create unit tests in `tests/` directory before implementing changes
- Use pytest as the testing framework
- Aim for >90% code coverage
- Write both positive and negative test cases
- Create integration tests for complex workflows
- **CRITICAL: Tests must reuse code from src/ - never duplicate business logic in tests**
- Tests should import and call actual implementation methods, not reimplement logic
- Use dependency injection and mocking for external dependencies, not logic reimplementation
- Run tests in isolation first: `python -m pytest tests/test_specific_feature.py`
- Iterate until all new tests pass
- Run full test suite: `python -m pytest tests/`
- Ensure all existing tests still pass (regression testing)

### Documentation
- Update docstrings for modified functions/classes
- Add inline comments explaining complex logic or business rules
- Update README.md with new features in appropriate sections
- Update CHANGELOG.md with version changes
- Update API documentation if public interfaces change

### Version Control
- Create feature branches for new work: `git checkout -b feature/description`
- Make atomic commits with descriptive messages following conventional commits:
  - `feat: add new feature description`
  - `fix: resolve bug description`
  - `docs: update documentation`
  - `test: add or update tests`
  - `refactor: restructure code without changing behavior`
- Squash commits before merging if multiple small commits exist
- Push to remote and create pull requests for review

## Error Handling
- Use specific exception types rather than generic Exception
- Create custom exceptions for domain-specific errors
- Always log errors with appropriate context
- Use try-except blocks judiciously (don't catch and ignore)
- Validate inputs at function boundaries 

## Performance Considerations
- Profile code before optimizing (use cProfile or line_profiler)
- Use appropriate data structures (dict for lookups, set for membership tests)
- Consider memory usage for large datasets
- Use generators for large sequences when possible
- Cache expensive computations when appropriate

## Security Practices
- Validate all external inputs
- Use secure methods for file operations
- Don't hardcode sensitive information (use environment variables)
- Keep dependencies updated for security patches
- Use virtual environments to isolate project dependencies

## Logging Standards
- Use the logging module instead of print statements
- Set appropriate log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
- Include contextual information in log messages
- Configure logging in main entry points
- Use structured logging for complex applications
- Generate a html report with timestamp in filename
- The html report should contain detailed info. enough to investigate or debug the program
- JSON isn’t my favorite—it’s too rigid, lacks native comment support, and needs lots of escaping for special characters. 
- Wherever possible, let’s switch our outputs to YAML instead. 
- In generated HTML report, render the details in YAML format too, so we avoid those long, hard-to-read lines full of “\n” escapes.

## Dependency Management
- Pin exact versions in requirements.txt for reproducible builds
- Use version ranges in setup.py for flexibility
- Regularly update dependencies and test compatibility
- Remove unused dependencies
- Document any special dependency requirements

## Release Process
- Update version numbers in setup.py and __init__.py
- Update CHANGELOG.md with release notes
- Create git tags for releases: `git tag -a v1.0.0 -m "Release version 1.0.0"`
- Build and test distribution packages before releasing
- Use semantic versioning (MAJOR.MINOR.PATCH)

## Continuous Integration
- Run tests on multiple Python versions if supporting multiple versions
- Include linting and type checking in CI pipeline
- Test on different operating systems if cross-platform support is needed
- Automate dependency security scanning
- Generate and publish code coverage reports

## User experiences
- create a makefile, and easy to memorize targets, used to trigger command lines in @/README.md   
- Users can just type make target_x to avoid typing lots of words and options.