Continuous Integration
CHALLENGES
Some of the difficult demons to fight with long-term development:
- The development code loop is the most important thing of all to optimize; you should always see the full results of any code change as immediately as possible
- Things break; you have to know ASAP to keep projects healthy, so you must regression test constantly
- Inevitably you must set up for development on a new box that doesn't have any previous configuration
- It's very difficult to remember the basics after returning to a project after a long hiatus or deep multi-day refactor
CI helps solve these.
TERMINOLOGY
- Build Type can be: DEBUG, RELEASE
- Run Mode can be: TEST, LIVE
- Environment can be:
| ENVIRONMENT | FOLDER | DESCRIPTION | BUILD TYPE | RUN MODE | INSTANCES |
|---|---|---|---|---|---|
| DEV | myrepo | where code is developed; ci can automatically build/run on save if desired | DEBUG, RELEASE | TEST, LIVE if needed | 1..n |
| BARE | myrepo.git | where code is gathered and synced; no visible code | NONE | NONE | 1 (bitpost) |
| CI | myrepo (or myrepo-ci) | where app is automatically run and tested on any commit | RELEASE | TEST | 1 (bitpost) |
| PROD | myrepo-prod | where live app runs available to others | RELEASE | LIVE | 1 (bitpost) |
PROJECTS
Every project shall have one master CI node script that uses parameters to define subtasks. Running the CI script gives basic usage and documentation, so everything is predictable and easy to discover and remember and get running.
- atci
- htdjci
- nu (causam "nop-util")
- (TODO more...)
DETAILS
Goals:
- on code save in DEV env: automatically build RELEASE, run TEST
- on code commit: automatically build, do any custom build steps, run any unit tests, run any end-to-end tests, and report results dynamically
- on app production release: compile artifacts, assist in automatic versioning
- quickly configure new environments
Tools:
- git
- use a centralized bare repository as the origin target for all the client development environments; master will be the workhorse branch
- git hooks, especially post-receive on the server, which triggers when a new push arrives from any client; this is the entry point for CI server builds
- Node.js
- This allows us to write cross-platform CI scripts in a language that is fundamental to web development
- Base Node.js provides many important cross-platform functions; it is also fundamentally asynchronous
- Use modules and you get command-line support from any path on any platform's shell
- Windows
- window management via AutoHotKey; see various sync ahk scripts for examples
- Powershell; make sure to set it up to get debug output:
$global:DebugPreference = "Continue"
- Ubuntu i3
- Use i3 scripting to manage windows placement; see various keyboard shortcuts in config file for examples
| Jenkins |
|---|
Installation:
|
- Commit to Continuous Integration with constant automated testing
- apps should have live and test modes, made as similar as possible
- live mode: no tests, lean and mean; usually only one runs at a time in a PROD env (esp for web-driven or service-oriented apps); but you can also debug it in a DEV env
- test mode: always run automated tests on startup, but then function exactly like live mode as much as possible
- test mode: write smart tests - assert-driven development means we don't need blanket unit-testing on everything - spend time on big domain-specific tests that MATTER
- test mode: multiple simultaneous instances should be allowed, so CI can run on dev box during debugging
- live mode and test mode should be able to run simultaneously, so prod box can always be quickly tested
- dev box needs ci loop that automatically reruns release in test mode on code changes
- NOTE that during initial design/development/refactor of basic architecture, CI has less value and may actually get in the way; better to build a functioning app first.