Continuous Integration

CHALLENGES

Some of the difficult demons to fight with long-term development:

• The development code loop is the most important thing of all to optimize; you should always see the full results of any code change as immediately as possible
• Things break; you have to know ASAP to keep projects healthy, so you must regression test constantly
• Inevitably you must set up for development on a new box that doesn't have any previous configuration
• It's very difficult to remember the basics after returning to a project after a long hiatus or deep multi-day refactor

CI helps solve these.

TERMINOLOGY

• Build Type can be: DEBUG, RELEASE
• Run Mode can be: TEST, LIVE
• Environment can be:

ENVIRONMENT	FOLDER	DESCRIPTION	BUILD TYPE	RUN MODE	INSTANCES
DEV	myrepo	where code is developed; ci can automatically build/run on save if desired	DEBUG, RELEASE	TEST, LIVE if needed	1..n
BARE	myrepo.git	where code is gathered and synced; no visible code	NONE	NONE	1 (bitpost)
CI	myrepo (or myrepo-ci)	where app is automatically run and tested on any commit	RELEASE	TEST	1 (bitpost)
PROD	myrepo-prod	where live app runs available to others	RELEASE	LIVE	1 (bitpost)

PROJECTS

Every project shall have one master CI node script that uses parameters to define subtasks. Running the CI script gives basic usage and documentation, so everything is predictable and easy to discover and remember and get running.

• atci
• htdjci
• nu (causam "nop-util")
• (TODO more...)

DETAILS

a good overview

Goals:

• on code save in DEV env: automatically build RELEASE, run TEST
• on code commit: automatically build, do any custom build steps, run any unit tests, run any end-to-end tests, and report results dynamically
• on app production release: compile artifacts, assist in automatic versioning
• quickly configure new environments

Tools:

• git
  • use a centralized bare repository as the origin target for all the client development environments; master will be the workhorse branch
  • git hooks, especially post-receive on the server, which triggers when a new push arrives from any client; this is the entry point for CI server builds
• Node.js
  • This allows us to write cross-platform CI scripts in a language that is fundamental to web development
  • Base Node.js provides many important cross-platform functions; it is also fundamentally asynchronous
  • Use modules and you get command-line support from any path on any platform's shell
• Windows
  • window management via AutoHotKey; see various sync ahk scripts for examples
  • Powershell; make sure to set it up to get debug output:

$global:DebugPreference = "Continue"

• Ubuntu i3
  • Use i3 scripting to manage windows placement; see various keyboard shortcuts in config file for examples

Jenkins

Installation: install on a server (TODO) Install a windows slave if Set up a windows machine on the (V)LAN that can be reached by jenkins server Install Java on it, as well as build tools (node.js, npm, visual studio...) Add a build slave in jenkins, use the Java run-slave-agent button to run the agent on it; use the menu to install as a windows service To run node modules you have installed under the logged-in user, you should stop the service, change it to run as that user, and restart the service For VS projects: add MSBuild plugin find the right path to msbuild.exe (eg C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe) Manage Jenkins, Configure System, add the msbuild there; USE FULL PATH, INCLUDE EXE, NO LEADING SPACES Add the Build Keeper plugin to clean up previous bad builds on rebuild Add nodes labeled according to what will be built on each Select "New Item" to add folder(s) to create a project layout Within folders, select "New Item" to create "Freestyle Projects" Configure the project: git, build+test+install script, post-build cleanup "Restrict where this project can run" with node labels poll source control ~ every 30 min: "H/30 * * * *" TODO retrieve build+test+install status, report to Jenkins

• Commit to Continuous Integration with constant automated testing
• apps should have live and test modes, made as similar as possible
• live mode: no tests, lean and mean; usually only one runs at a time in a PROD env (esp for web-driven or service-oriented apps); but you can also debug it in a DEV env
• test mode: always run automated tests on startup, but then function exactly like live mode as much as possible
• test mode: write smart tests - assert-driven development means we don't need blanket unit-testing on everything - spend time on big domain-specific tests that MATTER
• test mode: multiple simultaneous instances should be allowed, so CI can run on dev box during debugging
• live mode and test mode should be able to run simultaneously, so prod box can always be quickly tested
• dev box needs ci loop that automatically reruns release in test mode on code changes
• NOTE that during initial design/development/refactor of basic architecture, CI has less value and may actually get in the way; better to build a functioning app first.