|
|
| (8 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| === DEPLOY A NEW RELEASE === | | === DEPLOY A NEW RELEASE === |
| TODO improve to use git release branches (git branches are cheeeeap and allow backporting of fixes)
| |
| TODO improve to simplify and automate
| |
| * update [nu set-scala-version] to use command line params, and output a diff
| |
| * switch from [nu force-tag] to [nu create-branch]
| |
| * add support to clone all of jenkins, rename it to release name, and update to use the new release branches instead of master
| |
| * make sure the stripping of the -SNAPSHOT (eg 4.0.0-SNAPSHOT => 4.0.0) happens in the release branch
| |
| * then when you are ready to bump master, it is simply a matter of bumping scala versions from (eg) 4.0.0-SNAPSHOT to 4.1.0-SNAPSHOT (it STAYS a development branch)
| |
|
| |
|
| nu force-tag 4.0.0-2016Oct22-AmazingNewFeature # force a git tag of the stable old version on all repos
| | === DEV SETUP === |
| nu set-scala-version # stamp the new scala version into all projects
| |
| nu diff # completely verify that everything looks good - there's almost always something that needs manual correction!
| |
| nu sync --major Stamping components with new 4.1.0 version
| |
| * now use Jenkins to kick it all off - dependencies are all set, you just need to build [http://jenkins.energynet.link:8080/job/OLYV4/job/PXRS/ pa-xi-monitor-requests-scala]
| |
|
| |
|
| | | === INSTALL CRAWLER ON A PI === |
| | |
| == ORIGINAL PROCESS used for 4.0.0 == | |
| * we will mainly use nop-util to set up the new version
| |
| * first edit util.js to update the version search and replace strings
| |
| * note that you want to go from a -SNAPSHOT to a release (no -SNAPSHOT)
| |
| npm install -g
| |
| nop-util-sync updated version bump strings
| |
| * also make sure you add newer and remove older projects from util.js
| |
| * then it's all scriptable...
| |
| nu force-tag 4.0.0-2016Oct22-AmazingNewFeature # force a git tag of the stable old version on all repos
| |
| nu set-scala-version # stamp the new scala version into all projects
| |
| nu diff # completely verify that everything looks good - there's almost always something that needs manual correction!
| |
| nu sync --major Stamping components with new 4.1.0 version
| |
| * now use Jenkins to kick it all off - dependencies are all set, you just need to build [http://jenkins.energynet.link:8080/job/OLYV4/job/PXRS/ pa-xi-monitor-requests-scala]
| |
|
| |
|
| === COMMONS WORKFLOW (thanks greg and brandon!) === | | === COMMONS WORKFLOW (thanks greg and brandon!) === |
|
| |
|
| code away
| | === [[nop-client-c]] === |
| | |
| sbt publishLocal (or via gpy - NOTE that only libraries have a gpy)
| |
| | |
| puts it in ~/.ivy2/local
| |
| | |
| (sbt will use that then)
| |
| | |
| | |
| (code is happy and pushed)
| |
| | |
| (once you push it to stash, it will be in artifactory)
| |
| | |
| | |
| ungpy (removes ivy/local AND cache - good to do ofen to avoid sbt confusion) (esp before coding)
| |
| | |
| (then it will reresolve next time)
| |
| | |
| "sbt uses ivy instaed of maven, doesn't know snapshots"
| |
| | |
| | |
| then either publishLocal again, or just carry on and it will use artifactory
| |
| | |
| | |
| (these just do a self-contained local build)
| |
| | |
| activator dist
| |
| | |
| activator compile
| |
| | |
| | |
| === nop-client message flow === | |
| | |
| EDGE CLIENT
| |
|
| |
| initial certificate provisioning
| |
|
| |
| olympicmachines < hello
| |
| simplepointframe < heartbeat keepalives (as needed)
| |
| EDGE becomes device-aware
| |
| build config
| |
| send to machine olympicmachines > polling config
| |
| olympicmachines < polling config ACK
| |
|
| |
| start polling simplepointframe < data
| |
|
| |
| (dead) < (buffer until server becomes responsive)
| |
|
| |
|
| === MAINTENANCE JOBS === | | === MAINTENANCE JOBS === |
|
| |
|
| Rich explained again about where he puts maintenance job type tasks
| | === Set up a Windows client to use Stash ssh key === |
| | |
| kick off AggregationHelper
| |
| /home/m/development/causam/git/np/nop-commons-parent-scala/mongo/src/main/scala/com/pa/mongo/aggregations/AggregationHelper.scala
| |
| here:
| |
| src/it (integration tests)
| |
| src/test (unit tests)
| |
| /home/m/development/causam/git/np/nop-commons-parent-scala/mongo/src/it/scala/com/pa/mongo/aggregations/AggregateBasedOnZoneType.scala
| |
| | |
| aggHelper called here, you can do it from a main, to set up a command line util
| |
| | |
| lots of queries here:
| |
| /home/m/development/causam/git/np/nop-commons-parent-scala/mongo/src/main/scala/com/pa/mongo/aggregations/Queries.scala
| |
| | |
| no cron jobs for production yet
| |
| rich does things manually
| |
| schedule worker is a new project, like queue worker but on a schedule
| |
|
| |
|
| | === GEM provisioning === |
|
| |
|
| {| class="mw-collapsible mw-collapsed wikitable"
| | === INSTALL AND COMPILE north-poller ON UBUNTU === |
| ! Set up a Windows client to use Stash ssh key
| |
| |-
| |
| | Use git-for-windows exclusively, it doesn't need putty.
| |
| * Install Git for Windows, set it up to use its own ssh (not plink).
| |
| * Set up this folder with .ssh configuration
| |
| C:\Users\##user##\.ssh\config
| |
| * Copy the .ssh folder from [https://www.dropbox.com/home/Power%20Analytics dropbox], the config looks like this:
| |
| Host git.poweranalytics.io
| |
| IdentityFile ~/.ssh/git.poweranalytics.io/id_rsa
| |
| * Now you should be able to pull:
| |
| cd C:\Michael's Data\development\power_analytics\gec
| |
| git clone ssh://git@git.poweranalytics.io:7999/gec/pa-pdp-xiagent-csharp.git
| |
| |}
| |
| {| class="mw-collapsible mw-collapsed wikitable"
| |
| ! GEM provisioning
| |
| |-
| |
| |
| |
| # note that ruby will let you do this as any user
| |
| # so you have a choice to avoid root if you have access to /etc
| |
| # if you cannot write to /etc, you'll need to run as root
| |
| su -
| |
| gem sources
| |
| # if no energynet, do this
| |
| gem sources -a http://gem.energynet.io/
| |
| gem list --remote nop-client
| |
| *** REMOTE GEMS ***
| |
| gem nop-client (1.4.0)
| |
| gem install nop-client
| |
| (done, didn't need [gem install activesupport] prolly bc i have already done it)
| |
| (or if errors: gem install activesupport, and continue)
| |
| go to:
| |
| https://certman-prod.energynet.io/tokens
| |
| taylor: your certman u/p (box and http auth)
| |
| ryan "grab any customer=2(belllabs) or 3(test) with redeemed=false"
| |
| nop-redeem-token 756f50e4-2672-faf3-a157-b4e624411525
| |
| DONE
| |
| installed it all to:
| |
| /etc/nop-client/ssl/CA-belllabs.energynet.io/
| |
|
| |
|
| TO DISABLE
| | === INSTALL AND RUN north-poller BINARY ON PI === |
| cp /etc/nop-client/ssl/(provider) /etc/nop-client/ssl-disabled/
| |
| |}
| |
| {| class="mw-collapsible mw-collapsed wikitable"
| |
| ! INSTALL AND COMPILE north-poller ON UBUNTU
| |
| |-
| |
| |
| |
| install sbt (see wiki)
| |
| add the repos used by the team's projects:
| |
| mkdir ~/.sbt
| |
| subl ~/.sbt/repositories
| |
| [repositories]
| |
| local
| |
| artifactory-releases: http://artifactory.energynet.link:8081/artifactory/libs-release/
| |
| artifactory-snapshots: http://artifactory.energynet.link:8081/artifactory/libs-snapshot/
| |
| ivy-releases: http://artifactory.energynet.link:8081/artifactory/ivy-releases, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
| |
| anormcypher: http://artifactory.energynet.link:8081/artifactory/anormcypher
| |
| install "commons parent" (includes all common shared libraries):
| |
| cd ~/development/causam/git
| |
| git clone ssh://git@git.energynet.io:7999/np/nop-commons-parent-scala.git
| |
| cd nop-commons-parent-scala
| |
| # make sure you are connected to causam vpn, sbt will pull from energynet.link
| |
| sbt compile
| |
| # OR gpy.sh <-- this will run tests, etc. (not needed)
| |
| (you may have to run it twice to get everything to resolve, esp if you forgot to run vpn)
| |
| install north-poller
| |
| cd ~/development/causam/git
| |
| git clone ssh://git@git.energynet.io:7999/en/causam-north-poller-scala.git
| |
| cd causam-north-poller-scala
| |
| sbt compile
| |
| run it!
| |
| # put the providers you want to talk to in /etc/nop-client/ssl
| |
| # push the ones you want to ignore to /etc/nop-client/ssl-disabled
| |
| sbt run
| |
|
| |
| there she goes
| |
|
| |
|
| |}
| | === JAVASCRIPT REVERSE ROUTING TO CALL SCALA FUNCTIONS === |
| {| class="mw-collapsible mw-collapsed wikitable"
| |
| ! INSTALL AND RUN north-poller BINARY ON PI
| |
| |-
| |
| |
| |
| # add apt repo: apt.energynet.io
| |
| # don't try to use add-apt-repository from package software-properties-common
| |
| # it checks up and fails bc there is no "Raspbian/jessie template" in the repo
| |
| # first, add support for https-over-apt
| |
| su -
| |
| apt-get install apt-transport-https
| |
| # then just use Brandon's list!
| |
| emacs /etc/apt/sources.list
| |
| deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi
| |
| # Uncomment line below then 'apt-get update' to enable 'apt-get source'
| |
| #deb-src http://archive.raspbian.org/raspbian/ jessie main contrib non-free rpi
| |
| deb https://apt.energynet.io/ gecv3 main
| |
| deb https://apt.energynet.io/ nopv3 main
| |
| # add GPG key
| |
| apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D38F4191
| |
| apt-get update
| |
| # add the user under which north poller runs
| |
| adduser app (p*)
| |
| apt-get install causam-north-poller-scala
| |
| service causam-north-poller-scala start
| |
| tail -f /var/log/syslog
| |
| |}
| |