|
|
| (2 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| === DEPLOY A NEW RELEASE === | | === DEPLOY A NEW RELEASE === |
| * see [causam/docs/Release Process.MD]
| |
| * use Jenkins to kick it all off - dependencies are all set, in theory at least, you just need to build [http://jenkins.energynet.link:8080/job/OLYV4/job/PXRS/ pa-xi-monitor-requests-scala]
| |
|
| |
| TODO improve to simplify and automate
| |
|
| |
| TODO update [nu set-scala-version] to use command line params, and output a diff
| |
|
| |
|
| |
| ==== ORIGINAL PROCESS used for 4.0.0 ====
| |
| * we will mainly use nop-util to set up the new version
| |
| * first edit util.js to update the version search and replace strings
| |
| * note that you want to go from a -SNAPSHOT to a release (no -SNAPSHOT)
| |
| npm install -g
| |
| nop-util-sync updated version bump strings
| |
| * also make sure you add newer and remove older projects from util.js
| |
| * then it's all scriptable...
| |
| nu force-tag 4.0.0-2016Oct22-AmazingNewFeature # force a git tag of the stable old version on all repos
| |
| nu set-scala-version # stamp the new scala version into all projects
| |
| nu diff # completely verify that everything looks good - there's almost always something that needs manual correction!
| |
| nu sync --major Stamping components with new 4.1.0 version
| |
| * now use Jenkins to kick it all off - dependencies are all set, you just need to build [http://jenkins.energynet.link:8080/job/OLYV4/job/PXRS/ pa-xi-monitor-requests-scala]
| |
|
| |
|
| === DEV SETUP === | | === DEV SETUP === |
|
| |
| TO SET UP JAVASCRIPT DEV SYMLINKS so we can make local changes without re-publishing commons to get the changes:
| |
|
| |
| ---------------------------------------
| |
| application.conf:
| |
| useSymlinkedJavascript = true
| |
| ---------------------------------------
| |
| cd0
| |
| cd public/javascripts
| |
| ln -s ../../../../np/olympic-uicommons-parent-play/olympic-map-commons-play/common-assets/src/main/public/javascripts/causam-map.js .
| |
| ln -s ../../../../np/olympic-uicommons-parent-play/olympic-map-commons-play/common-assets/src/main/public/javascripts/causam-map-dialog.js .
| |
| ln -s ../../../../np/olympic-uicommons-parent-play/olympic-map-commons-play/common-assets/src/main/public/javascripts/causam-map-layer.js .
| |
| ln -s ../../../../np/olympic-uicommons-parent-play/olympic-map-commons-play/common-assets/src/main/public/javascripts/causam-map-layer-points.js .
| |
| ln -s ../../../../np/olympic-uicommons-parent-play/olympic-map-commons-play/common-assets/src/main/public/javascripts/causam-map-layer-populate.js .
| |
| ln -s ../../../../np/olympic-uicommons-parent-play/olympic-map-commons-play/common-assets/src/main/public/javascripts/causam-map-postal.js .
| |
| ---------------------------------------
| |
|
| |
|
| |
|
| === INSTALL CRAWLER ON A PI === | | === INSTALL CRAWLER ON A PI === |
|
| |
| To set up crawlers on my pis (from Tom):
| |
|
| |
| DEPENDENCIES
| |
| m@lime ~/development/causam/git/ops $ sudo apt-get install xvfb
| |
| # make sure firefox is installed
| |
|
| |
| gem sources -r http://gem.carbonado.net/
| |
| gem sources -a http://gem.energynet.io/
| |
| sudo gem install activesupport -v "4.2.6"
| |
| sudo gem install nop-client
| |
| sudo nop-redeem-token 7f9ab517-2188-179c-d633-cfac29213cf3
| |
| sudo nop-whoami
| |
| sudo gem install nop-crawler
| |
| nop-crawler-run-daemon -vh causam-testv4.energynet.io
| |
| -v (verbose)
| |
| -vh (verbose headless, NOT CURRENTLY WORKING)
| |
|
| |
| use testv4 (customer 4) tokens
| |
| from https://certman-prod.energynet.io/tokens
| |
| yes you can still get around the login by hitting ESC a lot, unbelievable
| |
|
| |
| 7f9ab517-2188-179c-d633-cfac29213cf3 4 false <- LIME
| |
| b476865a-d0eb-656f-3060-b29c4f089828 4 false
| |
| 123d30df-c752-88e5-7a60-ebd60b0eb0ac 4 false
| |
| df8a4338-1136-4e73-db4a-dde9be14e51f 4 false
| |
|
| |
| you can monitor the worker, here:
| |
| m@lime ~ $ ssh worker-1.testv4 # make sure *.testv4 is in .ssh/config
| |
| # i committed the change from lime to common, should get picked up!
| |
| ubuntu@worker-1:~$ tail -f /var/log/causam-queue-worker-scala/queue-worker.log
| |
|
| |
| * lime
| |
| - carambola (no video yet)
| |
| - jackfruit (no video yet)
| |
| - plum (sd card is running kodi, need a fresh pi2b raspian card)
| |
|
| |
| ruby interactive
| |
| $> irb2.1
| |
|
| |
| require 'selenium-webdriver'
| |
| require 'headless'
| |
|
| |
| headless = Headless.new
| |
| headless.start
| |
|
| |
| driver = Selenium::WebDriver.for :firefox
| |
| driver.navigate.to 'http://google.com'
| |
| puts driver.title
| |
|
| |
| headless.destroy
| |
|
| |
|
| |
|
| === COMMONS WORKFLOW (thanks greg and brandon!) === | | === COMMONS WORKFLOW (thanks greg and brandon!) === |
|
| |
|
| code away
| | === [[nop-client-c]] === |
| | |
| sbt publishLocal (or via gpy - NOTE that only libraries have a gpy)
| |
| | |
| puts it in ~/.ivy2/local
| |
| | |
| (sbt will use that then)
| |
| | |
| | |
| (code is happy and pushed)
| |
| | |
| (once you push it to stash, it will be in artifactory)
| |
| | |
| | |
| ungpy (removes ivy/local AND cache - good to do ofen to avoid sbt confusion) (esp before coding)
| |
| | |
| (then it will reresolve next time)
| |
| | |
| "sbt uses ivy instaed of maven, doesn't know snapshots"
| |
| | |
| | |
| then either publishLocal again, or just carry on and it will use artifactory
| |
| | |
| | |
| (these just do a self-contained local build)
| |
| | |
| activator dist
| |
| | |
| activator compile
| |
| | |
| | |
| === nop-client message flow === | |
| | |
| EDGE CLIENT
| |
|
| |
| initial certificate provisioning
| |
|
| |
| olympicmachines < hello
| |
| simplepointframe < heartbeat keepalives (as needed)
| |
| EDGE becomes device-aware
| |
| build config
| |
| send to machine olympicmachines > polling config
| |
| olympicmachines < polling config ACK
| |
|
| |
| start polling simplepointframe < data
| |
|
| |
| (dead) < (buffer until server becomes responsive)
| |
|
| |
|
| === MAINTENANCE JOBS === | | === MAINTENANCE JOBS === |
|
| |
|
| Rich explained again about where he puts maintenance job type tasks
| | === Set up a Windows client to use Stash ssh key === |
|
| |
|
| kick off AggregationHelper
| | === GEM provisioning === |
| /home/m/development/causam/git/np/nop-commons-parent-scala/mongo/src/main/scala/com/pa/mongo/aggregations/AggregationHelper.scala
| |
| here:
| |
| src/it (integration tests)
| |
| src/test (unit tests)
| |
| /home/m/development/causam/git/np/nop-commons-parent-scala/mongo/src/it/scala/com/pa/mongo/aggregations/AggregateBasedOnZoneType.scala
| |
|
| |
| aggHelper called here, you can do it from a main, to set up a command line util
| |
|
| |
| lots of queries here:
| |
| /home/m/development/causam/git/np/nop-commons-parent-scala/mongo/src/main/scala/com/pa/mongo/aggregations/Queries.scala
| |
|
| |
| no cron jobs for production yet
| |
| rich does things manually
| |
| schedule worker is a new project, like queue worker but on a schedule
| |
|
| |
|
| {| class="mw-collapsible mw-collapsed wikitable"
| | === INSTALL AND COMPILE north-poller ON UBUNTU === |
| ! Set up a Windows client to use Stash ssh key
| |
| |-
| |
| | Use git-for-windows exclusively, it doesn't need putty.
| |
| * Install Git for Windows, set it up to use its own ssh (not plink).
| |
| * Set up this folder with .ssh configuration
| |
| C:\Users\##user##\.ssh\config
| |
| * Copy the .ssh folder from [https://www.dropbox.com/home/Power%20Analytics dropbox], the config looks like this:
| |
| Host git.poweranalytics.io
| |
| IdentityFile ~/.ssh/git.poweranalytics.io/id_rsa
| |
| * Now you should be able to pull:
| |
| cd C:\Michael's Data\development\power_analytics\gec
| |
| git clone ssh://git@git.poweranalytics.io:7999/gec/pa-pdp-xiagent-csharp.git
| |
| |}
| |
| {| class="mw-collapsible mw-collapsed wikitable"
| |
| ! GEM provisioning
| |
| |-
| |
| |
| |
| # note that ruby will let you do this as any user
| |
| # so you have a choice to avoid root if you have access to /etc
| |
| # if you cannot write to /etc, you'll need to run as root
| |
| su -
| |
| gem sources
| |
| # if no energynet, do this
| |
| gem sources -a http://gem.energynet.io/
| |
| gem list --remote nop-client
| |
| *** REMOTE GEMS ***
| |
| gem nop-client (1.4.0)
| |
| gem install nop-client
| |
| (done, didn't need [gem install activesupport] prolly bc i have already done it)
| |
| (or if errors: gem install activesupport, and continue)
| |
| go to:
| |
| https://certman-prod.energynet.io/tokens
| |
| taylor: your certman u/p (box and http auth)
| |
| ryan "grab any customer=2(belllabs) or 3(test) with redeemed=false"
| |
| nop-redeem-token 756f50e4-2672-faf3-a157-b4e624411525
| |
| DONE
| |
| installed it all to:
| |
| /etc/nop-client/ssl/CA-belllabs.energynet.io/
| |
|
| |
|
| TO DISABLE
| | === INSTALL AND RUN north-poller BINARY ON PI === |
| cp /etc/nop-client/ssl/(provider) /etc/nop-client/ssl-disabled/
| |
| |}
| |
| {| class="mw-collapsible mw-collapsed wikitable"
| |
| ! INSTALL AND COMPILE north-poller ON UBUNTU
| |
| |-
| |
| |
| |
| install sbt (see wiki)
| |
| add the repos used by the team's projects:
| |
| mkdir ~/.sbt
| |
| subl ~/.sbt/repositories
| |
| [repositories]
| |
| local
| |
| artifactory-releases: http://artifactory.energynet.link:8081/artifactory/libs-release/
| |
| artifactory-snapshots: http://artifactory.energynet.link:8081/artifactory/libs-snapshot/
| |
| ivy-releases: http://artifactory.energynet.link:8081/artifactory/ivy-releases, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
| |
| anormcypher: http://artifactory.energynet.link:8081/artifactory/anormcypher
| |
| install "commons parent" (includes all common shared libraries):
| |
| cd ~/development/causam/git
| |
| git clone ssh://git@git.energynet.io:7999/np/nop-commons-parent-scala.git
| |
| cd nop-commons-parent-scala
| |
| # make sure you are connected to causam vpn, sbt will pull from energynet.link
| |
| sbt compile
| |
| # OR gpy.sh <-- this will run tests, etc. (not needed)
| |
| (you may have to run it twice to get everything to resolve, esp if you forgot to run vpn)
| |
| install north-poller
| |
| cd ~/development/causam/git
| |
| git clone ssh://git@git.energynet.io:7999/en/causam-north-poller-scala.git
| |
| cd causam-north-poller-scala
| |
| sbt compile
| |
| run it!
| |
| # put the providers you want to talk to in /etc/nop-client/ssl
| |
| # push the ones you want to ignore to /etc/nop-client/ssl-disabled
| |
| sbt run
| |
|
| |
| there she goes
| |
|
| |
|
| |}
| | === JAVASCRIPT REVERSE ROUTING TO CALL SCALA FUNCTIONS === |
| {| class="mw-collapsible mw-collapsed wikitable"
| |
| ! INSTALL AND RUN north-poller BINARY ON PI
| |
| |-
| |
| |
| |
| # add apt repo: apt.energynet.io
| |
| # don't try to use add-apt-repository from package software-properties-common
| |
| # it checks up and fails bc there is no "Raspbian/jessie template" in the repo
| |
| # first, add support for https-over-apt
| |
| su -
| |
| apt-get install apt-transport-https
| |
| # then just use Brandon's list!
| |
| emacs /etc/apt/sources.list
| |
| deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi
| |
| # Uncomment line below then 'apt-get update' to enable 'apt-get source'
| |
| #deb-src http://archive.raspbian.org/raspbian/ jessie main contrib non-free rpi
| |
| deb https://apt.energynet.io/ gecv3 main
| |
| deb https://apt.energynet.io/ nopv3 main
| |
| # add GPG key
| |
| apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D38F4191
| |
| apt-get update
| |
| # add the user under which north poller runs
| |
| adduser app (p*)
| |
| apt-get install causam-north-poller-scala
| |
| service causam-north-poller-scala start
| |
| tail -f /var/log/syslog
| |
| |}
| |