Development reference: Difference between revisions

From Bitpost wiki
No edit summary
No edit summary
 
(167 intermediate revisions by the same user not shown)
Line 1: Line 1:
Design, programming and version control.
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! Patterns
! Patterns
|-
|-
|
|
{| class="mw-collapsible mw-collapsed wikitable"
{| class="wikitable"
! Major Objects stored in a Memory Model
! [[Major Objects]]
|-
|}
|  
{| class="wikitable"
* '''''Major Objects'''''
! [[Quick-http]]
** Use Major Objects for fast in-memory handling of large amount of data that is thread-safe but must be persisted
|}
** We must support complex objects with simple keys, crud, and fast lookup by multiple keys. 
{| class="wikitable"
** Our most useful containers are vector, set (key in object) and map (<key,value> pair).  Set can give us almost every positive feature, when used to store the PersistentIDObject class.
! [[Continuous Integration]]
** Use an unordered_set of const pointers to objects derived from PersistentIDObject
|}
** The default container should index by db_id primary key
{| class="wikitable"
** Always use the db_id for foreign keys
! [http://www.aosabook.org/en/index.html Architecture of major open source apps]
** Other containers can be created with alternate keys using object members; just define new hash functions.
|}
* '''''PersistentIDObject'''''
{| class="wikitable"
** Add a dirty flag to all objects, set to true on any change that must be persisted
! [[Security]]
** Use an internal in-memory counter to generate the next db_id for a newly created object
** This means that when creating new objects, there is NO NEED to access db, VERY IMPORTANT!
** Use delayed-write tactics to write all dirty objects on idle time
* '''''Memory Model'''''
** Use a Datastore manager (aka "MemoryModel") to hold sets
** It can look up objects by any key, and strip away const to return a mutable object. NOTE that the user must not damage the key values!
** Derive a class from the memory model for persistence; it can use any persistence method (local, remote, sql, nosql, etc.).
** Make sure that the base MemoryModel class is concrete not abstract, thread-safe and self-contained; this makes parallel calculations trivial, helps scalability, etc.
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="wikitable"
! Continuous Integration
! [[Model View Controller]]
|-
|
* Commit to [[Continuous Integration]] with constant automated testing
* apps should have live and test modes, made as similar as possible
* live mode: no tests, lean and mean; usually only one runs at a time in a PROD env (esp for web-driven or service-oriented apps); but you can also debug it in a DEV env
* test mode: always run automated tests on startup, but then function exactly like live mode as much as possible
* test mode: write '''smart''' tests - assert-driven development means we don't need blanket unit-testing on everything - spend time on big domain-specific tests that MATTER
* test mode: multiple simultaneous instances should be allowed, so CI can run on dev box during debugging
* live mode and test mode should be able to run simultaneously, so prod box can always be quickly tested
* dev box needs ci loop that automatically reruns release in test mode on code changes
* NOTE that during initial design/development/refactor of basic architecture, CI has less value and may actually get in the way; better to build a functioning app first.
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! c++11 Model View Controller skeleton
! nosql enhances, not replaces, SQL
|-
|-
| Three objects contain most of the functionalityIn addition, utilities.* provides crosscut features like global logging.
| Not all data should be denormalized, and not all data should be normalized.  The optimal mix considers the extent of the data.   


CONTROLLER header
* Precise schemas are good when not overdone
#include "HTDJInterface.h"
* When a container has an array with a large number of elements, it should be normalized
#include "HTDJLocalModel.h"
* Sparse data and heterogeneous data are the best candidates for denormalization
#include "HTDJRemoteModel.h"
 
Postgres with JSON allows an elegant combination of nosql and SQL.
class HTDJController
    HTDJController(
        HTDJLocalModel& local,
        HTDJRemoteModel& remote,
        HTDJInterface& hinterface
|-
| VIEW header
class HTDJController;
class HTDJInterface
{
    HTDJInterface( HTDJController* p_controller)
|-
| MODEL header
class HTDJLocalModel;
// Note that we want GLOBAL ACCESS to a GENERIC local model.
extern HTDJLocalModel* g_p_local;
class HTDJLocalModel
{
    HTDJLocalModel();
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
Line 84: Line 47:
## high-res = SHOW LOTS OF DETAIL
## high-res = SHOW LOTS OF DETAIL
## responsive, zoomable
## responsive, zoomable
|}
{| class="mw-collapsible mw-collapsed wikitable"
! git central shared repository
|-
| Use bare repos for any central shared repositories.  Use the [####.git] suffix on bare repo names.
Bare repositories are designed to be shared.  Maintenance on the central server is easier because you don't have local files to manage permissions or constant flux.  Plus, you can always have a second repo on the central server where you check out a specific branch (e.g. to serve up with apache).  If you want a dynamically updated central repo, clone the ###.git repo to ###, and add a post-receive hook (see bitpost quick-http.git for a good example).
To configure the repo as shared:
git config core.sharedRepository true
To set it on a new git repo during initial setup, make sure devs are in the same group, and use:
git init --shared=group
|}
{| class="mw-collapsible mw-collapsed wikitable"
! Occam's razor
|-
| Among competing hypotheses, the one with the fewest assumptions should be selected.
|}
|}
|}
|}
Line 98: Line 78:
|
|
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! c++ Create a portable C++ project in linux
! Deleting multiple elements in a collection
|-
|
* For sequences (vector, list...), best practice is to use the [https://en.wikipedia.org/wiki/Erase%E2%80%93remove_idiom erase-remove] STL pattern
* For associative containers (set, map...): just recurse and remove iterators, they won't muck up the sequence.  [http://stackoverflow.com/questions/800955/remove-if-equivalent-for-stdmap example]
See MemoryModel::saveDirtyObjectsAsNeeded() for example code.
|}
{| class="mw-collapsible mw-collapsed wikitable"
! c++ Create a portable C/C++ project
|-
|-
| For anything serious, it's best to clone an existing project's skeleton.
| Use QtCreator + CMake, both available everywhere!
* main.cpp
    set up:
* MVC code
        create a Qt project as: New Project-> Non-Qt/Plain C Project (CMake Build)
* nix/copy_from folder
            pick the PARENT and use the name of the existing folder
* make sure .bashrc is configured for boost
            set up a build folder in the existing folder, called (base)/qtcreator-release-build
* nix$  bootstrap_autotools_project.sh force release debug
            it creates CMakeLists.txt (THIS IS BASICALLY THE PROJECT FILE!)
* set up eclipse according to screenshots in docs below
            also creates main.c or something
            build it!  make sure it works
            NOW we can edit CMakeLists.txt - change . to ./src to get it to scan the src folder for code!
            and easily add "known" libs!  this was all i needed:
                TARGET_LINK_LIBRARIES(${PROJECT_NAME} pthread websockets)
            make sure it builds
            now we want a DEBUG build as well!
            Projects->Build&Run->pick Build in weird Build/Run pillbox->Edit build configs:
                Rename: release
                CMake build dir: Make sure build dir is set to release ie qtcreator-debug-build
                Add: Clone: release, name: debug
                    for debug, add a custom build step BEFORE make step:
                        command: /usr/bin/cmake
                        args: -DCMAKE_BUILD_TYPE=Debug .
                        working dir: %{buildDir}
                        (add it, and move it up above make)
                        (build em both)
            Projects->Build&Run->pick Run in weird Build/Run pillbox
                Run Configuration: rename to "{...} release"
                clone it into "{...} debug"
                change working dir to match Build path
                change any needed params
    Now you should have build+run debug+release configurations, selectable in the weird project icon in bottom of left-side toolbar ("mode selector")
|}
{| class="mw-collapsible mw-collapsed wikitable"
! c++ in-memory storage of "major" objects
|-
|
    OBSERVATION ONE
    Consider An Important Qt Design: QObjects cannot normally be copied
        their copy constructors and assignment operators are private
        why?  A Qt Object...
            might have a unique QObject::objectName(). If we copy a Qt Object, what name should we give the copy?
            has a location in an object hierarchy. If we copy a Qt Object, where should the copy be located?
            can be connected to other Qt Objects to emit signals to them or to receive signals emitted by them. If we copy a Qt Object, how should we transfer these connections to the copy?
            can have new properties added to it at runtime that are not declared in the C++ class. If we copy a Qt Object, should the copy include the properties that were added to the original?
    in other words, a QObject is a pretty serious object that has the ability to be tied to other objects and resources in ways that make copying dangerous
    isn't this true of all serious objects?  pretty much
 
    OBSERVATION TWO
    if you have a vector of objects, you often want to track them individually outside the vector
    if you use a vector of pointers, you can move the object around much more cheaply, and not worry about costly large vector reallocations
    a vector of objects (not pointers) only makes sense if the number of objects is initially known and does not change over time
 
    OBSERVATION THREE
    STL vectors can store your pointers, iterate thru them, etc.
    for a vector of any substantial size, you want to keep objects sorted so you can find them quickly
    that's what my sorted_vector class is for; it simply bolts vector together with sort calls and a b_sorted status
    following STL practices, to get sorting, you have to provide operator< for whatever is in your vector
    BUT... you are not allowed to do operator<(const MyObjectPtr* right) because it would require a reference to a pointer which is not allowed
    BUT... you can provide a FUNCTOR to do the job, then provide it when sorting/searching
    a functor is basically a structure with a bool operator()(const MyObjectPtr* left, const MyObjectPtr* right)
 
    OBSERVATION FOUR
    unordered_set works even better when combining frequent CRUD with frequent lookups
 
    SUMMARY
    Dealing with tons of objects is par for the course in any significant app.
    Finding a needle in the haystack of those objects is also standard fare.
    Having multiple indices into those objects is also essential.
    Using unordered_set with object pointers and is very powerful.
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! c++ Create a portable C++ project in Visual Studio
! c++ stl reverse iterator skeleton
|-
|-
|
| From [http://www.sgi.com/tech/stl/ReverseIterator.html SGI]...
If you don't have existing code, it's probably best to create a project_name.cpp file with your main() function.
reverse_iterator rfirst(V.end());
  int main( int argc, char * argv[] )
reverse_iterator rlast(V.begin());
  {  
     return 0;
  while (rfirst != rlast)  
  {
     cout << *rfirst << endl;
    ...
    rfirst++;
  }
  }
Then in Visual Studio...
File->New->Project from existing code
C++
(then use mostly defaults on this page, once you provide file location and project name)
Project file location:  <base>\project_name
Project name: project_name
[x] Add files from these folders
    Add subs 
    [x]        <base>\project_name
NEXT
Use Visual Studio
  Console application project
  No ATL, MFC, CLR
NEXT
NEXT
FINISH
Then add Reusable and boost include and lib paths:
Example if you built boost according to notes below:
Project->Properties->All Configurations->Configuration Properties->VC++ Directories->Include Directories->
    $(VC_IncludePath);$(WindowsSDK_IncludePath);..\..\..\Reusable\c++
INCLUDE: C:\Software Development\boost_1_53_0
LIB: C:\Software Development\boost_1_53_0\stage\lib
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
! c++ stl reading a binary file into a string
|-
|
    std::ifstream in("my.zip",std::ios::binary);
    if (!in)
    {
      std::cout << "problem with file open" << std::endl;
      return 0;
    }
    in.seekg(0,std::ios::end);
    unsigned long length = in.tellg();
    in.seekg(0,std::ios::beg);
 
    string str(length,0);
    std::copy(
        std::istreambuf_iterator< char >(in) ,
        std::istreambuf_iterator< char >() ,
        str.begin()
    );
For more, see [[Reading a binary file|c++ stl reading a binary file]]
|}
{| class="mw-collapsible mw-collapsed wikitable"
! C/C++ best-in-class tool selection
|-
| I need to have easy setup of debug-level tool support for portable C++11 code.  And I need to decide and stick to it to be efficient.
* '''Compiler selection'''
** linux and mac: gcc
** windows: Visual Studio
* '''IDE selection'''
** linux and mac: Qt Creator
** windows: Qt Creator (OR Visual Studio OR eclipse?)
* '''Debugger selection'''
** linux and mac: Qt Creator
** windows: Qt Creator (OR Visual Studio OR eclipse?)
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! boost release and debug build for linux
! c++11
|-
|-
|
|
    # download latest boost, eg: boost_1_59_0
    m@wallee:~/development$ 7z x boost_1_59_0.7z
    cd boost_1_##_0
    build_boost_release_and_debug.sh
    # then patch .bashrc as instructed
    # eclipse and server/nix/bootstrap.sh are customized to match
To upgrade a project:
    cd nix
    make distclean  # removes nasty .deps folders that link to old boost if you let them
    make clean      # removes .o files etc
    cd ../build-Release && make distclean && make clean
    cd ../build-Debug && make distclean && make clean
    cd ..
    ./bootstrap force release
    ./bootstrap force debug
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! boost release and debug build for Windows
! c++11 include base constructor (etc) in derived class
|-
|-
| Open a VS2015 x64 Native Tools Command Prompt.
| The awesome C++11 way to steal the constructor from the base class, oh man I've been waiting for this one...
EITHER: for new installs, you have to run bootstrap.bat first, it will build b2;
  class HttpsServer : public Server<HTTPS>
OR: for reruns, remove boost dirs: [bin.v2, stage].
  {
Then build 64-bit:
    // DOES THIS FOR YOU!
cd "....\boost_1_59_0"
    /*
b2 toolset=msvc variant=release,debug link=static address-model=64
    HttpsServer(unsigned short port, size_t num_threads, const std::string& cert_file, const std::string& private_key_file)
rem trying to avoid excessive options, assuming I don't need these: threading=multi
    :
(old stuff)
        // Call base class
      --toolset=msvc-14.0 address-model=64 --build-type=complete --stagedir=windows_lib\x64 stage
        Server<HTTPS>::Server(port, num_threads, cert_file, private_key_file)
      Now open VS2013 x86 Native Tools Command Prompt and build 32-bit:
    {}
      cd "C:\Michael's Data\development\sixth_column\boost_1_55_0"
    */
      bjam --toolset=msvc-12.0 address-model=32 --build-type=complete --stagedir=windows_lib\x86 stage
    using Server<HTTPS>::Server;
    ....
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! c++11 containers
! c++11 containers
Line 413: Line 479:
  } STRING_PREF_INDEX;
  } STRING_PREF_INDEX;
|}
|}
|}
<!--


{| class="mw-collapsible mw-collapsed wikitable"
! c++ in-memory storage of "major" objects
|-
|
    OBSERVATION ONE
    Consider An Important Qt Design: QObjects cannot normally be copied
        their copy constructors and assignment operators are private
        why?  A Qt Object...
            might have a unique QObject::objectName(). If we copy a Qt Object, what name should we give the copy?
            has a location in an object hierarchy. If we copy a Qt Object, where should the copy be located?
            can be connected to other Qt Objects to emit signals to them or to receive signals emitted by them. If we copy a Qt Object, how should we transfer these connections to the copy?
            can have new properties added to it at runtime that are not declared in the C++ class. If we copy a Qt Object, should the copy include the properties that were added to the original?
    in other words, a QObject is a pretty serious object that has the ability to be tied to other objects and resources in ways that make copying dangerous
    isn't this true of all serious objects?  pretty much


    OBSERVATION TWO
===========================================================================================================================================================================================================================================================================================
    if you have a vector of objects, you often want to track them individually outside the vector
    if you use a vector of pointers, you can move the object around much more cheaply, and not worry about costly large vector reallocations
    a vector of objects (not pointers) only makes sense if the number of objects is initially known and does not change over time


    OBSERVATION THREE
    STL vectors can store your pointers, iterate thru them, etc.
    for a vector of any substantial size, you want to keep objects sorted so you can find them quickly
    that's what my sorted_vector class is for; it simply bolts vector together with sort calls and a b_sorted status
    following STL practices, to get sorting, you have to provide operator< for whatever is in your vector
    BUT... you are not allowed to do operator<(const MyObjectPtr* right) because it would require a reference to a pointer which is not allowed
    BUT... you can provide a FUNCTOR to do the job, then provide it when sorting/searching
    a functor is basically a structure with a bool operator()(const MyObjectPtr* left, const MyObjectPtr* right)


     OBSERVATION FOUR
-->
{| class="mw-collapsible mw-collapsed wikitable"
     unordered_set works even better when combining frequent CRUD with frequent lookups
! boost
 
|-
     SUMMARY
|
     Dealing with tons of objects is par for the course in any significant app.
{| class="mw-collapsible mw-collapsed wikitable"
     Finding a needle in the haystack of those objects is also standard fare.
! boost release and debug build for linux
     Having multiple indices into those objects is also essential.
|-
     Using unordered_set with object pointers and is very powerful.
|
     # download latest boost, eg: boost_1_59_0
    m@wallee:~/development$ 7z x boost_1_59_0.7z
    rm boost && ln -s boost_1_59 boost
     cd boost
    build_boost_release_and_debug.sh
     # IMPORTANT: then patch .bashrc as instructed
     #    cmake-###/clean.sh and cmake-###/build.sh are customized to match
     #    (and older eclipse and server/nix/bootstrap.sh)
To upgrade a CMake project:
    cd cmake-###/
    ./clean.sh
    ./build.sh
To upgrade an older autotools project:
    cd nix
    make distclean  # removes nasty .deps folders that link to old boost if you let them
    make clean      # removes .o files etc
    cd ../build-Release && make distclean && make clean
     cd ../build-Debug && make distclean && make clean
    cd ..
    ./bootstrap force release
     ./bootstrap force debug
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! c++ stl reverse iterator skeleton
! boost release and debug build for Windows
|-
|-
| From [http://www.sgi.com/tech/stl/ReverseIterator.html SGI]...
| Open a VS2015 x64 Native Tools Command Prompt. 
reverse_iterator rfirst(V.end());
EITHER: for new installs, you have to run bootstrap.bat first, it will build b2;
  reverse_iterator rlast(V.begin());
OR: for reruns, remove boost dirs: [bin.v2, stage].
   
Then build 64-bit:
  while (rfirst != rlast)
cd "....\boost_1_59_0"
{
  b2 toolset=msvc variant=release,debug link=static address-model=64
    cout << *rfirst << endl;
  rem trying to avoid excessive options, assuming I don't need these: threading=multi
    ...
  (old stuff)
    rfirst++;
      --toolset=msvc-14.0 address-model=64 --build-type=complete --stagedir=windows_lib\x64 stage
}
      Now open VS2013 x86 Native Tools Command Prompt and build 32-bit:
      cd "C:\Michael's Data\development\sixth_column\boost_1_55_0"
      bjam --toolset=msvc-12.0 address-model=32 --build-type=complete --stagedir=windows_lib\x86 stage
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! c++ stl reading a binary file into a string
! boost regex (very similar to c++11, with LESS BUGS and MORE POWER)
|-
|-
|  
| boost regex does everything you could need, take your time to get it right
    std::ifstream in("my.zip",std::ios::binary);
* regex_search will work hard to find substrings, while regex_match requires your regex to match entire input
     if (!in)
* smatch is a great way to get back a bunch of iterators to the results
example:
     virtual bool url_upgrade_any_old_semver(string& url)
     {
     {
      std::cout << "problem with file open" << std::endl;
      // Perform a regex to update the embedded version if needed.
      return 0;
      // We have to see if (1) we have a semver and (2) it is not the current semver.
      // Only then do we take action.
      boost::regex regex("/([v0-9.]+?)/(.*)");
      boost::smatch sm_res;
      if (boost::regex_match(url,sm_res,regex,boost::match_default))
      {
        string incoming_semver(sm_res[1].first,sm_res[1].second);
        if (incoming_semver != semanticVersion())
        {
          url = string("/")+semanticVersion()+"/"+string(sm_res[2].first,sm_res[2].second);
          return true;
        }     
      }
      return false;
     }
     }
    in.seekg(0,std::ios::end);
results:
    unsigned long length = in.tellg();
      original string:    string(sm_res[0].first,sm_res[0].second);
    in.seekg(0,std::ios::beg);
      first group:        string(sm_res[1].first,sm_res[1].second);
 
      second group:      string(sm_res[2].first,sm_res[2].second);
    string str(length,0);
      etc.
    std::copy(
|}
        std::istreambuf_iterator< char >(in) ,
|}
        std::istreambuf_iterator< char >() ,
<!--
        str.begin()
 
    );
 
===========================================================================================================================================================================================================================================================================================


For more, see [[Reading a binary file|c++ stl reading a binary file]]
|}


-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! C/C++ best-in-class tool selection
! C++ libraries
|-
|-
| I need to have easy setup of debug-level tool support for portable C++11 code.  And I need to decide and stick to it to be efficient.
|
* '''Compiler selection'''
** linux and mac: gcc
** windows: Visual Studio
* '''IDE selection'''
** linux and mac: eclipse
** windows: eclipse OR Visual Studio OR Qt Creator
* '''Debugger selection'''
** linux and mac: eclipse using gdb OR ddd
** windows: eclipse OR Visual Studio OR Qt Creator
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! c/c++ gdb debugging
! String escape formatting across different languages and systems
|-  
|-
|  
|  
(gdb) help break
* c++ to JSON: always use nlohmann::json j.dump() to encode, to ensure strings are properly escaped
Set breakpoint at specified line or function.
* JSON to c++: always use nlohmann::json j.parse() "
Argument may be line number, function name, or "*" and an address.
* c++ to Javascript: use raw_to_Javascript() to properly escape
If line number is specified, break at start of code for that line.
* c++ to sqlite: use SqliteLocalModel::safestr(), which uses double_doublequotes(str)
If function is specified, break at start of code for that function.
|}
If an address is specified, break at that exact address.
[[Postgres]]
With no arg, uses current execution address of selected stack frame.
 
This is useful for breaking on return to a stack frame.
[[Simple-Web-Server]]
 
Multiple breakpoints at one place are permitted, and useful if conditional.   
[[Robot Operating System]]
 
Do "help breakpoints" for info on other commands dealing with breakpoints.
[[C++ https libraries]]
|-
 
| ddd gives you a front end.  I need to use it more, compare to other options
[[Configure Qt development on Windows + Mac + linux]]
 
[[Build the TagLib library with Visual Studio 2013]]
 
|}
|}
<!--


===========================================================================================================================================================================================================================================================================================
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! C - Create a portable command line C project in Visual Studio
! C/C++ building/linking
|-
|-
|
|
    Visual Studio: File -> New -> project
{| class="mw-collapsible mw-collapsed wikitable"
    Visual C++ -> Win32 -> Win32 Console Application
! gcc makefile pain
    name: oms_with_emap
|-
    next -> click OFF precompiled header checkbox (even tho it didn't seem to respect it)
|
    you'll get a _tmain(..., TCHAR*...)
* I went through a LOT of pain to determine that gcc requires libraries to be listed AFTER the source and output parameters
    change it to main(..., char*...)
* set LD_LIBRARY_PATH to point to your libs if they are not in system location
    change the project to explicitly say "Not using precompiled header"
* Make sure Makefile uses TABS NOT SPACES. it's a FACT OF LIFE. hate it if you want. plenty more things to hate about linux/C as well.
    remove the f'in stdafx.h
    recompile!  should be clean
    vs will recognize C files and compile accordingly
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! gcc install multiple versions in ubuntu (4 and 5 in wily, eg)
! gcc install multiple versions in ubuntu (4 and 5 in wily, eg)
Line 578: Line 649:
  CC=gcc-4.9
  CC=gcc-4.9
|}
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================
-->
{| class="wikitable"
! [[git]]
|}
|}
<!--  
<!--  
Line 587: Line 668:
-->
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! JavaScript
! Debugging
|-
|-
|
|
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! install Node.js
! Chrome capture large JSON variable
|-
| This is just pointlessly bizarre:
* hit a breakpoint in the chrome debugger
* right-click a variable and say "copy to global variable" (console will show name, typically "temp1")
* push the variable to the clipboard by typing this in the console:
copy(temp1)
|}
{| class="mw-collapsible mw-collapsed wikitable"
! Visual Studio Code capture large string variable
|-
| While debugging, you can use the Debug Console to print memory, including the content of strings that are clipped by default in the variables and watch windows.
View > Open View > Debug Console
From there, send gdb a command to print memory – 300 characters of a string in this example:
-exec x/300sb Query.c_str()
|}
{| class="mw-collapsible mw-collapsed wikitable"
! Qt Creator conditional breakpoint
|-
|-
| '''Windows'''
| To break on line number with string condition:
* Use the [https://nodejs.org/en/download/ latest 64-bit installer]
* Set the breakpoint on the line as usual: F9 or click in left border
* From a cmd prompt: npm install -g express
* In the breakpoints list, rclick Edit
'''Linux'''
* Add a condition; to break on a string value, THIS WORKS:
* install Node.js using the "Node.js Version Manager" nvm [https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-an-ubuntu-14-04-server details]
     
** find the [https://github.com/creationix/nvm/releases latest nvm version]
          ((int)strcmp(strSymbol.c_str(), "GMO")) == 0
** curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.29.0/install.sh | bash
     
** source ~/.profile
* BUT THERE IS A PRICE TO PAY. qt/gcc won't continue properly, unless you sset ANOTHER common breakpoint, hit it, and resume from that.  It appears gcc continues (we hit the breakpoint after all), but qt does no logging or debugging, bah
** nvm ls-remote
** nvm install 4.2.1
** npm install -g express # to set the package manager to use a globally shared location
*** also: nvm use 4.2.1; node -v; nvm ls; nvm alias default 0.11.13; nvm use default
*** also: You can create an .nvmrc file containing version number in the project root
directory and it will default to that version
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! auto AWS
! Qt Creator and linux debug libraries
|-
|-
|  
|
* npm install -g aws-sdk
* I was able to build lws in debug, install in /usr/local, then I could step right into its code
* Add credentials here: C:\Users\Administrator\.aws
m@case:~/development/causam/git/np/nop-bigress-client-c$ ./build_lws_debug
* see existing scripts, anything is possible
  Install the project...
  -- Install configuration: "DEBUG"
  -- Installing: /usr/local/lib/pkgconfig/libwebsockets.pc
  -- Installing: /usr/local/lib/libwebsockets.a
  -- Up-to-date: /usr/local/include/libwebsockets.h
  (etc)
* Signals: At some point I needed to force gdb to pass SIGINT, SIGPIPE... signals to the app instead of gdb, after ssl kept causing Qt to pop a message and break in the debugger.  To do so:
        Qt Creator->Options->Debugger->
            [ ] Show a message box when receiving a signal
            Debugger Helper Customization: handle SIGPIPE pass nostop noprint
* You should be able to install debug builds of libraries into /usr/local (as mentioned above).  Older notes if you can't get that going:  To debug into an included library that is not installed system-wide:
use Tools->Options->General->Source Paths Map
  source: /home/m/development/causam/git/nop-bigress-client-c/cmake-debug
    that's the COMPLETE PATH TO DEBUG EXE
  target: /home/m/development/causam/git/libwebsockets-master/lib
    at first i thought this would work: /usr/local/lib
    but i did a static build so i needed teh actual exe!  COOL
|}
{| class="mw-collapsible mw-collapsed wikitable"
! QT Creator valgrind EASY
|-
| Valgrind is easy to use.  Just find debugger pane top-left dropdown, switch mode from Debugger to Memcheck, and restart debugger.
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! install bootstrap
! c/c++ gdb debugging
|-
|-  
|  
|  
* npm install -g grunt-cli
(gdb) help break
* mkdir mysite && cd mysite
Set breakpoint at specified line or function.
* npm install bootstrap
Argument may be line number, function name, or "*" and an address.
* cd node_modules/bootstrap
If line number is specified, break at start of code for that line.
* npm install # to actually pull down dependencies
If function is specified, break at start of code for that function.
* grunt dist # builds and minifies so you're good to go!
If an address is specified, break at that exact address.
With no arg, uses current execution address of selected stack frame.
This is useful for breaking on return to a stack frame.
Multiple breakpoints at one place are permitted, and useful if conditional.   
Do "help breakpoints" for info on other commands dealing with breakpoints.
|-
| ddd gives you a front end.  I need to use it more, compare to other options
|}
|}
|}
|}
Line 636: Line 757:
-->
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! git
! C
|-
|-
|
|
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! git convert to a bare repo
! Library handling in linux
|-
|-
| Start with a normal git repo via [git init]; add your files, get it all set up. Then do this:
| gcc library basics (good to know for autotools and CMake too)
  cd repo
* The gcc -l command line option specifies a specific library NAME.  It will add the rest to give you (eg) lib[NAME].a
mv .git .. && rm -fr *
* Specify the library search path with the -L option. Things that you as a user install seem to default to /usr/local/lib
mv ../.git .
* This will link myprogram with the static library libfoo.a in the folder /home/me/foo/lib.
mv .git/* .
        gcc -o myprogram -lfoo -L/home/me/foo/lib myprogram.c
rmdir .git
Tools to check library dependencies:
git config --bool core.bare true
* ldd [executable]
cd ..; mv repo repo.git # renaming just for clarity
* objdump -p /path/to/program | grep NEEDED
* objdump -x [object.so]
* readelf -d [exe]
* sudo pldd <PID>
* sudo pmap <PID>
 
|}
|}
[[Cross Compiling]]
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! git use kdiff3 as difftool and mergetool
! C - Create a portable command line C project in Visual Studio
|-
|-
| It's been made easy on linux...
|
* LINUX - put this in ~/.gitconfig
    Visual Studio: File -> New -> project
  [diff]
    Visual C++ -> Win32 -> Win32 Console Application
    tool = kdiff3
    name: oms_with_emap
    next -> click OFF precompiled header checkbox (even tho it didn't seem to respect it)
[merge]
    you'll get a _tmain(..., TCHAR*...)
    tool = kdiff3
    change it to main(..., char*...)
* WINDOZE
    change the project to explicitly say "Not using precompiled header"
[difftool "kdiff3"]
    remove the f'in stdafx.h
    path = C:/Progra~1/KDiff3/kdiff3.exe
    recompile! should be clean
    trustExitCode = false
    vs will recognize C files and compile accordingly
[difftool]
|}
    prompt = false
 
[diff]
|}
    tool = kdiff3
<!--
[mergetool "kdiff3"]
 
    path = C:/Progra~1/KDiff3/kdiff3.exe
 
    trustExitCode = false
===========================================================================================================================================================================================================================================================================================
[mergetool]
 
    keepBackup = false
 
[merge]
-->
    tool = kdiff3
{| class="wikitable"
* LINUX Before - What a ridiculous pita... copy this into .git/config...
! [[Javascript]]
[difftool "kdiff3"]
|}<!--
    path = /usr/bin/kdiff3
 
    trustExitCode = false
 
[difftool]
===========================================================================================================================================================================================================================================================================================
    prompt = false
 
[diff]
 
    tool = kdiff3
-->
[mergetool "kdiff3"]
{| class="wikitable"
    path = /usr/bin/kdiff3
! [[Node.js]]
    trustExitCode = false
|}<!--
[mergetool]
 
    keepBackup = false
 
[merge]
===========================================================================================================================================================================================================================================================================================
    tool = kdiff3
 
 
-->
{| class="wikitable"
! [[React]]
|}<!--
 
 
===========================================================================================================================================================================================================================================================================================
 
 
-->
{| class="wikitable"
! [[Vite]]
|}
<!--
 
 
===========================================================================================================================================================================================================================================================================================
 
 
-->
{| class="wikitable"
! [[JSON]]
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================


{| class="mw-collapsible mw-collapsed wikitable"
-->
! git create merge-to command
{| class="wikitable"
|-
! [[HTML]]
| Add this handy alias command to all git repos' .config file...
[alias]
    merge-to = "!gitmergeto() { export tmp_branch=`git branch | grep '* ' | tr -d '* '` && git checkout $1 && git merge $tmp_branch && git checkout $tmp_branch; unset tmp_branch; }; gitmergeto"
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================


{| class="mw-collapsible mw-collapsed wikitable"
 
! git shared repo
-->
|-
{| class="wikitable"
| There are two possibilities (I prefer the latter). 
! [[CSS]]
# Bare repositories are designed to be shared.  Maintenance on the central server is easier because you don't have local files to manage permissions or constant flux.  Plus, you can always have a second repo on the central server where you check out a specific branch (e.g. to serve up with apache).
# If you want to have a location on the central repo to see the files, use the master-daily pattern, and config the repo as shared:
git config core.sharedRepository true
To set it on a new git repo during initial setup, make sure devs are in the same group, and use:
git init --shared=group
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! git create new branch on server, pull to client
! SQL
|-
|-
|  
|
# ON CENTRAL SERVER
git checkout master # as needed; we are assuming that master is clean enough as a starting point
git checkout -b mynewbranchy
# HOWEVER, use this instead if you need a new "clean" repo and even master is dirty...
# You need the rm because git "leaves your working folder intact".
git checkout --orphan mynewbranchy
git rm -rf .
# ON CLIENT
git pull
git checkout -b mynewbranchy origin/mynewbranchy
# if files are in the way from the previously checked-out branch, you can force it...
git checkout -f -b mynewbranchy origin/mynewbranchy
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! git windows configure notepad++ editor
! Count records within a range
|-
|-
|  
| This groups records into ranges, sorts by them, and gives a count, sweet:
git config --global core.editor "'C:/Program Files (x86)/Notepad++/notepad++.exe' -multiInst -notabbar -nosession -noPlugin"
    select count(*), id/1000000 as groupid from AccountHistory group by groupid;
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
[[postgres]] - [[sqlite]] - [[mysql]] - [[SQL Server]] - [[Robo 3T]] - [[DBeaver]] - [[pgadmin4]]
! git pull when untracked files are in the way
|-
| This will pull, forcing untracked files to be overwritten by newly tracked ones in the repo:
git fetch --all
git reset --hard origin/mymatchingbranch
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
<!--  
! git create new branch when untracked files are in the way
 
|-
 
|
===========================================================================================================================================================================================================================================================================================
  git checkout -b bj143 origin/bj143
 
      git : error: The following untracked working tree files would be overwritten by checkout:
 
      (etc)
-->
 
{| class="wikitable"
  TOTAL PURGE FIX (too much):
! [[Meteor]]
      git clean  --fn ""
        -d dirs too
        -f force, required
        -x include ignored files (don't use this)
        -n dry run
 
  BEST FIX (just overwrite what is in the way):
      git checkout -f -b bj143 origin/bj143
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================


{| class="mw-collapsible mw-collapsed wikitable"
 
! git fix push behavior - ONLY PUSH CURRENT doh
-->
|-
{| class="wikitable"
|
! [[Android]]
git config --global push.default current
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================


{| class="mw-collapsible mw-collapsed wikitable"
-->
! git recreate repo
{| class="wikitable"
|-
! [[Arduino]]
|
git clone ssh://m@thedigitalmachine.com/home/m/development/thedigitalage/ampache-with-hangthedj-module
cd ampache-with-hangthedj-module
git checkout -b daily_grind origin/daily_grind
If you already have the daily_grind branches and just need to connect them:
git branch -u origin/daily_grind daily_grind
|}
|}
<!--


{| class="mw-collapsible mw-collapsed wikitable"
 
! git connect to origin after the fact
===========================================================================================================================================================================================================================================================================================
|-
 
|
-->
git remote add origin ssh:// m@bitpost.com/home/m/development/logs
{| class="wikitable"
git fetch
! [[Raspberry Pi]]
    From ssh:// bitpost/home/m/development/logs
      * [new branch]     daily_grind -> origin/daily_grind
      * [new branch]     master    -> origin/master
git branch -u origin/daily_grind daily_grind
git checkout master
git branch -u origin/master master
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
<!--  
! git multiple upstreams
 
|-
 
| Use this to cause AUTOMATIC push/pull to a second origin:
===========================================================================================================================================================================================================================================================================================
git remote set-url origin --push --add user1@repo1
 
git remote set-url origin --push --add user2@repo2
-->
git remote -v show
{| class="wikitable"
Leave out --push if you want to pull as well... but I'd be careful, it's better if code is changed in one client with this config, and then pushed to the multiple origins from there.  Otherwise, things are GOING TO GET SYNCY-STINKY.
! [[iOS]]
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================


-->
{| class="wikitable"
! [[.NET Core]]
|}
|}
<!--  
<!--  
Line 812: Line 948:


-->
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="wikitable"
! Eclipse
! [[Java]]
|-
|
{| class="mw-collapsible mw-collapsed wikitable"
! Eclipse Mars installation
|-
| We always want these: CDT, PDT, WTP (Web Tools Platform, includes JSTP for Javascript support)
# Get and run the [http://www.eclipse.org/downloads/ Eclipse Installer] and install one of the big ones
## PDT is good but there are instructions below to bolt on any of the three - so anything goes with your starting choice
# Install to development/eclipse/mars
# Run and select this as the default workspace, do not prompt again: /home/m/development/eclipse-workspace
# Install PDT
## Help-> Install New Software -> Add... -> Find latest PDT site [ here] -> add https://wiki.eclipse.org/PDT/Installation
## e.g. for Mars1, use http://download.eclipse.org/tools/pdt/updates/3.6/
# Install WTP
## Help-> Install New Software -> Add... -> add -> "WTP Mars", http://download.eclipse.org/webtools/repository/mars/ -> select WTP 3.7.1
# Install CDT
## Help-> Install New Software -> Add... -> Find the latest CDT site [https://eclipse.org/cdt/downloads.php here], and add it
## e.g. for Mars1, use http://download.eclipse.org/tools/cdt/releases/8.8
## Install CDT Main Features; CDT Optional Features: autotools memoryview misc multicore qt unittest vc++ visualizer
# Close eclipse and update the settings folder to point to the common shared location (make sure the development/config repo is available):
cd ~/development/eclipse-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings
ln -fs /home/m/development/config/common/home/m/development/eclipse-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/* .
# Import all the existing projects that you need (A better Trader, Hang The DJ, etc.).  You can import existing projects from ~/development, easy peasey!
# Install the [https://marketplace.eclipse.org/content/terminal-plug terminal plugin] so you can run CI right in eclipse.  Just drag the "install" button there to the eclipse toolbar.
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
<!--  
! Eclipse settings for all portable C++ boost projects
 
|-
 
|
===========================================================================================================================================================================================================================================================================================
* Set up .bashrc with my standard ENV vars for boost, c++ (and use notes there to build latest boost, if needed)
 
* Set up build-Debug and build-Release folders via bootstrap force [debug|release], then configure build configurations for them both
 
* Configure the project according to these [[Eclipse project configuration screenshots]]
-->
{| class="wikitable"
! [[Kotlin]]
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================


{| class="mw-collapsible mw-collapsed wikitable"
-->
! Eclipse annoyances
{| class="wikitable"
|-
! [[Maven]]
|
* To get problems to reset on build, I had to turn on (for all configs, then exit/restart): Project->Properties->C++ Build->Settings->Error Parsers-> [x] GNU gmake Error Parser 7
* Click only ERRORS on Annotations dropdown arrow to bypass noise - I still can't get Ctrl-[,|.] to navigate errors, insanity
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
<!--  
! eclipse java project layout format
 
|-
 
| Eclipse uses a workspace which holds projects.  Java apps written with Eclipse are organized as follows:
===========================================================================================================================================================================================================================================================================================
* Eclipse workspace (can also be the top version-control folder)
 
** project folder (typically one "app" that you can "run")
 
*** package(s) (named something like "com.developer.project.application")
-->
**** classes (each class is contained in one file)
{| class="wikitable"
! [[Scala]]
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
<!--  
! eclipse new project from existing code
 
|-
 
| You can set up a new project without specifying anything about it, just to browse the code.
===========================================================================================================================================================================================================================================================================================
File->New Project->Empty
 
Name: bbby2e05
 
Location: c:\
-->
[ ] Create subdir
{| class="wikitable"
---
! [[Python]]
Show all files
select everything in include, rc->include in project
repeat for src
dont worry about mak or install folders for now, just add files later as needed
save all
---
then set up a repo for it!
cd c:\bbb2e05
git init (plus add cpp c hpp h, commit, set up daily, sync on bitpost)
|-
| It is also possible to set up a C++ makefile or PHP project from existing code.
(rclick projects area)->New->Project...->C++->Makefile Project with existing code
(name it and make sure Show all files is selected)
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================
-->
{| class="wikitable"
! [[Go]]
|}
|}
<!--  
<!--  
Line 896: Line 1,009:
-->
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! misc
! PHP
|-
|-
|
|
Line 911: Line 1,024:
  /etc/init.d/apache restart
  /etc/init.d/apache restart
|}
|}
|}
<!--
===========================================================================================================================================================================================================================================================================================
-->
{| class="wikitable"
! [[Bash basics]] but please prefer node or python :-)
|}
<!--
===========================================================================================================================================================================================================================================================================================
-->
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! emacs configuration
! misc
|-
|-
| This common config should work well in both terminal and UI:
|
/home/m/development/config/common/home/m/.emacs
NOTE that you need some other things to be configured properly:
* the terminal must use a light light blue background since it will be carried over into [emacs -nw]
* the terminal must have 256-color support; set this in .bashrc:
export TERM=xterm-256color
* Make sure you check out undo support via [ctrl-x u]
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! Jenkins
! SVN repo move across servers
|-
|-
| Installation:
|  
* install on a server (TODO)
* use a tool like VisualSVN to stop SVN server
* For VS projects:
* copy the repo's entire svn server directory, eg: c:\svn\Software
** add MSBuild plugin
* copy it into the new server under a unique name, eg: c:\svn\NewSoftware
** find the right path to msbuild.exe (eg C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe)
* use a tool like VisualSVN to restart SVN server
** Manage Jenkins, Configure System, add the msbuild there; USE FULL PATH, INCLUDE EXE, NO LEADING SPACES
You should now have the new code, accessible as usual from svn.
* Add the Build Keeper plugin to clean up previous bad builds on rebuild
NOTE from Tom: If you move code around, make sure you tag it, then copy the tag, to preserve history (as opposed to directly moving the folder).  Weird but true.
* Add nodes labeled according to what will be built on each
* Select "New Item" to add folder(s) to create a project layout
* Within folders, select "New Item" to create "Freestyle Projects"
* Configure the project:  
** git, build+test+install script, post-build cleanup
** "Restrict where this project can run" with node labels
** poll source control ~ every 30 min: "H/30 * * * *"
* TODO retrieve build+test+install status, report to Jenkins
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! SQL Server 2008+ proper upsert using MERGE
! SQL Server 2008+ proper upsert using MERGE
Line 988: Line 1,103:
               );
               );
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="wikitable"
! Windows command prompt FULL SCREEN
! [[Web Services]]
|-
|Type cmd in start search box and right-click on the cmd shortcut which appears in the results. Select Run CMD as administrator.
Next, in the command prompt, type wmic and hit Enter.
Now try to maximize it!
Close it and again open it. It will open as a maximized window!
You may have to ensure that the Quick Edit Mode in the Options tab is checked.
|}
|}


{| class="wikitable"
! [[Firefox Addon development]]
|}
{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! bash chmod dirs
! c++ Create a portable autotools C++ project in linux
|-
|-
|
| (OLD, USE CMAKE NOW)
find /path/to/base/dir -type d -exec chmod g+x {} \;
|}


{| class="wikitable"
For anything serious, it's best to clone an existing project's skeleton.
! [[Web Services]]
* main.cpp
* MVC code
* nix/copy_from folder
* make sure .bashrc is configured for boost
* nix$  bootstrap_autotools_project.sh force release debug
* set up eclipse according to screenshots in [[Eclipse]]
|}
|}
{| class="mw-collapsible mw-collapsed wikitable"
! c++ Create a portable C++ project in Visual Studio
|-
| (OLD, USE CMAKE NOW)


{| class="wikitable"
If you don't have existing code, it's probably best to create a project_name.cpp file with your main() function.
! [[Firefox Addon development]]
int main( int argc, char * argv[] )
{  
    return 0;
}
Then in Visual Studio...
File->New->Project from existing code
C++
(then use mostly defaults on this page, once you provide file location and project name)
Project file location:  <base>\project_name
Project name: project_name
[x] Add files from these folders
    Add subs 
    [x]       <base>\project_name
NEXT
Use Visual Studio
  Console application project
  No ATL, MFC, CLR
NEXT
NEXT
FINISH
Then add Reusable and boost include and lib paths:
Example if you built boost according to notes below:
Project->Properties->All Configurations->Configuration Properties->VC++ Directories->Include Directories->
    $(VC_IncludePath);$(WindowsSDK_IncludePath);..\..\..\Reusable\c++
INCLUDE: C:\Software Development\boost_1_53_0
LIB: C:\Software Development\boost_1_53_0\stage\lib
|}
|}
|}
|}

Latest revision as of 23:15, 25 November 2023

Design, programming and version control.

Patterns
Major Objects
Quick-http
Continuous Integration
Architecture of major open source apps
Security
Model View Controller
nosql enhances, not replaces, SQL
Not all data should be denormalized, and not all data should be normalized. The optimal mix considers the extent of the data.
  • Precise schemas are good when not overdone
  • When a container has an array with a large number of elements, it should be normalized
  • Sparse data and heterogeneous data are the best candidates for denormalization

Postgres with JSON allows an elegant combination of nosql and SQL.

Web design mantra
  1. DESIGN INPUT TO THE INPUT DEVICE
    1. respect the input device
    2. detect it and use a reasonable default guess
    3. allow manual override
    4. [mouse/pen]...[finger]
    5. sm-md-lg ... sm-md-lg
  2. DESIGN VISUALS TO THE SCREEN
    1. high-res = SHOW LOTS OF DETAIL
    2. responsive, zoomable
git central shared repository
Use bare repos for any central shared repositories. Use the [####.git] suffix on bare repo names.

Bare repositories are designed to be shared. Maintenance on the central server is easier because you don't have local files to manage permissions or constant flux. Plus, you can always have a second repo on the central server where you check out a specific branch (e.g. to serve up with apache). If you want a dynamically updated central repo, clone the ###.git repo to ###, and add a post-receive hook (see bitpost quick-http.git for a good example).

To configure the repo as shared:

git config core.sharedRepository true

To set it on a new git repo during initial setup, make sure devs are in the same group, and use:

git init --shared=group
Occam's razor
Among competing hypotheses, the one with the fewest assumptions should be selected.
C++
Deleting multiple elements in a collection
  • For sequences (vector, list...), best practice is to use the erase-remove STL pattern
  • For associative containers (set, map...): just recurse and remove iterators, they won't muck up the sequence. example

See MemoryModel::saveDirtyObjectsAsNeeded() for example code.

c++ Create a portable C/C++ project
Use QtCreator + CMake, both available everywhere!
   set up:
       create a Qt project as: New Project-> Non-Qt/Plain C Project (CMake Build)
           pick the PARENT and use the name of the existing folder
           set up a build folder in the existing folder, called (base)/qtcreator-release-build
           it creates CMakeLists.txt (THIS IS BASICALLY THE PROJECT FILE!)
           also creates main.c or something
           build it!  make sure it works
           NOW we can edit CMakeLists.txt - change . to ./src to get it to scan the src folder for code!
           and easily add "known" libs!  this was all i needed:
               TARGET_LINK_LIBRARIES(${PROJECT_NAME} pthread websockets)
           make sure it builds
           now we want a DEBUG build as well!
           Projects->Build&Run->pick Build in weird Build/Run pillbox->Edit build configs:
               Rename: release
               CMake build dir: Make sure build dir is set to release ie qtcreator-debug-build
               Add: Clone: release, name: debug
                   for debug, add a custom build step BEFORE make step:
                       command: /usr/bin/cmake
                       args: -DCMAKE_BUILD_TYPE=Debug .
                       working dir: %{buildDir}
                       (add it, and move it up above make)
                       (build em both)
           Projects->Build&Run->pick Run in weird Build/Run pillbox
               Run Configuration: rename to "{...} release"
               clone it into "{...} debug"
               change working dir to match Build path
               change any needed params
   Now you should have build+run debug+release configurations, selectable in the weird project icon in bottom of left-side toolbar ("mode selector")
c++ in-memory storage of "major" objects
   OBSERVATION ONE

   Consider An Important Qt Design: QObjects cannot normally be copied
       their copy constructors and assignment operators are private
       why?  A Qt Object...
           might have a unique QObject::objectName(). If we copy a Qt Object, what name should we give the copy?
           has a location in an object hierarchy. If we copy a Qt Object, where should the copy be located?
           can be connected to other Qt Objects to emit signals to them or to receive signals emitted by them. If we copy a Qt Object, how should we transfer these connections to the copy?
           can have new properties added to it at runtime that are not declared in the C++ class. If we copy a Qt Object, should the copy include the properties that were added to the original?
   in other words, a QObject is a pretty serious object that has the ability to be tied to other objects and resources in ways that make copying dangerous
   isn't this true of all serious objects?  pretty much
   OBSERVATION TWO

   if you have a vector of objects, you often want to track them individually outside the vector
   if you use a vector of pointers, you can move the object around much more cheaply, and not worry about costly large vector reallocations
   a vector of objects (not pointers) only makes sense if the number of objects is initially known and does not change over time
   OBSERVATION THREE

   STL vectors can store your pointers, iterate thru them, etc.
   for a vector of any substantial size, you want to keep objects sorted so you can find them quickly
   that's what my sorted_vector class is for; it simply bolts vector together with sort calls and a b_sorted status
   following STL practices, to get sorting, you have to provide operator< for whatever is in your vector
   BUT... you are not allowed to do operator<(const MyObjectPtr* right) because it would require a reference to a pointer which is not allowed
   BUT... you can provide a FUNCTOR to do the job, then provide it when sorting/searching
   a functor is basically a structure with a bool operator()(const MyObjectPtr* left, const MyObjectPtr* right)
   OBSERVATION FOUR

   unordered_set works even better when combining frequent CRUD with frequent lookups
   SUMMARY
   Dealing with tons of objects is par for the course in any significant app.
   Finding a needle in the haystack of those objects is also standard fare.
   Having multiple indices into those objects is also essential.
   Using unordered_set with object pointers and is very powerful.
c++ stl reverse iterator skeleton
From SGI...
reverse_iterator rfirst(V.end());
reverse_iterator rlast(V.begin());

while (rfirst != rlast) 
{
    cout << *rfirst << endl;
    ...
    rfirst++;
}
c++ stl reading a binary file into a string
   std::ifstream in("my.zip",std::ios::binary);
   if (!in)
   {
      std::cout << "problem with file open" << std::endl;
      return 0;
   }
   in.seekg(0,std::ios::end);
   unsigned long length = in.tellg();
   in.seekg(0,std::ios::beg);
 
   string str(length,0);
   std::copy( 
       std::istreambuf_iterator< char >(in) ,
       std::istreambuf_iterator< char >() ,
       str.begin() 
   );

For more, see c++ stl reading a binary file

C/C++ best-in-class tool selection
I need to have easy setup of debug-level tool support for portable C++11 code. And I need to decide and stick to it to be efficient.
  • Compiler selection
    • linux and mac: gcc
    • windows: Visual Studio
  • IDE selection
    • linux and mac: Qt Creator
    • windows: Qt Creator (OR Visual Studio OR eclipse?)
  • Debugger selection
    • linux and mac: Qt Creator
    • windows: Qt Creator (OR Visual Studio OR eclipse?)
c++11
c++11 include base constructor (etc) in derived class
The awesome C++11 way to steal the constructor from the base class, oh man I've been waiting for this one...
 class HttpsServer : public Server<HTTPS>
 {
   // DOES THIS FOR YOU!
   /*
   HttpsServer(unsigned short port, size_t num_threads, const std::string& cert_file, const std::string& private_key_file)
   :
       // Call base class
       Server<HTTPS>::Server(port, num_threads, cert_file, private_key_file)
   {}
   */
   using Server<HTTPS>::Server;
   ....
c++11 containers
sorted_vector use when doing lots of unsorted insertions and maintaining constant sort would be expensive; vector is good for a big pile of things that only occasionally needs a sorted lookup
map sorted binary search tree; always sorted by key; you can walk through in sorted order (choose unordered if not needed!)
multimap same as map but allows dupe keys (not as common)
unordered_map hashmap; always sorted by key; additional bucket required for hash collisions; no defined order when walking through
unordered_multimap same as map but allows dupe keys; dupes are obviously in the same bucket, and you can walk just the dupes if needed
set
multiset
unordered_set
unordered_multiset
sets are just like maps, except the key is embedded in the object, nice for encapsulation.

Items must be const (!) since they are the key - sounds bad, but this is mitigated by the mutable keyword.
You can use mutable on the variables that are not part of the key to remove the const.
This changes the constness of the object from binary (completely const) to logical (constness is defined by the developer).
So... set is a good way to achieve both encapsulation and logical const - make const work for you, not against!  :-)

set (etc.) of pointers sets of pointers are the pinnacle of object stores

The entire object can be dereferenced and accessed then without const issues.
A pointer functor can be provided that does a sort by dereferencing the pointer to the object.
Two requirements: you must make sure yourself that you do not change the key values - you can mark them const, provided in constructor;
you must create sort/equal/hash functors that dereference the pointers to use object contents
(the default will be by pointer address).
The arguably biggest advantage, as a result, is that you can create multiple sets
to reference the same group of objects with different sort funtors to create multiple indices.
You just have to manage the keys carefully, so that they don't change (which would invalidate the sorting).
The primary container can manage object allocation; using a heap-based unique_ptr allocation

   map vs key redux
               
       use a key in the set, derive a class from it with the contents
           + small key
           + encapsulation
           - requires mutable to solve the const problem
       use a key in the set, key includes a mutable object
           + encapsulation
           - weird bc everything uses a const object but we have const functions like save() that change the mutable subobject
       use a map
           + small key
           - no encapsulation, have to deal with a pair instead of an object
               can we just put a ref to key in the value?  sure why not - err, bc we don't have access to it
           + solves const problem bc value is totally mutable by design
           + we can have multiple keys - and the value can have multiple refs to them
           + simpler equal and hash functions
       map:
           create an object with internal key(s)
           create map index(es) with duplicate key values outside the object - dupe data is the downside
       use set(s) with one static key for find(): 
           create an object with internal key(s)
           create set index(es) with specific hash/equals functor(s)
           when finding, use one static key object (even across indexes!) so there isn't a big construction each time; just set the necessary key values
               that proves difficult when dealing with member vars that are references
               but to solve it, just set up a structure of dummy static key objects that use each other; then provide a function to setKey(Object& keyref) { keyref_ = keyref; }
               nope, can't reassign refs
               the solution: use pointers not references
               yes that's right
               just do it
               apparently there was a reason i was anti-reference for all those years
               two reasons to use pointers:
                   dynamically allocated
                   reassignment required
               there ya go.  simple.  get it done. 
           when accessing find results from the set, use a const_cast on the object!
           WARNING: a separate base class with the key sounds good... but fails when you have more than one index on the object.  just use a static key object for them all!
c++11 example for large groups of objects with frequent crud AND search
Best solution is an unordered set of pointers:
typedef boost::unordered_set<MajorObject*> MajorObjects;
c++11 example for large groups of objects with infrequent crud and frequent search
Best solution is a vector of pointers sorted on demand (sorted_vector):
TODO
c++11 example to associate two complex objects (one the map key, one the map value)
Use unordered_map with a custom object as key. You must add hash and equals functions. Boost makes it easy:
static bool operator==(MyKeyObject const& m1, MyKeyObject const& m2)
{
    return 
            m1.id_0 == m2.id_0
        &&  m1.id_1 == m2.id_1;
}
static std::size_t hash_value(MyKeyObject const& mko)
{
    std::size_t seed = 0;
    boost::hash_combine(seed, mko.id_0);
    boost::hash_combine(seed, mko.id_1);
    return seed;
}
typedef boost::unordered_map<MyKeyObject, MyValueObject*> MyMap;

Note that you can extend this to use a pointer to a key object, whoop.

c++11 example for multiple unordered_set indexes into one group of objects
Objects will be dynamically created. One set should include them all and be responsible for memory allocation cleanup:
TODO
c++11 example for set with specific sorting
Use set with a specific sort functor. You can create as many of these indexes as you want!
struct customers_set_sort_functor
{
    bool operator()(const MyObject* l, const MyObject* r) const
    {
        // the id is the key
        return l->id_ < r->id_;
    }
};
typedef set<MyObject*,myobject_sort_by_id_functor> MyObjectsById;
c++11 loop through vector to erase some items
Note that other containers' iterators may not be invalidated so you can just erase() as needed...

For vectors, you have to play with iterators to get it right - watch for proper ++ pre/postfix!

for (it = numbers.begin(); it != numbers.end(); )  // NOTE we increment below, only if we don't erase
{
    if (*it.no_good()) 
    {
        numbers.erase(it++);  // NOTE that we ERASE THEN INCREMENT here.
    }
    else 
    {
        ++it;
    }
}

I thought I had always looped backwards to do this, I *think* that's ok too, but I don't see it used in my code, I think I'll avoid.  :-)

c++11 range based for loop, jacked with boost index if needed
No iterator usage at all. Nice at times, not enough at others. Make SURE to always use a reference or you will be working on a COPY. Make it const if you aren't changing the object.
for (auto& mc : my_container)
    mc.setString("default");
for (const auto& cmc : my_container)
    cout << cmc.asString();

boost index can give you the index if you need it, sweet:

#include <boost/range/adaptor/indexed.hpp>
...
for (const auto &element: boost::adaptors::index(mah_container))
    cout << element.value() << element.index();
c++11 for loop using lambda
This C++11 for loop is clean and elegant and a perfect way to check if your compiler is ready for c++11:
vector<int> v;
for_each( v.begin(), v.end(), [] (int val)
{
   cout << val;
} );

This is using a lambda function, we should switch from iterators and functors to those - but not quite yet, since we're writing cross-platform code. Do not touch this until we can be sure that all platforms provide compatible C++11 handling.

c++11 integer types
I really like the "fast" C++11 types, that give best performance for a guaranteed minimum bit width.

Use them when you know a variable will not exceed the maximum value of that bit width, but does not have to be a precise bit width in memory or elsewhere.

Pick specific-width fields whenever data is shared with other processes and components and you want a guarantee of its bit width.

And when using pointer size and array indices you should use types defined for those specific situations.

FAST types:

   int_fast8_t
   int_fast16_t                fastest signed integer type with width of
   int_fast32_t                at least 8, 16, 32 and 64 bits respectively
   int_fast64_t
   uint_fast8_t
   uint_fast16_t               fastest unsigned integer type with width of
   uint_fast32_t               at least 8, 16, 32 and 64 bits respectively
   uint_fast64_t

SMALL types:

   int_least8_t
   int_least16_t               smallest signed integer type with width of
   int_least32_t               at least 8, 16, 32 and 64 bits respectively
   int_least64_t
   uint_least8_t
   uint_least16_t		smallest unsigned integer type with width of
   uint_least32_t		at least 8, 16, 32 and 64 bits respectively
   uint_least64_t

EXACT types:

   int8_t                      signed integer type with width of
   int16_t                     exactly 8, 16, 32 and 64 bits respectively
   int32_t                     with no padding bits and using 2's complement for negative values
   int64_t                     (provided only if the implementation directly supports the type)
   uint8_t                     unsigned integer type with width of
   uint16_t                    exactly 8, 16, 32 and 64 bits respectively
   uint32_t                    (provided only if the implementation directly supports the type)
   uint64_t

SPECIFIC-USE types:

   intptr_t                    integer type capable of holding a pointer
   uintptr_t                   unsigned integer type capable of holding a pointer 
   size_t                      unsigned integer type capable of holding an array index (same size as uintptr_t)
C++11 scoped enumeration
C++11 has scoped enumeration, which lets you specify the SPECIFIC VARIABLE TYPE for the enum. Perfect, let's use uint_fast32_t.
enum class STRING_PREF_INDEX int_fast32_t: { ... };

Unfortunately, gcc gives me a scary warning, and stuff fails. For some reason, it does not know about the provided type, although it is definitely defined. Revisit this later if you have time.

warning: elaborated-type-specifier for a scoped enum must not use the ‘class’ keyword

Old skool is still cool:

typedef enum
{
    // assert( SP_COUNT == 2 );
    SP_FIRST = 0                ,
    SP_SOME_PREF = SP_FIRST     ,
    SP_ANOTHA                   ,

    SP_COUNT
} STRING_PREF_INDEX;
boost
boost release and debug build for linux
   # download latest boost, eg: boost_1_59_0 
   m@wallee:~/development$ 7z x boost_1_59_0.7z
   rm boost && ln -s boost_1_59 boost
   cd boost
   build_boost_release_and_debug.sh
   # IMPORTANT: then patch .bashrc as instructed
   #     cmake-###/clean.sh and cmake-###/build.sh are customized to match
   #     (and older eclipse and server/nix/bootstrap.sh) 

To upgrade a CMake project:

   cd cmake-###/
   ./clean.sh
   ./build.sh

To upgrade an older autotools project:

   cd nix
   make distclean  # removes nasty .deps folders that link to old boost if you let them
   make clean      # removes .o files etc
   cd ../build-Release && make distclean && make clean
   cd ../build-Debug && make distclean && make clean
   cd ..
   ./bootstrap force release
   ./bootstrap force debug
boost release and debug build for Windows
Open a VS2015 x64 Native Tools Command Prompt.

EITHER: for new installs, you have to run bootstrap.bat first, it will build b2; OR: for reruns, remove boost dirs: [bin.v2, stage]. Then build 64-bit:

cd "....\boost_1_59_0"
b2 toolset=msvc variant=release,debug link=static address-model=64
rem trying to avoid excessive options, assuming I don't need these: threading=multi
(old stuff)
      --toolset=msvc-14.0 address-model=64 --build-type=complete --stagedir=windows_lib\x64 stage
      Now open VS2013 x86 Native Tools Command Prompt and build 32-bit:
      cd "C:\Michael's Data\development\sixth_column\boost_1_55_0"
      bjam --toolset=msvc-12.0 address-model=32 --build-type=complete --stagedir=windows_lib\x86 stage
boost regex (very similar to c++11, with LESS BUGS and MORE POWER)
boost regex does everything you could need, take your time to get it right
  • regex_search will work hard to find substrings, while regex_match requires your regex to match entire input
  • smatch is a great way to get back a bunch of iterators to the results

example:

   virtual bool url_upgrade_any_old_semver(string& url)
   {
     // Perform a regex to update the embedded version if needed.
     // We have to see if (1) we have a semver and (2) it is not the current semver.
     // Only then do we take action.
     boost::regex regex("/([v0-9.]+?)/(.*)");
     boost::smatch sm_res;
     if (boost::regex_match(url,sm_res,regex,boost::match_default))
     {
       string incoming_semver(sm_res[1].first,sm_res[1].second);
       if (incoming_semver != semanticVersion())
       {
         url = string("/")+semanticVersion()+"/"+string(sm_res[2].first,sm_res[2].second);
         return true;
       }      
     }
     return false;
   }

results:

     original string:    string(sm_res[0].first,sm_res[0].second);
     first group:        string(sm_res[1].first,sm_res[1].second);
     second group:       string(sm_res[2].first,sm_res[2].second);
     etc.
C++ libraries
String escape formatting across different languages and systems
  • c++ to JSON: always use nlohmann::json j.dump() to encode, to ensure strings are properly escaped
  • JSON to c++: always use nlohmann::json j.parse() "
  • c++ to Javascript: use raw_to_Javascript() to properly escape
  • c++ to sqlite: use SqliteLocalModel::safestr(), which uses double_doublequotes(str)

Postgres

Simple-Web-Server

Robot Operating System

C++ https libraries

Configure Qt development on Windows + Mac + linux

Build the TagLib library with Visual Studio 2013

C/C++ building/linking
gcc makefile pain
  • I went through a LOT of pain to determine that gcc requires libraries to be listed AFTER the source and output parameters
  • set LD_LIBRARY_PATH to point to your libs if they are not in system location
  • Make sure Makefile uses TABS NOT SPACES. it's a FACT OF LIFE. hate it if you want. plenty more things to hate about linux/C as well.
gcc install multiple versions in ubuntu (4 and 5 in wily, eg)
My code will not compile with gcc 5, the version provided with Ubuntu wily.

It gives warnings like this:

/home/m/development/boost_1_59_0/boost/smart_ptr/shared_ptr.hpp:547:34: warning: ‘template<class> class std::auto_ptr’ is deprecated [-Wdeprecated-declarations]

and outright errors like this:

depbase=`echo AtServer.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
g++ -DPACKAGE_NAME=\"at_server\" -DPACKAGE_TARNAME=\"at_server\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"at_server\ 1.0\" -DPACKAGE_BUGREPORT=\"m@abettersoftware.com\" -DPACKAGE_URL=\"\" -DPACKAGE=\"at_server\" -DVERSION=\"1.0\" -I. -I../../src  -I/home/m/development/Reusable/c++ -I/home/m/development/Reusable/c++/sqlite -std=c++11 -I/home/m/development/boost_1_59_0  -ggdb3 -O0 -std=c++11 -MT AtServer.o -MD -MP -MF $depbase.Tpo -c -o AtServer.o ../../src/AtServer.cpp &&\
mv -f $depbase.Tpo $depbase.Po
In file included from /usr/include/c++/5/bits/stl_algo.h:60:0,
                from /usr/include/c++/5/algorithm:62,
                from /usr/include/c++/5/ext/slist:47,
                from /home/m/development/boost_1_59_0/boost/algorithm/string/std/slist_traits.hpp:16,
                from /home/m/development/boost_1_59_0/boost/algorithm/string/std_containers_traits.hpp:23,
                from /home/m/development/boost_1_59_0/boost/algorithm/string.hpp:18,
                from /home/m/development/Reusable/c++/utilities.hpp:4,
                from ../../src/MemoryModel.hpp:11,
                from ../../src/SqliteLocalModel.hpp:13,
                from ../../src/AtServer.cpp:70:
/usr/include/c++/5/bits/algorithmfwd.h:573:13: error: initializer provided for function
    noexcept(__and_<is_nothrow_move_constructible<_Tp>,
            ^
/usr/include/c++/5/bits/algorithmfwd.h:582:13: error: initializer provided for function
    noexcept(noexcept(swap(*__a, *__b)))

You can set up the update-alternatives tool to switch out the symlinks:

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 20 --slave /usr/bin/g++ g++ /usr/bin/g++-5

BUT that seems BAD to me to switch out the compiler used by the SYSTEM. Instead, we should specify the compiler required by the PROJECT. This is supposed to do it, but it still uses /usr/include/c++/5, which seems wrong, and gives me errors:

CC=gcc-4.9
git
Debugging
Chrome capture large JSON variable
This is just pointlessly bizarre:
  • hit a breakpoint in the chrome debugger
  • right-click a variable and say "copy to global variable" (console will show name, typically "temp1")
  • push the variable to the clipboard by typing this in the console:
copy(temp1)
Visual Studio Code capture large string variable
While debugging, you can use the Debug Console to print memory, including the content of strings that are clipped by default in the variables and watch windows.
View > Open View > Debug Console

From there, send gdb a command to print memory – 300 characters of a string in this example:

-exec x/300sb Query.c_str()
Qt Creator conditional breakpoint
To break on line number with string condition:
  • Set the breakpoint on the line as usual: F9 or click in left border
  • In the breakpoints list, rclick Edit
  • Add a condition; to break on a string value, THIS WORKS:
         ((int)strcmp(strSymbol.c_str(), "GMO")) == 0
     
  • BUT THERE IS A PRICE TO PAY. qt/gcc won't continue properly, unless you sset ANOTHER common breakpoint, hit it, and resume from that. It appears gcc continues (we hit the breakpoint after all), but qt does no logging or debugging, bah
Qt Creator and linux debug libraries
  • I was able to build lws in debug, install in /usr/local, then I could step right into its code
m@case:~/development/causam/git/np/nop-bigress-client-c$ ./build_lws_debug
 Install the project...
 -- Install configuration: "DEBUG"
 -- Installing: /usr/local/lib/pkgconfig/libwebsockets.pc
 -- Installing: /usr/local/lib/libwebsockets.a
 -- Up-to-date: /usr/local/include/libwebsockets.h
 (etc)
  • Signals: At some point I needed to force gdb to pass SIGINT, SIGPIPE... signals to the app instead of gdb, after ssl kept causing Qt to pop a message and break in the debugger. To do so:
       Qt Creator->Options->Debugger->
           [ ] Show a message box when receiving a signal
           Debugger Helper Customization: handle SIGPIPE pass nostop noprint
  • You should be able to install debug builds of libraries into /usr/local (as mentioned above). Older notes if you can't get that going: To debug into an included library that is not installed system-wide:
use Tools->Options->General->Source Paths Map
 source: /home/m/development/causam/git/nop-bigress-client-c/cmake-debug
   that's the COMPLETE PATH TO DEBUG EXE
 target: /home/m/development/causam/git/libwebsockets-master/lib
   at first i thought this would work: /usr/local/lib
   but i did a static build so i needed teh actual exe!  COOL
QT Creator valgrind EASY
Valgrind is easy to use. Just find debugger pane top-left dropdown, switch mode from Debugger to Memcheck, and restart debugger.
c/c++ gdb debugging
(gdb) help break
Set breakpoint at specified line or function.
Argument may be line number, function name, or "*" and an address.
If line number is specified, break at start of code for that line.
If function is specified, break at start of code for that function.
If an address is specified, break at that exact address.
With no arg, uses current execution address of selected stack frame.
This is useful for breaking on return to a stack frame.

Multiple breakpoints at one place are permitted, and useful if conditional.    

Do "help breakpoints" for info on other commands dealing with breakpoints.
ddd gives you a front end. I need to use it more, compare to other options
C
Library handling in linux
gcc library basics (good to know for autotools and CMake too)
  • The gcc -l command line option specifies a specific library NAME. It will add the rest to give you (eg) lib[NAME].a
  • Specify the library search path with the -L option. Things that you as a user install seem to default to /usr/local/lib
  • This will link myprogram with the static library libfoo.a in the folder /home/me/foo/lib.
       gcc -o myprogram -lfoo -L/home/me/foo/lib myprogram.c

Tools to check library dependencies:

  • ldd [executable]
  • objdump -p /path/to/program | grep NEEDED
  • objdump -x [object.so]
  • readelf -d [exe]
  • sudo pldd <PID>
  • sudo pmap <PID>

Cross Compiling

C - Create a portable command line C project in Visual Studio
   Visual Studio: File -> New -> project
   Visual C++ -> Win32 -> Win32 Console Application
   name: oms_with_emap
   next -> click OFF precompiled header checkbox (even tho it didn't seem to respect it)
   you'll get a _tmain(..., TCHAR*...)
   change it to main(..., char*...)
   change the project to explicitly say "Not using precompiled header"
   remove the f'in stdafx.h
   recompile!  should be clean
   vs will recognize C files and compile accordingly
Javascript
Node.js
React
Vite
JSON
HTML
CSS
SQL
Count records within a range
This groups records into ranges, sorts by them, and gives a count, sweet:
   select count(*), id/1000000 as groupid from AccountHistory group by groupid;

postgres - sqlite - mysql - SQL Server - Robo 3T - DBeaver - pgadmin4

Meteor
Android
Arduino
Raspberry Pi
iOS
.NET Core
Java
Kotlin
Maven
Scala
Python
Go
PHP
php debugging
Tail these:
tail -f /var/log/apache2/sitelogs/thedigitalage.org/ssl_error_log
tail -f /var/log/ampache-tda/ampache.(today).log
This leads to too much noise, not needed...
emacs /etc/php/apache2-php5.3/php.ini
  display_errors = On
/etc/init.d/apache restart
Bash basics but please prefer node or python :-)
misc
SVN repo move across servers
  • use a tool like VisualSVN to stop SVN server
  • copy the repo's entire svn server directory, eg: c:\svn\Software
  • copy it into the new server under a unique name, eg: c:\svn\NewSoftware
  • use a tool like VisualSVN to restart SVN server

You should now have the new code, accessible as usual from svn. NOTE from Tom: If you move code around, make sure you tag it, then copy the tag, to preserve history (as opposed to directly moving the folder). Weird but true.

SQL Server 2008+ proper upsert using MERGE
       -- We need an "upsert": if record exists, update it, otherwise insert.
       -- There are several options to do that.
       -- Trying to do it correctly means...
       --		1) use a lock or transaction to make the upsert atomic
       --		2) use the best-available operation to maximize performance
       -- SQL Server 2008 has MERGE which may be slightly more efficient than 
       -- separate check && (insert||update) steps.  And we can do it with
       -- a single lock instead of a full transaction (which may be better?).
       -- It's messy to code up though since three blocks of fields must be specified.  
       -- Cest la vie.
       MERGE [dbo].[FACT_DCSR_RemPeriodMonthlyReport] WITH (HOLDLOCK) AS rpmr
       USING (SELECT @ID AS ID) AS new_foo
             ON rpmr.ID = new_foo.ID



               @last_months_year as DCSRYear,
               @last_month as DCSRMonth,
               @last_month_name as MonthName,
               Device_Type_ID,




       WHEN MATCHED THEN
           UPDATE
                   SET f.UpdateSpid = @@SPID, 
                   UpdateTime = SYSDATETIME() 
       WHEN NOT MATCHED THEN
           INSERT
             (
                   ID, 
                   InsertSpid, 
                   InsertTime
             )
           VALUES
             (
                   new_foo.ID, 
                   @@SPID, 
                   SYSDATETIME()
             );
Web Services
Firefox Addon development
c++ Create a portable autotools C++ project in linux
(OLD, USE CMAKE NOW)

For anything serious, it's best to clone an existing project's skeleton.

  • main.cpp
  • MVC code
  • nix/copy_from folder
  • make sure .bashrc is configured for boost
  • nix$ bootstrap_autotools_project.sh force release debug
  • set up eclipse according to screenshots in Eclipse
c++ Create a portable C++ project in Visual Studio
(OLD, USE CMAKE NOW)

If you don't have existing code, it's probably best to create a project_name.cpp file with your main() function.

int main( int argc, char * argv[] )
{ 
    return 0;
}

Then in Visual Studio...

File->New->Project from existing code
C++
(then use mostly defaults on this page, once you provide file location and project name)
Project file location:  <base>\project_name
Project name: project_name
[x] Add files from these folders
   Add subs  
   [x]        <base>\project_name
NEXT
Use Visual Studio
  Console application project
  No ATL, MFC, CLR
NEXT
NEXT
FINISH

Then add Reusable and boost include and lib paths: Example if you built boost according to notes below:

Project->Properties->All Configurations->Configuration Properties->VC++ Directories->Include Directories->
   $(VC_IncludePath);$(WindowsSDK_IncludePath);..\..\..\Reusable\c++
INCLUDE: C:\Software Development\boost_1_53_0
LIB: C:\Software Development\boost_1_53_0\stage\lib