Development reference: Difference between revisions
No edit summary |
No edit summary |
||
(43 intermediate revisions by the same user not shown) | |||
Line 16: | Line 16: | ||
{| class="wikitable" | {| class="wikitable" | ||
! [http://www.aosabook.org/en/index.html Architecture of major open source apps] | ! [http://www.aosabook.org/en/index.html Architecture of major open source apps] | ||
|} | |||
{| class="wikitable" | |||
! [[Security]] | |||
|} | |} | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[Model View Controller]] | ! [[Model View Controller]] | ||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! nosql enhances, not replaces, SQL | |||
|- | |||
| Not all data should be denormalized, and not all data should be normalized. The optimal mix considers the extent of the data. | |||
* Precise schemas are good when not overdone | |||
* When a container has an array with a large number of elements, it should be normalized | |||
* Sparse data and heterogeneous data are the best candidates for denormalization | |||
Postgres with JSON allows an elegant combination of nosql and SQL. | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
Line 105: | Line 119: | ||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! c++ in-memory storage of "major" objects | ||
|- | |||
| | |||
OBSERVATION ONE | |||
Consider An Important Qt Design: QObjects cannot normally be copied | |||
their copy constructors and assignment operators are private | |||
why? A Qt Object... | |||
might have a unique QObject::objectName(). If we copy a Qt Object, what name should we give the copy? | |||
has a location in an object hierarchy. If we copy a Qt Object, where should the copy be located? | |||
can be connected to other Qt Objects to emit signals to them or to receive signals emitted by them. If we copy a Qt Object, how should we transfer these connections to the copy? | |||
can have new properties added to it at runtime that are not declared in the C++ class. If we copy a Qt Object, should the copy include the properties that were added to the original? | |||
in other words, a QObject is a pretty serious object that has the ability to be tied to other objects and resources in ways that make copying dangerous | |||
isn't this true of all serious objects? pretty much | |||
OBSERVATION TWO | |||
if you have a vector of objects, you often want to track them individually outside the vector | |||
if you use a vector of pointers, you can move the object around much more cheaply, and not worry about costly large vector reallocations | |||
a vector of objects (not pointers) only makes sense if the number of objects is initially known and does not change over time | |||
OBSERVATION THREE | |||
STL vectors can store your pointers, iterate thru them, etc. | |||
for a vector of any substantial size, you want to keep objects sorted so you can find them quickly | |||
that's what my sorted_vector class is for; it simply bolts vector together with sort calls and a b_sorted status | |||
following STL practices, to get sorting, you have to provide operator< for whatever is in your vector | |||
BUT... you are not allowed to do operator<(const MyObjectPtr* right) because it would require a reference to a pointer which is not allowed | |||
BUT... you can provide a FUNCTOR to do the job, then provide it when sorting/searching | |||
a functor is basically a structure with a bool operator()(const MyObjectPtr* left, const MyObjectPtr* right) | |||
OBSERVATION FOUR | |||
unordered_set works even better when combining frequent CRUD with frequent lookups | |||
SUMMARY | |||
Dealing with tons of objects is par for the course in any significant app. | |||
Finding a needle in the haystack of those objects is also standard fare. | |||
Having multiple indices into those objects is also essential. | |||
Using unordered_set with object pointers and is very powerful. | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! c++ stl reverse iterator skeleton | |||
|- | |- | ||
| | | From [http://www.sgi.com/tech/stl/ReverseIterator.html SGI]... | ||
reverse_iterator rfirst(V.end()); | |||
reverse_iterator rlast(V.begin()); | |||
while (rfirst != rlast) | |||
{ | |||
cout << *rfirst << endl; | |||
... | |||
rfirst++; | |||
} | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! c++ stl reading a binary file into a string | ||
|- | |- | ||
| | | | ||
std::ifstream in("my.zip",std::ios::binary); | |||
if (!in) | |||
{ | |||
std::cout << "problem with file open" << std::endl; | |||
return 0; | |||
} | |||
in.seekg(0,std::ios::end); | |||
unsigned long length = in.tellg(); | |||
in.seekg(0,std::ios::beg); | |||
string str(length,0); | |||
std::copy( | |||
std::istreambuf_iterator< char >(in) , | |||
std::istreambuf_iterator< char >() , | |||
str.begin() | |||
); | |||
For more, see [[Reading a binary file|c++ stl reading a binary file]] | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! C/C++ best-in-class tool selection | |||
|- | |||
| I need to have easy setup of debug-level tool support for portable C++11 code. And I need to decide and stick to it to be efficient. | |||
* '''Compiler selection''' | |||
** linux and mac: gcc | |||
** windows: Visual Studio | |||
* '''IDE selection''' | |||
** linux and mac: Qt Creator | |||
** windows: Qt Creator (OR Visual Studio OR eclipse?) | |||
* '''Debugger selection''' | |||
** linux and mac: Qt Creator | |||
** windows: Qt Creator (OR Visual Studio OR eclipse?) | |||
|} | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! c++11 | |||
|- | |||
| | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! c++11 include base constructor (etc) in derived class | ! c++11 include base constructor (etc) in derived class | ||
Line 393: | Line 479: | ||
} STRING_PREF_INDEX; | } STRING_PREF_INDEX; | ||
|} | |} | ||
|} | |||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! boost | |||
|- | |||
| | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! boost release and debug build for linux | |||
|- | |||
| | |||
# download latest boost, eg: boost_1_59_0 | |||
m@wallee:~/development$ 7z x boost_1_59_0.7z | |||
rm boost && ln -s boost_1_59 boost | |||
cd boost | |||
build_boost_release_and_debug.sh | |||
# IMPORTANT: then patch .bashrc as instructed | |||
# cmake-###/clean.sh and cmake-###/build.sh are customized to match | |||
# (and older eclipse and server/nix/bootstrap.sh) | |||
To upgrade a CMake project: | |||
cd cmake-###/ | |||
./clean.sh | |||
./build.sh | |||
To upgrade an older autotools project: | |||
cd nix | |||
make distclean # removes nasty .deps folders that link to old boost if you let them | |||
make clean # removes .o files etc | |||
cd ../build-Release && make distclean && make clean | |||
cd ../build-Debug && make distclean && make clean | |||
cd .. | |||
./bootstrap force release | |||
./bootstrap force debug | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! boost release and debug build for Windows | ||
|- | |- | ||
| | | Open a VS2015 x64 Native Tools Command Prompt. | ||
EITHER: for new installs, you have to run bootstrap.bat first, it will build b2; | |||
OR: for reruns, remove boost dirs: [bin.v2, stage]. | |||
Then build 64-bit: | |||
cd "....\boost_1_59_0" | |||
b2 toolset=msvc variant=release,debug link=static address-model=64 | |||
rem trying to avoid excessive options, assuming I don't need these: threading=multi | |||
(old stuff) | |||
--toolset=msvc-14.0 address-model=64 --build-type=complete --stagedir=windows_lib\x64 stage | |||
Now open VS2013 x86 Native Tools Command Prompt and build 32-bit: | |||
cd "C:\Michael's Data\development\sixth_column\boost_1_55_0" | |||
bjam --toolset=msvc-12.0 address-model=32 --build-type=complete --stagedir=windows_lib\x86 stage | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! c++ | ! boost regex (very similar to c++11, with LESS BUGS and MORE POWER) | ||
|- | |- | ||
| | | boost regex does everything you could need, take your time to get it right | ||
* regex_search will work hard to find substrings, while regex_match requires your regex to match entire input | |||
* smatch is a great way to get back a bunch of iterators to the results | |||
example: | |||
virtual bool url_upgrade_any_old_semver(string& url) | |||
{ | { | ||
// Perform a regex to update the embedded version if needed. | |||
// We have to see if (1) we have a semver and (2) it is not the current semver. | |||
// Only then do we take action. | |||
boost::regex regex("/([v0-9.]+?)/(.*)"); | |||
boost::smatch sm_res; | |||
if (boost::regex_match(url,sm_res,regex,boost::match_default)) | |||
{ | |||
string incoming_semver(sm_res[1].first,sm_res[1].second); | |||
if (incoming_semver != semanticVersion()) | |||
{ | |||
url = string("/")+semanticVersion()+"/"+string(sm_res[2].first,sm_res[2].second); | |||
return true; | |||
} | |||
} | |||
return false; | |||
} | } | ||
results: | |||
original string: string(sm_res[0].first,sm_res[0].second); | |||
first group: string(sm_res[1].first,sm_res[1].second); | |||
second group: string(sm_res[2].first,sm_res[2].second); | |||
etc. | |||
|} | |||
|} | |||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! C++ libraries | ||
|- | |||
| | | | ||
| | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! String escape formatting across different languages and systems | ||
|- | |- | ||
| | | | ||
* c++ to JSON: always use nlohmann::json j.dump() to encode, to ensure strings are properly escaped | |||
* JSON to c++: always use nlohmann::json j.parse() " | |||
* c++ to Javascript: use raw_to_Javascript() to properly escape | |||
* c++ to sqlite: use SqliteLocalModel::safestr(), which uses double_doublequotes(str) | |||
|} | |||
[[Postgres]] | |||
[[Simple-Web-Server]] | |||
[[C++ https libraries]] | |||
[[Configure Qt development on Windows + Mac + linux]] | |||
[[Build the TagLib library with Visual Studio 2013]] | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! C | ! C/C++ building/linking | ||
|- | |- | ||
| | | | ||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! gcc makefile pain | |||
|- | |||
| | |||
* I went through a LOT of pain to determine that gcc requires libraries to be listed AFTER the source and output parameters | |||
* set LD_LIBRARY_PATH to point to your libs if they are not in system location | |||
* Make sure Makefile uses TABS NOT SPACES. it's a FACT OF LIFE. hate it if you want. plenty more things to hate about linux/C as well. | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! gcc install multiple versions in ubuntu (4 and 5 in wily, eg) | ! gcc install multiple versions in ubuntu (4 and 5 in wily, eg) | ||
Line 567: | Line 656: | ||
--> | --> | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! C++ | ! C/C++ debugging | ||
|- | |- | ||
| | | | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Qt Creator conditional breakpoint | ||
|- | |- | ||
| | | To break on line number with string condition: | ||
* | * Set the breakpoint on the line as usual: F9 or click in left border | ||
* | * In the breakpoints list, rclick Edit | ||
* | * Add a condition; to break on a string value, THIS WORKS: | ||
((int)strcmp(strSymbol.c_str(), "GMO")) == 0 | |||
* BUT THERE IS A PRICE TO PAY. qt/gcc won't continue properly, unless you sset ANOTHER common breakpoint, hit it, and resume from that. It appears gcc continues (we hit the breakpoint after all), but qt does no logging or debugging, bah | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! Qt Creator and linux debug libraries | |||
! Qt Creator | |||
|- | |- | ||
| | | | ||
Line 617: | Line 683: | ||
-- Up-to-date: /usr/local/include/libwebsockets.h | -- Up-to-date: /usr/local/include/libwebsockets.h | ||
(etc) | (etc) | ||
* Signals: At some point I needed to force gdb to pass SIGINT, SIGPIPE... signals to the app instead of gdb, after ssl kept causing Qt to pop a message and break in the debugger. To do so: | * Signals: At some point I needed to force gdb to pass SIGINT, SIGPIPE... signals to the app instead of gdb, after ssl kept causing Qt to pop a message and break in the debugger. To do so: | ||
Qt Creator->Options->Debugger-> | Qt Creator->Options->Debugger-> | ||
Line 629: | Line 694: | ||
at first i thought this would work: /usr/local/lib | at first i thought this would work: /usr/local/lib | ||
but i did a static build so i needed teh actual exe! COOL | but i did a static build so i needed teh actual exe! COOL | ||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! QT Creator valgrind EASY | |||
|- | |||
| Valgrind is easy to use. Just find debugger pane top-left dropdown, switch mode from Debugger to Memcheck, and restart debugger. | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! c/c++ gdb debugging | |||
|- | |||
| | |||
(gdb) help break | |||
Set breakpoint at specified line or function. | |||
Argument may be line number, function name, or "*" and an address. | |||
If line number is specified, break at start of code for that line. | |||
If function is specified, break at start of code for that function. | |||
If an address is specified, break at that exact address. | |||
With no arg, uses current execution address of selected stack frame. | |||
This is useful for breaking on return to a stack frame. | |||
Multiple breakpoints at one place are permitted, and useful if conditional. | |||
Do "help breakpoints" for info on other commands dealing with breakpoints. | |||
|- | |||
| ddd gives you a front end. I need to use it more, compare to other options | |||
|} | |} | ||
|} | |} | ||
Line 661: | Line 750: | ||
[[Cross Compiling]] | [[Cross Compiling]] | ||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! C - Create a portable command line C project in Visual Studio | |||
|- | |||
| | |||
Visual Studio: File -> New -> project | |||
Visual C++ -> Win32 -> Win32 Console Application | |||
name: oms_with_emap | |||
next -> click OFF precompiled header checkbox (even tho it didn't seem to respect it) | |||
you'll get a _tmain(..., TCHAR*...) | |||
change it to main(..., char*...) | |||
change the project to explicitly say "Not using precompiled header" | |||
remove the f'in stdafx.h | |||
recompile! should be clean | |||
vs will recognize C files and compile accordingly | |||
|} | |||
|} | |} | ||
Line 670: | Line 775: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[Javascript]] | ||
|}<!-- | |||
< | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! RESTFul http | ||
|- | |- | ||
| | | | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Verbs | ||
|- | |- | ||
| | | | ||
* | * get - to make requests that are simple -response may be complex | ||
* | * post - to push a form or JSON to the server | ||
|} | |} | ||
|} | |} | ||
Line 840: | Line 804: | ||
--> | --> | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! css | ||
|- | |- | ||
| | | | ||
/* class="first second" */ | |||
.first.second {} | |||
/* class="first" OR class="second" */ | |||
.first, .second {} | |||
* | /* class="first second", or class="second", or class="third second", or class="second third" */ | ||
.second {} | |||
/* apply to any .child at any depth under .parent */ | |||
.parent .child {} | |||
/* apply to .child if it is DIRECTLY under .parent */ | |||
.parent > .child {} | |||
* | |||
|} | |} | ||
<!-- | <!-- | ||
Line 917: | Line 827: | ||
--> | --> | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! SQL | ||
|- | |- | ||
| | | | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Sqlite timestamp-to-readable-date query | ||
|- | |||
| | |||
select quote, timestamp, strftime('%Y-%m-%d %H:%M:%S', datetime(timestamp, 'unixepoch')) from StockQuotes as s where s.symbol="TSLA" order by timestamp; | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! Sqlite copy row (pita) | |||
|- | |- | ||
| | | Here's how you can clone a row in sqlite: | ||
* | CREATE TEMPORARY TABLE tmp AS SELECT * FROM StockPicks where id=1; | ||
UPDATE tmp SET id = NULL; | |||
INSERT INTO StockPicks SELECT * FROM tmp; | |||
DROP TABLE tmp; | |||
* | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Count records within a range | ||
|- | |- | ||
| | | This groups records into ranges, sorts by them, and gives a count, sweet: | ||
* | select count(*), id/1000000 as groupid from AccountHistory group by groupid; | ||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Sqlite table size analysis | ||
|- | |- | ||
| | | | ||
sqlite db bloat found with sqlite3_analyzer, here: | |||
http://www.renedohmen.nl/blog/2014/05/determine-sqlite-table-size-on-disk/ | |||
http://www.sqlite.org/download.html | |||
(put it under viper ~/apps) | |||
ran it like this: | |||
cd ~/development/thedigitalage/AbetterTrader/server/db_archive | |||
~/apps/sqlite-tools/sqlite-tools-linux-x86-3220000/sqlite3_analyzer at_server_test.sqlite__2018-03-14_EOD__.sqlite | |||
|} | |} | ||
|} | |} | ||
Line 983: | Line 871: | ||
--> | --> | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Android]] | ||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Java]] | ||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[Kotlin]] | |||
|} | |} | ||
<!-- | <!-- | ||
Line 1,061: | Line 901: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[Maven]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,084: | Line 911: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[Scala]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,103: | Line 922: | ||
--> | --> | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Python]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,113: | Line 932: | ||
--> | --> | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Go]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,148: | Line 967: | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[git]] | ! [[git]] | ||
|} | |||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[Bash basics]] but please prefer node or python :-) | |||
|} | |} | ||
<!-- | <!-- |
Revision as of 11:10, 24 October 2018
Design, programming and version control.
Patterns | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
C++ | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
|
c++11 | |||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
boost | ||||||
---|---|---|---|---|---|---|
|
C++ libraries | ||
---|---|---|
|
C/C++ building/linking | ||||
---|---|---|---|---|
|
C/C++ debugging | |||||||||
---|---|---|---|---|---|---|---|---|---|
|
C | ||||
---|---|---|---|---|
|
Javascript |
---|
RESTFul http | ||
---|---|---|
|
css |
---|
/* class="first second" */ .first.second {} /* class="first" OR class="second" */ .first, .second {} /* class="first second", or class="second", or class="third second", or class="second third" */ .second {} /* apply to any .child at any depth under .parent */ .parent .child {} /* apply to .child if it is DIRECTLY under .parent */ .parent > .child {} |
SQL | ||||||||
---|---|---|---|---|---|---|---|---|
|
Android |
---|
Java |
---|
Kotlin |
---|
Maven |
---|
Scala |
---|
Python |
---|
Go |
---|
PHP | |||
---|---|---|---|
|
git |
---|
Bash basics but please prefer node or python :-) |
---|
misc | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|